github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-11-06 21:58:01 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph

ALSA compress / ASoC compress / SOF: Compressed audio support with IPC4

Browse Source 
Peter Ujfalusi <peter.ujfalusi@linux.intel.com> says:

This series adds compressed audio support for SOF with IPC4 firmware.
Supported formats are PCM, MP3, AAC and VORBIS, the feature set is
PAUSE_PUSH/RELEASE, DRAIN and position reporting.

Patches 1-5 are prerequisites in the ALSA compress core and in
soc-compress.c.

These are core fixes rather than driver level workarounds on purpose.
The other users of compressed DPCM have DSP internal ports on their
BEs, driven by code they own. An Intel/SOF BE ends at a third party
CODEC driver which is shared with the non DPCM case and has no reason
to know that the FE is compressed. That set of CODECs is open ended,
so teaching each of them to cope with a compressed FE does not scale.
The core has to hand a BE the same substream state whether the FE is
a PCM or a compressed one.

Two of these patches change behaviour for the existing users of
compressed DPCM, so they would need a look from Qualcomm and NXP:

 - "ASoC: soc-compress: Implement trigger FE-BE sequencing as with
   normal PCMs" makes the FE/BE trigger order follow
   dai_link->trigger[], the way dpcm_fe_dai_do_trigger() does it.
   Neither sound/soc/qcom/ nor sound/soc/fsl/ sets it, so both default
   to SND_SOC_DPCM_TRIGGER_PRE: START keeps the current order, but
   STOP, SUSPEND and PAUSE_PUSH now trigger the BE before the FE.

 - "ASoC: soc-compress: Stop running dpcm on free" sends a STOP trigger
   from soc_compr_free_fe() if the DPCM is still in START state, which
   is the case when the last trigger the stream received was a DRAIN.

Patches 6-20 implement the IPC4 compressed support, patch 21 is a
Kconfig cleanup. The last two patches enable the feature, in Kconfig
for TGL and later and in the topology parser, so the compressed device
type only becomes usable once the code behind it is in place.

Testing:

 - IPC4: Arrow Lake, compressed playback on speaker and headphone
   (cs42l43), MP3, AAC and VORBIS.
 - IPC3: tested by NXP. This does not cover the STOP on free path
   above, as they do not implement DRAIN.

Link: https://patch.msgid.link/20261007084955.1256-1-peter.ujfalusi@linux.intel.com
This commit is contained in:
Mark Brown 2026-10-07 13:16:50 +02:00
commit fed620a78a
513 changed files with 8702 additions and 3150 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.mailmap
View File

@ -286,6 +286,8 @@ Ed L. Cashin <ecashin@coraid.com>
Elliot Berman <quic_eberman@quicinc.com> <eberman@codeaurora.org>
Enric Balletbo i Serra <eballetbo@kernel.org> <enric.balletbo@collabora.com>
Enric Balletbo i Serra <eballetbo@kernel.org> <eballetbo@iseebcn.com>
Eric Dumazet <edumazet@kernel.org> <edumazet@google.com>
Eric Dumazet <edumazet@kernel.org> <eric.dumazet@gmail.com>
Erik Kaneda <erik.kaneda@intel.com> <erik.schmauss@intel.com>
Ethan Carter Edwards <ethan@ethancedwards.com> Ethan Edwards <ethancarteredwards@gmail.com>
Eugen Hristev <ehristev@kernel.org> <eugen.hristev@microchip.com>

2
Documentation/arch/arm64/silicon-errata.rst
View File

@ -174,6 +174,8 @@ stable kernels.
+----------------+-----------------+-----------------+-----------------------------+
| ARM | Cortex-A725 | #3456106 | ARM64_ERRATUM_3194386 |
+----------------+-----------------+-----------------+-----------------------------+
| ARM | Cortex-A725 | #3821522 | ARM64_ERRATUM_3821522 |
+----------------+-----------------+-----------------+-----------------------------+
| ARM | Cortex-X1 | #1502854 | N/A |
+----------------+-----------------+-----------------+-----------------------------+
| ARM | Cortex-X1 | #3324344 | ARM64_ERRATUM_3194386 |

18
Documentation/devicetree/bindings/iio/adc/rockchip-saradc.yaml
View File

@ -12,19 +12,19 @@ maintainers:
properties:
compatible:
oneOf:
- const: rockchip,saradc
- const: rockchip,rk3066-tsadc
- const: rockchip,rk3399-saradc
- const: rockchip,rk3528-saradc
- enum:
- rockchip,saradc
- rockchip,rk3066-tsadc
- rockchip,rk3399-saradc
- rockchip,rk3528-saradc
- rockchip,rk3562-saradc
- rockchip,rk3588-saradc
- rockchip,rv1106-saradc
- items:
- const: rockchip,rk3506-saradc
- const: rockchip,rk3528-saradc
- const: rockchip,rk3562-saradc
- const: rockchip,rk3588-saradc
- items:
- enum:
- rockchip,rk3576-saradc
- rockchip,rv1106-saradc
- const: rockchip,rk3576-saradc
- const: rockchip,rk3588-saradc
- items:
- enum:

26
Documentation/filesystems/netfs_library.rst
View File

@ -451,6 +451,32 @@ one.
The inode should be marked ``NETFS_ICTX_SINGLE_NO_UPLOAD`` if this API is to be
used. The writeback function requires the buffer to be of ITER_FOLIOQ type.
Clearing Stale Post-EOF Pagecache
---------------------------------
When a file is extended, data left in the pagecache past the old EOF by a write
through an mmap must not be exposed as file content. Netfslib clears this on
its own write paths, and exports a helper so a filesystem can do the same from
a resize path (truncate, setattr, fallocate and the like) that holds the
inode's ``i_rwsem`` exclusively across the whole resize::
void netfs_clear_stale_post_isize(struct inode *inode, uoff_t from,
uoff_t to);
This zeroes any such data within the ``[from, to)`` hole to be made, where
@from is the old EOF and @to is the new one, and the caller must have already
updated ``i_size`` to @to before calling it. Only the folio straddling @from
can hold data written past the EOF through an mmap, as pages wholly beyond the
EOF can't be faulted in, so the zeroing is limited to that folio. The folio is
zeroed rather than dropped so that a concurrent extending write can't lose
data.
This overlaps with ``pagecache_isize_extended()`` but can't reuse it: that
helper is keyed on a sub-page block size and is a no-op when the block size is
``>= PAGE_SIZE`` (as on network filesystems), and it doesn't wait for
writeback. As both address the same problem, a change to one should probably
be reflected in the other to keep them in sync.
High-Level VM API
==================

85
MAINTAINERS
View File

@ -5151,6 +5151,7 @@ L: netdev@vger.kernel.org
L: openwrt-devel@lists.openwrt.org (subscribers-only)
S: Supported
F: Documentation/devicetree/bindings/net/dsa/brcm,b53.yaml
F: Documentation/networking/dsa/b53.rst
F: drivers/net/dsa/b53/*
F: drivers/net/dsa/bcm_sf2*
F: include/linux/dsa/brcm.h
@ -17224,31 +17225,6 @@ F: include/trace/events/ksm.h
F: mm/ksm.c
F: mm/mm_slot.h
MEMORY MANAGEMENT - MEMORY POLICY AND MIGRATION
M: Andrew Morton <akpm@linux-foundation.org>
M: David Hildenbrand <david@kernel.org>
R: Zi Yan <ziy@nvidia.com>
R: Matthew Brost <matthew.brost@intel.com>
R: Joshua Hahn <joshua.hahnjy@gmail.com>
R: Rakie Kim <rakie.kim@sk.com>
R: Byungchul Park <byungchul@sk.com>
R: Gregory Price <gourry@gourry.net>
R: Ying Huang <ying.huang@linux.alibaba.com>
R: Alistair Popple <apopple@nvidia.com>
L: linux-mm@kvack.org
S: Maintained
W: http://www.linux-mm.org
T: git git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
F: Documentation/ABI/testing/sysfs-kernel-mm-mempolicy
F: Documentation/ABI/testing/sysfs-kernel-mm-mempolicy-weighted-interleave
F: include/linux/mempolicy.h
F: include/uapi/linux/mempolicy.h
F: include/linux/migrate.h
F: include/linux/migrate_mode.h
F: mm/mempolicy.c
F: mm/migrate.c
F: mm/migrate_device.c
MEMORY MANAGEMENT - MGLRU (MULTI-GEN LRU)
M: Andrew Morton <akpm@linux-foundation.org>
R: Kairui Song <kasong@tencent.com>
@ -17271,6 +17247,25 @@ F: include/linux/mmzone.h
F: mm/vmscan.c
F: mm/workingset.c
MEMORY MANAGEMENT - MIGRATION
M: Andrew Morton <akpm@linux-foundation.org>
M: David Hildenbrand <david@kernel.org>
R: Zi Yan <ziy@nvidia.com>
R: Matthew Brost <matthew.brost@intel.com>
R: Joshua Hahn <joshua.hahnjy@gmail.com>
R: Byungchul Park <byungchul@sk.com>
R: Gregory Price <gourry@gourry.net>
R: Ying Huang <ying.huang@linux.alibaba.com>
R: Alistair Popple <apopple@nvidia.com>
L: linux-mm@kvack.org
S: Maintained
W: http://www.linux-mm.org
T: git git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
F: include/linux/migrate.h
F: include/linux/migrate_mode.h
F: mm/migrate.c
F: mm/migrate_device.c
MEMORY MANAGEMENT - MISC
M: Andrew Morton <akpm@linux-foundation.org>
M: David Hildenbrand <david@kernel.org>
@ -17286,7 +17281,6 @@ W: http://www.linux-mm.org
T: git git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
F: Documentation/ABI/testing/sysfs-kernel-mm
F: Documentation/ABI/testing/sysfs-kernel-mm-cma
F: Documentation/ABI/testing/sysfs-kernel-mm-memory-tiers
F: Documentation/ABI/testing/sysfs-kernel-mm-numa
F: Documentation/admin-guide/mm/
F: Documentation/mm/
@ -17294,7 +17288,6 @@ F: drivers/char/mem.c
F: include/linux/cma.h
F: include/linux/dmapool.h
F: include/linux/ioremap.h
F: include/linux/memory-tiers.h
F: include/linux/page_idle.h
F: mm/backing-dev.c
F: mm/cma.c
@ -17306,7 +17299,6 @@ F: mm/early_ioremap.c
F: mm/fadvise.c
F: mm/ioremap.c
F: mm/mapping_dirty_helpers.c
F: mm/memory-tiers.c
F: mm/page_idle.c
F: mm/pgalloc-track.h
F: mm/process_vm_access.c
@ -17323,6 +17315,27 @@ F: mm/numa.c
F: mm/numa_emulation.c
F: mm/numa_memblks.c
MEMORY MANAGEMENT - NUMA PLACEMENT
M: Andrew Morton <akpm@linux-foundation.org>
M: David Hildenbrand <david@kernel.org>
M: Gregory Price <gourry@gourry.net>
R: Zi Yan <ziy@nvidia.com>
R: Joshua Hahn <joshua.hahnjy@gmail.com>
R: Rakie Kim <rakie.kim@sk.com>
R: Ying Huang <ying.huang@linux.alibaba.com>
L: linux-mm@kvack.org
S: Maintained
W: http://www.linux-mm.org
T: git git://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
F: Documentation/ABI/testing/sysfs-kernel-mm-memory-tiers
F: Documentation/ABI/testing/sysfs-kernel-mm-mempolicy
F: Documentation/ABI/testing/sysfs-kernel-mm-mempolicy-weighted-interleave
F: include/linux/memory-tiers.h
F: include/linux/mempolicy.h
F: include/uapi/linux/mempolicy.h
F: mm/memory-tiers.c
F: mm/mempolicy.c
MEMORY MANAGEMENT - OOM KILLER
M: Michal Hocko <mhocko@suse.com>
R: David Rientjes <rientjes@google.com>
@ -18900,7 +18913,7 @@ F: net/core/drop_monitor.c
NETWORKING DRIVERS
M: Andrew Lunn <andrew+netdev@lunn.ch>
M: "David S. Miller" <davem@davemloft.net>
M: Eric Dumazet <edumazet@google.com>
M: Eric Dumazet <edumazet@kernel.org>
M: Jakub Kicinski <kuba@kernel.org>
M: Paolo Abeni <pabeni@redhat.com>
L: netdev@vger.kernel.org
@ -19017,7 +19030,7 @@ K: struct\s+phy_port|phy_port_
NETWORKING [GENERAL]
M: "David S. Miller" <davem@davemloft.net>
M: Eric Dumazet <edumazet@google.com>
M: Eric Dumazet <edumazet@kernel.org>
M: Jakub Kicinski <kuba@kernel.org>
M: Paolo Abeni <pabeni@redhat.com>
R: Simon Horman <horms@kernel.org>
@ -19243,7 +19256,7 @@ F: net/ipv6/seg6*
F: tools/testing/selftests/net/srv6*
NETWORKING [TCP]
M: Eric Dumazet <edumazet@google.com>
M: Eric Dumazet <edumazet@kernel.org>
M: Neal Cardwell <ncardwell@google.com>
R: Kuniyuki Iwashima <kuniyu@google.com>
L: netdev@vger.kernel.org
@ -19276,7 +19289,7 @@ F: net/tls/
F: tools/testing/selftests/net/tls.c
NETWORKING [SOCKETS]
M: Eric Dumazet <edumazet@google.com>
M: Eric Dumazet <edumazet@kernel.org>
M: Kuniyuki Iwashima <kuniyu@google.com>
M: Paolo Abeni <pabeni@redhat.com>
M: Willem de Bruijn <willemb@google.com>
@ -20572,6 +20585,7 @@ ORACLE CLUSTER FILESYSTEM 2 (OCFS2)
M: Mark Fasheh <mark@fasheh.com>
M: Joel Becker <jlbec@evilplan.org>
M: Joseph Qi <joseph.qi@linux.alibaba.com>
R: Heming Zhao <heming.zhao@suse.com>
L: ocfs2-devel@lists.linux.dev
S: Supported
W: http://ocfs2.wiki.kernel.org
@ -24490,6 +24504,11 @@ R: Marc Murphy <marc.murphy@sancloud.com>
S: Supported
F: arch/arm/boot/dts/ti/omap/am335x-sancloud*
SB1250-DUART BROADCOM SOC SERIAL DRIVER
M: "Maciej W. Rozycki" <macro@orcam.me.uk>
S: Maintained
F: drivers/tty/serial/sb1250-duart.c
SBOM
M: Luis Augenstein <luis.augenstein@tngtech.com>
M: Maximilian Huber <maximilian.huber@tngtech.com>
@ -25200,7 +25219,7 @@ R: David Rientjes <rientjes@google.com>
R: Roman Gushchin <roman.gushchin@linux.dev>
L: linux-mm@kvack.org
S: Maintained
T: git git://git.kernel.org/pub/scm/linux/kernel/git/vbabka/slab.git
T: git git://git.kernel.org/pub/scm/linux/kernel/git/mm/slab.git
F: Documentation/admin-guide/mm/slab.rst
F: Documentation/mm/slab.rst
F: include/linux/mempool.h

2
Makefile
View File

@ -2,7 +2,7 @@
VERSION = 7
PATCHLEVEL = 3
SUBLEVEL = 0
EXTRAVERSION = -rc5
EXTRAVERSION = -rc6
NAME = Baby Opossum Posse
# *DOCUMENTATION*

4
arch/arc/Kconfig
View File

@ -331,13 +331,9 @@ config ARC_FPU_SAVE_RESTORE
for floating point exceptions and rounding modes. These are
preserved across task context switch when enabled.
config ARC_CANT_LLSC
def_bool n
config ARC_HAS_LLSC
bool "Insn: LLOCK/SCOND (efficient atomic ops)"
default y
depends on !ARC_CANT_LLSC
if ISA_ARCV2

6
arch/arc/include/asm/atomic64-arcv2.h
View File

@ -156,6 +156,12 @@ static inline u64 __arch_cmpxchg64_relaxed(volatile void *ptr, u64 old, u64 new)
}
#define arch_cmpxchg64_relaxed __arch_cmpxchg64_relaxed
static inline s64 arch_atomic64_cmpxchg_relaxed(atomic64_t *v, s64 old, s64 new)
{
return arch_cmpxchg64_relaxed(&v->counter, old, new);
}
#define arch_atomic64_cmpxchg_relaxed arch_atomic64_cmpxchg_relaxed
static inline s64 arch_atomic64_xchg(atomic64_t *ptr, s64 new)
{
s64 prev;

4
arch/arc/include/asm/cmpxchg.h
View File

@ -46,7 +46,7 @@
__typeof__(*(ptr)) _n_ = (new); \
__typeof__(*(ptr)) _prev_; \
\
switch(sizeof((_p_))) { \
switch (sizeof(*(_p_))) { \
case 1: \
_prev_ = (__typeof__(*(ptr)))cmpxchg_emu_u8((volatile u8 *__force)_p_, (uintptr_t)_o_, (uintptr_t)_n_); \
break; \
@ -102,7 +102,7 @@
__typeof__(ptr) _p_ = (ptr); \
__typeof__(*(ptr)) _val_ = (val); \
\
switch(sizeof(*(_p_))) { \
switch (sizeof(*(_p_))) { \
case 4: \
_val_ = __arch_xchg(_p_, _val_); \
break; \

1
arch/arc/kernel/setup.c
View File

@ -593,6 +593,7 @@ static int show_cpuinfo(struct seq_file *m, void *v)
cpu_id);
} else {
freq = clk_get_rate(cpu_clk);
clk_put(cpu_clk);
}
if (freq)
seq_printf(m, "CPU speed\t: %lu.%02lu Mhz\n",

1
arch/arc/kernel/smp.c
View File

@ -12,7 +12,6 @@
#include <linux/spinlock.h>
#include <linux/sched/mm.h>
#include <linux/interrupt.h>
#include <linux/profile.h>
#include <linux/mm.h>
#include <linux/cpu.h>
#include <linux/irq.h>

30
arch/arm64/Kconfig
View File

@ -1040,9 +1040,17 @@ config ARM64_ERRATUM_1902691
If unsure, say Y.
config ARM64_WORKAROUND_BROKEN_AMU_CONSTCNT
bool
# Targeting errata affecting AMEVCNTR01, the AMU constant counter.
# On affected CPUs the counter may not increment at the expected rate,
# making it unreliable. As a result, AMU users should treat it as
# unavailable rather than relying on misleading counter values.
config ARM64_ERRATUM_2457168
bool "Cortex-A510: 2457168: workaround for AMEVCNTR01 incrementing incorrectly"
depends on ARM64_AMU_EXTN
select ARM64_WORKAROUND_BROKEN_AMU_CONSTCNT
default y
help
This option adds the workaround for ARM Cortex-A510 erratum 2457168.
@ -1073,6 +1081,28 @@ config ARM64_ERRATUM_2645198
If unsure, say Y.
config ARM64_ERRATUM_3821522
bool "Cortex-A725: 3821522: workaround for possible CNT_CYCLES increment error due to WFE/WFI"
depends on ARM64_AMU_EXTN
select ARM64_WORKAROUND_BROKEN_AMU_CONSTCNT
default y
help
This option adds the workaround for ARM Cortex-A725 erratum 3821522.
On affected A725 cores, the CNT_CYCLES event may incur a significant
increment error when entering and subsequently exiting WFx.
As a result, the CNT_CYCLES may diverge from the system counter
frequency at which it is expected to increment.
This renders the AMU counter AMEVCNTR01, that implements CNT_CYCLES,
being unreliable and unsuitable for use.
Since there is no hardware workaround, reads of the affected CNT_CYCLES
counter return 0 in the relevant paths. This causes users of the counter
to treat it as unavailable and is functionally equivalent to firmware
disabling the affected counter.
If unsure, say Y.
config ARM64_WORKAROUND_SPECULATIVE_UNPRIV_LOAD
bool

24
arch/arm64/kernel/cpu_errata.c
View File

@ -384,6 +384,20 @@ static const struct arm64_cpu_capabilities arm64_repeat_tlbi_list[] = {
};
#endif
#ifdef CONFIG_ARM64_WORKAROUND_BROKEN_AMU_CONSTCNT
static const struct midr_range workaround_amu_constcnt_list[] = {
#ifdef CONFIG_ARM64_ERRATUM_2457168
/* Cortex-A510 r0p0-r1p1 */
MIDR_RANGE(MIDR_CORTEX_A510, 0, 0, 1, 1),
#endif
#ifdef CONFIG_ARM64_ERRATUM_3821522
/* Cortex-A725 r0p0 - r0p2 */
MIDR_RANGE(MIDR_CORTEX_A725, 0, 0, 0, 2),
#endif
{}
};
#endif /* CONFIG_ARM64_WORKAROUND_BROKEN_AMU_CONSTCNT */
#ifdef CONFIG_CAVIUM_ERRATUM_23154
static const struct midr_range cavium_erratum_23154_cpus[] = {
MIDR_ALL_VERSIONS(MIDR_THUNDERX),
@ -919,14 +933,12 @@ const struct arm64_cpu_capabilities arm64_errata[] = {
ERRATA_MIDR_REV_RANGE(MIDR_CORTEX_A510, 0, 0, 2)
},
#endif
#ifdef CONFIG_ARM64_ERRATUM_2457168
#ifdef CONFIG_ARM64_WORKAROUND_BROKEN_AMU_CONSTCNT
{
.desc = "ARM erratum 2457168",
.capability = ARM64_WORKAROUND_2457168,
.desc = "Broken AMU AMEVCNTR01 (const counter)",
.capability = ARM64_WORKAROUND_BROKEN_AMU_CONSTCNT,
.type = ARM64_CPUCAP_WEAK_LOCAL_CPU_FEATURE,
/* Cortex-A510 r0p0-r1p1 */
CAP_MIDR_RANGE(MIDR_CORTEX_A510, 0, 0, 1, 1)
CAP_MIDR_RANGE_LIST(workaround_amu_constcnt_list)
},
#endif
#ifdef CONFIG_ARM64_ERRATUM_2038923

2
arch/arm64/kernel/cpufeature.c
View File

@ -2089,7 +2089,7 @@ static void cpu_amu_enable(struct arm64_cpu_capabilities const *cap)
cpumask_set_cpu(smp_processor_id(), &amu_cpus);
/* 0 reference values signal broken/disabled counters */
if (!this_cpu_has_cap(ARM64_WORKAROUND_2457168))
if (!this_cpu_has_cap(ARM64_WORKAROUND_BROKEN_AMU_CONSTCNT))
update_freq_counters_refs();
}
}

34
arch/arm64/kernel/signal.c
View File

@ -433,6 +433,7 @@ static int restore_sve_fpsimd_context(struct user_ctxs *user)
unsigned int vl, vq;
struct user_fpsimd_state fpsimd;
u16 user_vl, flags;
bool fpsimd_only;
bool sm;
if (user->sve_size < sizeof(*user->sve))
@ -443,19 +444,33 @@ static int restore_sve_fpsimd_context(struct user_ctxs *user)
if (err)
return err;
fpsimd_only = (user->sve_size == sizeof(*user->sve));
sm = flags & SVE_SIG_FLAG_SM;
if (sm) {
if (!system_supports_sme())
return -EINVAL;
/*
* Streaming SVE state is always preserved with an SVE payload.
* Only accept streaming state which has an SVE payload.
*/
if (fpsimd_only)
return -EINVAL;
vl = task_get_sme_vl(current);
} else {
/*
* A SME only system use SVE for streaming mode so can
* have a SVE formatted context with a zero VL and no
* payload data.
* Non-streaming SVE state may be preserved without an SVE
* payload, in which case all state is saved in the FPSIMD
* context.
*
* On SME-only systems, non-streaming (FPSIMD-only) state is
* always preserved without an SVE payload, and with VL==0. On
* such systems, only accept non-streaming state without an SVE
* payload.
*/
if (!system_supports_sve() && !system_supports_sme())
if (!system_supports_sve() && !fpsimd_only)
return -EINVAL;
vl = task_get_sve_vl(current);
@ -464,16 +479,7 @@ static int restore_sve_fpsimd_context(struct user_ctxs *user)
if (user_vl != vl)
return -EINVAL;
/*
* Non-streaming SVE state may be preserved without an SVE payload, in
* which case the SVE context only has a header with VL==0, and all
* state can be restored from the FPSIMD context.
*
* Streaming SVE state is always preserved with an SVE payload. For
* consistency and robustness, reject restoring streaming SVE state
* without an SVE payload.
*/
if (!sm && user->sve_size == sizeof(*user->sve))
if (fpsimd_only)
return restore_fpsimd_context(user);
vq = sve_vq_from_vl(vl);

22
arch/arm64/kernel/topology.c
View File

@ -186,7 +186,6 @@ int arch_freq_get_on_cpu(int cpu)
struct amu_cntr_sample *amu_sample;
unsigned int start_cpu = cpu;
unsigned long last_update;
unsigned int freq = 0;
u64 scale;
if (!amu_fie_cpu_supported(cpu) || !arch_scale_freq_ref(cpu))
@ -245,9 +244,8 @@ int arch_freq_get_on_cpu(int cpu)
* (see amu_scale_freq_tick for details)
*/
scale = arch_scale_freq_capacity(cpu);
freq = scale * arch_scale_freq_ref(cpu);
freq >>= SCHED_CAPACITY_SHIFT;
return freq;
return (scale * arch_scale_freq_ref(cpu)) >> SCHED_CAPACITY_SHIFT;
}
static void amu_fie_setup(const struct cpumask *cpus)
@ -397,12 +395,13 @@ static void cpu_read_corecnt(void *val)
static void cpu_read_constcnt(void *val)
{
/*
* Return 0 if the current CPU is affected by erratum 2457168. A value
* of 0 is also returned if the current CPU does not support AMUs or if
* the counter is disabled. A return value of 0 at counter read is
* properly handled as an error case by the users of the counter.
* Return 0 if the current CPU is affected by a HW erratum.
* A value of 0 is also returned if the current CPU does not
* support AMUs or if the counter is disabled. A return
* value of 0 at counter read is properly handled as an error
* case by the users of the counter.
*/
*(u64 *)val = this_cpu_has_cap(ARM64_WORKAROUND_2457168) ?
*(u64 *)val = this_cpu_has_cap(ARM64_WORKAROUND_BROKEN_AMU_CONSTCNT) ?
0UL : read_constcnt();
}
@ -463,8 +462,9 @@ static void amu_read_core_const_ctrs(void *val)
/*
* cpu_read_constcnt() incurs slight latency due to the
* ARM64_WORKAROUND_2457168 check. Read it first to minimize
* the sampling skew between the const and core counters.
* ARM64_WORKAROUND_BROKEN_AMU_CONSTCNT check.
* Read it first to minimize the sampling skew between the const
* and core counters.
*/
cpu_read_constcnt(&ctrs->constcnt);
cpu_read_corecnt(&ctrs->corecnt);

2
arch/arm64/kernel/vdso/Makefile
View File

@ -41,7 +41,7 @@ CC_FLAGS_REMOVE_VDSO := $(CC_FLAGS_FTRACE) -Os $(CC_FLAGS_SCS) \
$(CC_FLAGS_LTO) $(CC_FLAGS_CFI) \
-Wmissing-prototypes -Wmissing-declarations
CC_FLAGS_ADD_VDSO := -O2 -mcmodel=tiny -fasynchronous-unwind-tables
CC_FLAGS_ADD_VDSO := -O2 -mcmodel=tiny -fasynchronous-unwind-tables $(CONFIG_CC_OPT_INLINE_MEMSET)
CFLAGS_REMOVE_vgettimeofday.o = $(CC_FLAGS_REMOVE_VDSO)
CFLAGS_REMOVE_vgetrandom.o = $(CC_FLAGS_REMOVE_VDSO)

3
arch/arm64/kvm/emulate-nested.c
View File

@ -2386,9 +2386,6 @@ int __init populate_nv_trap_config(void)
print_nv_trap_error(fgt, "FGT bit is reserved", ret);
}
if (!cpus_have_final_cap(ARM64_HAS_FGT))
continue;
prev = xa_store(&sr_forward_xa, enc,
xa_mk_value(tc.val), GFP_KERNEL);

9
arch/arm64/kvm/hyp/include/nvhe/pkvm.h
View File

@ -12,6 +12,15 @@
#include <nvhe/gfp.h>
#include <nvhe/spinlock.h>
/*
* HCR_EL2 bits EL2 takes from the host on each entry, per VM type. The rest
* are EL2's own and nothing the host sets there reaches the guest.
*/
#define PKVM_HCR_EL2_HOST_PVM (HCR_EL2_TWI | HCR_EL2_TWE | HCR_EL2_VSE)
#define PKVM_HCR_EL2_HOST_NPVM (PKVM_HCR_EL2_HOST_PVM | HCR_EL2_VI | HCR_EL2_VF | \
HCR_EL2_TVM | HCR_EL2_TID2 | HCR_EL2_TID4 | \
HCR_EL2_TID5 | HCR_EL2_TTLBOS)
/*
* Holds the relevant data for maintaining the vcpu state completely at hyp.
*/

7
arch/arm64/kvm/hyp/nvhe/hyp-main.c
View File

@ -216,6 +216,7 @@ static void sync_debug_state(struct pkvm_hyp_vcpu *hyp_vcpu)
static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu)
{
struct kvm_vcpu *host_vcpu = hyp_vcpu->host_vcpu;
u64 host_hcr_mask = PKVM_HCR_EL2_HOST_PVM;
fpsimd_sve_flush();
flush_debug_state(hyp_vcpu);
@ -228,6 +229,7 @@ static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu)
if (!pkvm_hyp_vcpu_is_protected(hyp_vcpu)) {
if (vcpu_get_flag(host_vcpu, PKVM_HOST_STATE_DIRTY))
flush_hyp_vcpu_state(hyp_vcpu);
host_hcr_mask = PKVM_HCR_EL2_HOST_NPVM;
} else {
hyp_vcpu->vcpu.arch.ctxt = host_vcpu->arch.ctxt;
}
@ -241,9 +243,8 @@ static void flush_hyp_vcpu(struct pkvm_hyp_vcpu *hyp_vcpu)
* trap-control bit, so it must flow to the hyp vCPU alongside TWI/TWE
* for the vSError to be delivered. sync_hyp_vcpu() reflects it back.
*/
hyp_vcpu->vcpu.arch.hcr_el2 &= ~(HCR_TWI | HCR_TWE | HCR_VSE);
hyp_vcpu->vcpu.arch.hcr_el2 |= READ_ONCE(host_vcpu->arch.hcr_el2) &
(HCR_TWI | HCR_TWE | HCR_VSE);
hyp_vcpu->vcpu.arch.hcr_el2 &= ~host_hcr_mask;
hyp_vcpu->vcpu.arch.hcr_el2 |= READ_ONCE(host_vcpu->arch.hcr_el2) & host_hcr_mask;
hyp_vcpu->vcpu.arch.iflags = host_vcpu->arch.iflags;

25
arch/arm64/kvm/hyp/nvhe/pkvm.c
View File

@ -30,6 +30,7 @@ unsigned int kvm_host_sve_max_vl;
*/
static DEFINE_PER_CPU(struct pkvm_hyp_vcpu *, loaded_hyp_vcpu);
/* The PKVM_HCR_EL2_HOST_{PVM,NPVM} bits of this value come from the host on each entry. */
static void pkvm_vcpu_reset_hcr(struct kvm_vcpu *vcpu)
{
vcpu->arch.hcr_el2 = HCR_GUEST_FLAGS;
@ -47,18 +48,17 @@ static void pkvm_vcpu_reset_hcr(struct kvm_vcpu *vcpu)
if (cpus_have_final_cap(ARM64_HAS_STAGE2_FWB))
vcpu->arch.hcr_el2 |= HCR_FWB;
if (cpus_have_final_cap(ARM64_HAS_EVT) &&
!cpus_have_final_cap(ARM64_MISMATCHED_CACHE_TYPE) &&
kvm_read_vm_id_reg(vcpu->kvm, SYS_CTR_EL0) == read_cpuid(CTR_EL0))
vcpu->arch.hcr_el2 |= HCR_TID4;
else
vcpu->arch.hcr_el2 |= HCR_TID2;
/*
* Without AArch32 EL1, leave RW set and let the entry fail with an
* illegal exception return: the *32_EL2 registers EL2 would otherwise
* switch are UNDEFINED there.
*/
if (vcpu_has_feature(vcpu, KVM_ARM_VCPU_EL1_32BIT) &&
cpus_have_final_cap(ARM64_HAS_32BIT_EL1))
vcpu->arch.hcr_el2 &= ~HCR_EL2_RW;
if (vcpu_has_ptrauth(vcpu))
vcpu->arch.hcr_el2 |= (HCR_API | HCR_APK);
if (kvm_has_mte(vcpu->kvm))
vcpu->arch.hcr_el2 |= HCR_ATA;
}
static void pvm_init_traps_hcr(struct kvm_vcpu *vcpu)
@ -76,6 +76,13 @@ static void pvm_init_traps_hcr(struct kvm_vcpu *vcpu)
*/
val |= HCR_TACR | HCR_TIDCP | HCR_TID3 | HCR_TID1;
if (cpus_have_final_cap(ARM64_HAS_EVT) &&
!cpus_have_final_cap(ARM64_MISMATCHED_CACHE_TYPE) &&
kvm_read_vm_id_reg(kvm, SYS_CTR_EL0) == read_cpuid(CTR_EL0))
val |= HCR_EL2_TID4;
else
val |= HCR_EL2_TID2;
if (!kvm_has_feat(kvm, ID_AA64PFR0_EL1, RAS, IMP)) {
val |= HCR_TERR | HCR_TEA;
val &= ~(HCR_FIEN);

47
arch/arm64/kvm/mmu.c
View File

@ -1468,32 +1468,11 @@ transparent_hugepage_adjust(struct kvm *kvm, struct kvm_memory_slot *memslot,
return PAGE_SIZE;
}
static int get_vma_page_shift(struct vm_area_struct *vma, unsigned long hva)
static int get_vma_page_shift(struct vm_area_struct *vma)
{
unsigned long pa;
if (is_vm_hugetlb_page(vma) && !(vma->vm_flags & VM_PFNMAP))
if (is_vm_hugetlb_page(vma))
return huge_page_shift(hstate_vma(vma));
if (!(vma->vm_flags & VM_PFNMAP))
return PAGE_SHIFT;
VM_BUG_ON(is_vm_hugetlb_page(vma));
pa = (vma->vm_pgoff << PAGE_SHIFT) + (hva - vma->vm_start);
#ifndef __PAGETABLE_PMD_FOLDED
if ((hva & (PUD_SIZE - 1)) == (pa & (PUD_SIZE - 1)) &&
ALIGN_DOWN(hva, PUD_SIZE) >= vma->vm_start &&
ALIGN(hva, PUD_SIZE) <= vma->vm_end)
return PUD_SHIFT;
#endif
if ((hva & (PMD_SIZE - 1)) == (pa & (PMD_SIZE - 1)) &&
ALIGN_DOWN(hva, PMD_SIZE) >= vma->vm_start &&
ALIGN(hva, PMD_SIZE) <= vma->vm_end)
return PMD_SHIFT;
return PAGE_SHIFT;
}
@ -1794,7 +1773,7 @@ static short kvm_s2_resolve_vma_size(const struct kvm_s2_fault_desc *s2fd,
vma_shift = PAGE_SHIFT;
} else {
s2vi->max_map_size = PUD_SIZE;
vma_shift = get_vma_page_shift(vma, s2fd->hva);
vma_shift = get_vma_page_shift(vma);
}
switch (vma_shift) {
@ -1952,16 +1931,6 @@ static int kvm_s2_fault_pin_pfn(const struct kvm_s2_fault_desc *s2fd,
return -EFAULT;
}
} else {
/*
* If the page was identified as device early by looking at
* the VMA flags, vma_pagesize is already representing the
* largest quantity we can map. If instead it was mapped
* via __kvm_faultin_pfn(), vma_pagesize is set to PAGE_SIZE
* and must not be upgraded.
*
* In both cases, we don't let transparent_hugepage_adjust()
* change things at the last minute.
*/
s2vi->map_non_cacheable = true;
}
@ -2051,10 +2020,10 @@ static int kvm_s2_fault_map(const struct kvm_s2_fault_desc *s2fd,
/*
* If we are not forced to use page mapping, check if we are
* backed by a THP and thus use block mapping if possible.
* backed by a huge stage-1 mapping and thus use block mapping if
* possible.
*/
if (mapping_size == PAGE_SIZE &&
!(s2vi->max_map_size == PAGE_SIZE || s2vi->map_non_cacheable)) {
if (mapping_size == PAGE_SIZE && s2vi->max_map_size != PAGE_SIZE) {
if (perm_fault_granule > PAGE_SIZE) {
mapping_size = perm_fault_granule;
} else {
@ -2135,10 +2104,6 @@ static int user_mem_abort(const struct kvm_s2_fault_desc *s2fd)
return ret;
}
/*
* Let's check if we will get back a huge page backed by hugetlbfs, or
* get block mapping for device MMIO region.
*/
ret = kvm_s2_fault_pin_pfn(s2fd, &s2vi);
if (ret != 1)
return ret;

12
arch/arm64/kvm/vgic/vgic-init.c
View File

@ -97,6 +97,9 @@ int kvm_vgic_create(struct kvm *kvm, u32 type)
/*
* - Acquiring the vCPU mutex for every *online* vCPU to prevent
* concurrent vCPU ioctls for vCPUs already visible to userspace.
* This also ensures KVM isn't in the middle of creating a vCPU,
* i.e. that there are no vCPUs that have been created but aren't
* yet fully online.
*/
ret = -EBUSY;
if (kvm_trylock_all_vcpus(kvm))
@ -105,18 +108,11 @@ int kvm_vgic_create(struct kvm *kvm, u32 type)
/*
* - Taking the config_lock which protects VGIC data structures such
* as the per-vCPU arrays of private IRQs (SGIs, PPIs).
*/
mutex_lock(&kvm->arch.config_lock);
/*
* - Bailing on the entire thing if a vCPU is in the middle of creation,
* dropped the kvm->lock, but hasn't reached kvm_arch_vcpu_create().
*
* The whole combination of this guarantees that no vCPU can get into
* KVM with a VGIC configuration inconsistent with the VM's VGIC.
*/
if (kvm->created_vcpus != atomic_read(&kvm->online_vcpus))
goto out_unlock;
mutex_lock(&kvm->arch.config_lock);
if (irqchip_in_kernel(kvm)) {
ret = -EEXIST;

12
arch/arm64/kvm/vgic/vgic-its.c
View File

@ -319,12 +319,16 @@ static int update_lpi_config(struct kvm *kvm, struct vgic_irq *irq,
return ret;
}
static int update_affinity(struct vgic_irq *irq, struct kvm_vcpu *vcpu)
static int update_affinity(struct vgic_irq *irq,
struct kvm_vcpu *from_vcpu, struct kvm_vcpu *vcpu)
{
struct its_vlpi_map map;
int ret;
guard(raw_spinlock_irqsave)(&irq->irq_lock);
if (from_vcpu && irq->target_vcpu != from_vcpu)
return 0;
irq->target_vcpu = vcpu;
if (!irq->hw)
@ -362,7 +366,7 @@ static void update_affinity_ite(struct kvm *kvm, struct its_ite *ite)
return;
vcpu = collection_to_vcpu(kvm, ite->collection);
update_affinity(ite->irq, vcpu);
update_affinity(ite->irq, NULL, vcpu);
}
/*
@ -856,7 +860,7 @@ static int vgic_its_cmd_handle_movi(struct kvm *kvm, struct vgic_its *its,
vgic_its_invalidate_cache(its);
return update_affinity(ite->irq, vcpu);
return update_affinity(ite->irq, NULL, vcpu);
}
static bool __is_visible_gfn_locked(struct vgic_its *its, gpa_t gpa)
@ -1383,7 +1387,7 @@ static int vgic_its_cmd_handle_movall(struct kvm *kvm, struct vgic_its *its,
if (!irq)
continue;
update_affinity(irq, vcpu2);
update_affinity(irq, vcpu1, vcpu2);
vgic_put_irq(kvm, irq);
}

6
arch/arm64/kvm/vgic/vgic-v2.c
View File

@ -122,6 +122,10 @@ void vgic_v2_fold_lr_state(struct kvm_vcpu *vcpu)
for (int lr = 0; lr < vgic_cpu->vgic_v2.used_lrs; lr++)
vgic_v2_fold_lr(vcpu, cpuif->vgic_lr[lr]);
cpuif->used_lrs = 0;
if (!irq)
return;
/* See the GICv3 equivalent for the EOIcount handling rationale */
list_for_each_entry_continue(irq, &vgic_cpu->ap_list_head, ap_list) {
u32 lr;
@ -144,8 +148,6 @@ void vgic_v2_fold_lr_state(struct kvm_vcpu *vcpu)
vgic_v2_fold_lr(vcpu, lr);
eoicount--;
}
cpuif->used_lrs = 0;
}
void vgic_v2_deactivate(struct kvm_vcpu *vcpu, u32 val)

6
arch/arm64/kvm/vgic/vgic-v3.c
View File

@ -155,6 +155,10 @@ void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu)
for (int lr = 0; lr < cpuif->used_lrs; lr++)
vgic_v3_fold_lr(vcpu, cpuif->vgic_lr[lr]);
cpuif->used_lrs = 0;
if (!irq)
return;
/*
* EOIMode=0: use EOIcount to emulate deactivation. We are
* guaranteed to deactivate in reverse order of the activation, so
@ -188,8 +192,6 @@ void vgic_v3_fold_lr_state(struct kvm_vcpu *vcpu)
vgic_v3_fold_lr(vcpu, lr);
eoicount--;
}
cpuif->used_lrs = 0;
}
void vgic_v3_deactivate(struct kvm_vcpu *vcpu, u64 val)

21
arch/arm64/kvm/vgic/vgic.c
View File

@ -853,6 +853,17 @@ static void vgic_prune_ap_list(struct kvm_vcpu *vcpu)
goto retry;
}
/*
* Fix the last_lr_irq refcount which was obtained while
* populating the LRs. This can also result in the LPI being
* deleted.
*/
irq = *host_data_ptr(last_lr_irq);
if (irq) {
deleted_lpis |= vgic_put_irq_norelease(vcpu->kvm, irq);
*host_data_ptr(last_lr_irq) = NULL;
}
raw_spin_unlock(&vgic_cpu->ap_list_lock);
if (unlikely(deleted_lpis))
@ -866,9 +877,6 @@ static void vgic_fold_state(struct kvm_vcpu *vcpu)
return;
}
if (!*host_data_ptr(last_lr_irq))
return;
if (kvm_vgic_global_state.type == VGIC_V2)
vgic_v2_fold_lr_state(vcpu);
else
@ -1021,11 +1029,14 @@ static void vgic_flush_lr_state(struct kvm_vcpu *vcpu)
scoped_guard(raw_spinlock, &irq->irq_lock) {
if (likely(vgic_target_oracle(irq) == vcpu)) {
vgic_populate_lr(vcpu, irq, count++);
*host_data_ptr(last_lr_irq) = irq;
if (count == kvm_vgic_global_state.nr_lr) {
vgic_get_irq_ref(irq);
*host_data_ptr(last_lr_irq) = irq;
}
}
}
if (count == kvm_vgic_global_state.nr_lr)
if (*host_data_ptr(last_lr_irq))
break;
}

2
arch/arm64/mm/mmu.c
View File

@ -2295,7 +2295,7 @@ pte_t modify_prot_start_ptes(struct vm_area_struct *vma, unsigned long addr,
* in cases where cpu is affected with errata #2645198.
*/
if (pte_accessible(vma->vm_mm, pte) && pte_user_exec(pte))
__flush_tlb_range(vma, addr, nr * PAGE_SIZE,
__flush_tlb_range(vma, addr, addr + nr * PAGE_SIZE,
PAGE_SIZE, 3, TLBF_NOWALKCACHE);
}

37
arch/arm64/net/bpf_jit_comp.c
View File

@ -2418,10 +2418,11 @@ bool bpf_jit_supports_subprog_tailcalls(void)
return true;
}
static void invoke_bpf_prog(struct jit_ctx *ctx, struct bpf_tramp_node *node,
int bargs_off, int retval_off, int run_ctx_off,
bool save_ret)
static void invoke_bpf_prog(struct jit_ctx *ctx, struct bpf_tramp_image *im,
struct bpf_tramp_node *node, int bargs_off,
int retval_off, int run_ctx_off, bool save_ret)
{
void *skip;
__le32 *branch;
u64 enter_prog;
u64 exit_prog;
@ -2431,6 +2432,10 @@ static void invoke_bpf_prog(struct jit_ctx *ctx, struct bpf_tramp_node *node,
enter_prog = (u64)bpf_trampoline_enter(p);
exit_prog = (u64)bpf_trampoline_exit(p);
/* nop, patched to skip this prog when it is detached */
skip = ctx->ro_image + ctx->idx;
emit(A64_NOP, ctx);
if (node->cookie == 0) {
/* if cookie is zero, one instruction is enough to store it */
emit(A64_STR64I(A64_ZR, A64_SP, run_ctx_off + cookie_off), ctx);
@ -2483,11 +2488,13 @@ static void invoke_bpf_prog(struct jit_ctx *ctx, struct bpf_tramp_node *node,
emit(A64_ADD_I(1, A64_R(2), A64_SP, run_ctx_off), ctx);
emit_call(exit_prog, ctx);
bpf_tramp_image_add_skip(im, p, skip, ctx->ro_image + ctx->idx);
}
static void invoke_bpf_mod_ret(struct jit_ctx *ctx, struct bpf_tramp_nodes *tn,
int bargs_off, int retval_off, int run_ctx_off,
__le32 **branches)
static void invoke_bpf_mod_ret(struct jit_ctx *ctx, struct bpf_tramp_image *im,
struct bpf_tramp_nodes *tn, int bargs_off,
int retval_off, int run_ctx_off, __le32 **branches)
{
int i;
@ -2496,7 +2503,7 @@ static void invoke_bpf_mod_ret(struct jit_ctx *ctx, struct bpf_tramp_nodes *tn,
*/
emit(A64_STR64I(A64_ZR, A64_SP, retval_off), ctx);
for (i = 0; i < tn->nr_nodes; i++) {
invoke_bpf_prog(ctx, tn->nodes[i], bargs_off, retval_off,
invoke_bpf_prog(ctx, im, tn->nodes[i], bargs_off, retval_off,
run_ctx_off, true);
/* if (*(u64 *)(sp + retval_off) != 0)
* goto do_fexit;
@ -2882,7 +2889,7 @@ static int prepare_trampoline(struct jit_ctx *ctx, struct bpf_tramp_image *im,
store_func_meta(ctx, meta, func_meta_off);
cookie_bargs_off--;
}
invoke_bpf_prog(ctx, fentry->nodes[i], bargs_off,
invoke_bpf_prog(ctx, im, fentry->nodes[i], bargs_off,
retval_off, run_ctx_off,
flags & BPF_TRAMP_F_RET_FENTRY_RET);
}
@ -2893,7 +2900,7 @@ static int prepare_trampoline(struct jit_ctx *ctx, struct bpf_tramp_image *im,
if (!branches)
return -ENOMEM;
invoke_bpf_mod_ret(ctx, fmod_ret, bargs_off, retval_off,
invoke_bpf_mod_ret(ctx, im, fmod_ret, bargs_off, retval_off,
run_ctx_off, branches);
}
@ -2906,9 +2913,6 @@ static int prepare_trampoline(struct jit_ctx *ctx, struct bpf_tramp_image *im,
emit(A64_RET(A64_R(10)), ctx);
/* store return value */
emit(A64_STR64I(A64_R(0), A64_SP, retval_off), ctx);
/* reserve a nop for bpf_tramp_image_put */
im->ip_after_call = ctx->ro_image + ctx->idx;
emit(A64_NOP, ctx);
}
/* update the branches saved in invoke_bpf_mod_ret with cbnz */
@ -2930,12 +2934,11 @@ static int prepare_trampoline(struct jit_ctx *ctx, struct bpf_tramp_image *im,
store_func_meta(ctx, meta, func_meta_off);
cookie_bargs_off--;
}
invoke_bpf_prog(ctx, fexit->nodes[i], bargs_off, retval_off,
invoke_bpf_prog(ctx, im, fexit->nodes[i], bargs_off, retval_off,
run_ctx_off, false);
}
if (flags & BPF_TRAMP_F_CALL_ORIG) {
im->ip_epilogue = ctx->ro_image + ctx->idx;
/* for the first pass, assume the worst case */
if (!ctx->image)
ctx->idx += 4;
@ -2994,7 +2997,7 @@ int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags,
.image = NULL,
.idx = 0,
};
struct bpf_tramp_image im;
struct bpf_tramp_image im = {};
struct arg_aux aaux;
int ret;
@ -3281,6 +3284,10 @@ int bpf_arch_text_poke(void *ip, enum bpf_text_poke_type old_t,
* longer reachable, since bpf_tramp_image_put() function already
* uses percpu_ref and task-based rcu to do the sync, no need to call
* the sync version here, see bpf_tramp_image_put() for details.
*
* 3. when a detached prog is patched out of a trampoline, a CPU that
* still executes the old nop calls the prog before it went through
* a quiescent state, and the prog is freed after grace periods.
*/
ret = aarch64_insn_patch_text_nosync(ip, new_insn);
out:

2
arch/arm64/tools/cpucaps
View File

@ -105,7 +105,6 @@ WORKAROUND_1902691
WORKAROUND_2038923
WORKAROUND_2064142
WORKAROUND_2077057
WORKAROUND_2457168
WORKAROUND_2645198
WORKAROUND_2658417
WORKAROUND_4193714
@ -132,3 +131,4 @@ WORKAROUND_REPEAT_TLBI_SYNC
WORKAROUND_SPECULATIVE_AT
WORKAROUND_SPECULATIVE_SSBS
WORKAROUND_SPECULATIVE_UNPRIV_LOAD
WORKAROUND_BROKEN_AMU_CONSTCNT

60
arch/loongarch/net/bpf_jit.c
View File

@ -1578,6 +1578,24 @@ void *bpf_arch_text_copy(void *dst, void *src, size_t len)
return ret ? ERR_PTR(-EINVAL) : dst;
}
int arch_bpf_trampoline_skip(void *nop, void *target)
{
u32 old_insn = INSN_NOP;
u32 new_insn = larch_insn_gen_b((unsigned long)nop, (unsigned long)target);
int ret;
if (memcmp(nop, &old_insn, LOONGARCH_INSN_SIZE))
return -EFAULT;
cpus_read_lock();
mutex_lock(&text_mutex);
ret = larch_insn_text_copy(nop, &new_insn, LOONGARCH_INSN_SIZE);
mutex_unlock(&text_mutex);
cpus_read_unlock();
return ret;
}
int bpf_arch_text_poke(void *ip, enum bpf_text_poke_type old_t,
enum bpf_text_poke_type new_t, void *old_addr,
void *new_addr)
@ -1696,13 +1714,18 @@ static void restore_stk_args(struct jit_ctx *ctx, int nr_stk_args, int args_off,
}
}
static int invoke_bpf_prog(struct jit_ctx *ctx, struct bpf_tramp_node *n,
int args_off, int retval_off, int run_ctx_off, bool save_ret)
static int invoke_bpf_prog(struct jit_ctx *ctx, struct bpf_tramp_image *im,
struct bpf_tramp_node *n, int args_off, int retval_off,
int run_ctx_off, bool save_ret)
{
int ret;
u32 *branch;
struct bpf_prog *p = n->link->prog;
int cookie_off = offsetof(struct bpf_tramp_run_ctx, bpf_cookie);
void *skip = ctx->ro_image + ctx->idx;
/* nop, patched to a b over this prog when it is detached */
emit_insn(ctx, nop);
if (n->cookie)
emit_store_stack_imm64(ctx, LOONGARCH_GPR_T1,
@ -1755,13 +1778,17 @@ static int invoke_bpf_prog(struct jit_ctx *ctx, struct bpf_tramp_node *n,
/* arg3: &run_ctx */
emit_insn(ctx, addid, LOONGARCH_GPR_A2, LOONGARCH_GPR_FP, -run_ctx_off);
ret = emit_call(ctx, (const u64)bpf_trampoline_exit(p));
if (ret)
return ret;
return ret;
bpf_tramp_image_add_skip(im, p, skip, ctx->ro_image + ctx->idx);
return 0;
}
static int invoke_bpf(struct jit_ctx *ctx, struct bpf_tramp_nodes *tn,
int args_off, int retval_off, int run_ctx_off,
int func_meta_off, bool save_ret, u64 func_meta, int cookie_off)
static int invoke_bpf(struct jit_ctx *ctx, struct bpf_tramp_image *im,
struct bpf_tramp_nodes *tn, int args_off, int retval_off,
int run_ctx_off, int func_meta_off, bool save_ret,
u64 func_meta, int cookie_off)
{
int i, cur_cookie = (cookie_off - args_off) / 8;
@ -1774,7 +1801,8 @@ static int invoke_bpf(struct jit_ctx *ctx, struct bpf_tramp_nodes *tn,
emit_store_stack_imm64(ctx, LOONGARCH_GPR_T1, -func_meta_off, meta);
cur_cookie--;
}
err = invoke_bpf_prog(ctx, tn->nodes[i], args_off, retval_off, run_ctx_off, save_ret);
err = invoke_bpf_prog(ctx, im, tn->nodes[i], args_off, retval_off,
run_ctx_off, save_ret);
if (err)
return err;
}
@ -2017,7 +2045,7 @@ static int __arch_prepare_bpf_trampoline(struct jit_ctx *ctx, struct bpf_tramp_i
}
if (fentry->nr_nodes) {
ret = invoke_bpf(ctx, fentry, args_off, retval_off, run_ctx_off, func_meta_off,
ret = invoke_bpf(ctx, im, fentry, args_off, retval_off, run_ctx_off, func_meta_off,
flags & BPF_TRAMP_F_RET_FENTRY_RET, func_meta, cookie_off);
if (ret)
return ret;
@ -2029,7 +2057,7 @@ static int __arch_prepare_bpf_trampoline(struct jit_ctx *ctx, struct bpf_tramp_i
emit_insn(ctx, std, LOONGARCH_GPR_ZERO, LOONGARCH_GPR_FP, -retval_off);
for (i = 0; i < fmod_ret->nr_nodes; i++) {
ret = invoke_bpf_prog(ctx, fmod_ret->nodes[i],
ret = invoke_bpf_prog(ctx, im, fmod_ret->nodes[i],
args_off, retval_off, run_ctx_off, true);
if (ret)
goto out;
@ -2051,10 +2079,6 @@ static int __arch_prepare_bpf_trampoline(struct jit_ctx *ctx, struct bpf_tramp_i
goto out;
emit_insn(ctx, std, LOONGARCH_GPR_A0, LOONGARCH_GPR_FP, -retval_off);
emit_insn(ctx, std, regmap[BPF_REG_0], LOONGARCH_GPR_FP, -(retval_off - 8));
im->ip_after_call = ctx->ro_image + ctx->idx;
/* Reserve space for the move_imm + jirl instruction */
for (i = 0; i < LOONGARCH_LONG_JUMP_NINSNS; i++)
emit_insn(ctx, nop);
}
for (i = 0; ctx->image && i < fmod_ret->nr_nodes; i++) {
@ -2068,14 +2092,13 @@ static int __arch_prepare_bpf_trampoline(struct jit_ctx *ctx, struct bpf_tramp_i
emit_store_stack_imm64(ctx, LOONGARCH_GPR_T1, -func_meta_off, func_meta);
if (fexit->nr_nodes) {
ret = invoke_bpf(ctx, fexit, args_off, retval_off, run_ctx_off,
ret = invoke_bpf(ctx, im, fexit, args_off, retval_off, run_ctx_off,
func_meta_off, false, func_meta, cookie_off);
if (ret)
goto out;
}
if (flags & BPF_TRAMP_F_CALL_ORIG) {
im->ip_epilogue = ctx->ro_image + ctx->idx;
move_addr(ctx, LOONGARCH_GPR_A0, (const u64)im);
ret = emit_call(ctx, (const u64)__bpf_tramp_exit);
if (ret)
@ -2177,11 +2200,8 @@ int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags,
struct bpf_tramp_nodes *tnodes, void *func_addr)
{
int ret;
struct jit_ctx ctx;
struct bpf_tramp_image im;
ctx.image = NULL;
ctx.idx = 0;
struct jit_ctx ctx = {};
struct bpf_tramp_image im = {};
ret = __arch_prepare_bpf_trampoline(&ctx, &im, m, tnodes, func_addr, flags);

1
arch/loongarch/vdso/Makefile
View File

@ -16,6 +16,7 @@ ccflags-vdso := \
$(filter -m64,$(KBUILD_CFLAGS)) \
$(filter -march=%,$(KBUILD_CFLAGS)) \
$(filter -m%-float,$(KBUILD_CFLAGS)) \
$(CONFIG_CC_OPT_INLINE_MEMSET) \
$(CLANG_FLAGS) \
-D__VDSO__

1
arch/powerpc/kernel/vdso/Makefile
View File

@ -43,6 +43,7 @@ ccflags-y := -fno-common -fno-builtin -DBUILD_VDSO
ccflags-y += $(DISABLE_LATENT_ENTROPY_PLUGIN)
ccflags-y += $(call cc-option, -fno-stack-protector)
ccflags-y += -DDISABLE_BRANCH_PROFILING
ccflags-y += $(CONFIG_CC_OPT_INLINE_MEMSET)
ccflags-y += -ffreestanding -fasynchronous-unwind-tables
ccflags-remove-y := $(CC_FLAGS_FTRACE)
ldflags-y := -Wl,--hash-style=both -nostdlib -shared -z noexecstack $(CLANG_FLAGS)

2
arch/powerpc/kvm/book3s_hv.c
View File

@ -3058,7 +3058,6 @@ static int kvmppc_core_vcpu_create_hv(struct kvm_vcpu *vcpu)
init_waitqueue_head(&vcpu->arch.cpu_run);
mutex_lock(&kvm->lock);
vcore = NULL;
err = -EINVAL;
if (cpu_has_feature(CPU_FTR_ARCH_300)) {
@ -3091,7 +3090,6 @@ static int kvmppc_core_vcpu_create_hv(struct kvm_vcpu *vcpu)
mutex_unlock(&kvm->arch.mmu_setup_lock);
}
}
mutex_unlock(&kvm->lock);
if (!vcore)
return err;

45
arch/powerpc/net/bpf_jit_comp.c
View File

@ -602,14 +602,18 @@ int arch_protect_bpf_trampoline(void *image, unsigned int size)
}
static int invoke_bpf_prog(u32 *image, u32 *ro_image, struct codegen_context *ctx,
struct bpf_tramp_node *n, int regs_off, int retval_off,
int run_ctx_off, bool save_ret)
struct bpf_tramp_image *im, struct bpf_tramp_node *n,
int regs_off, int retval_off, int run_ctx_off, bool save_ret)
{
struct bpf_prog *p = n->link->prog;
ppc_inst_t branch_insn;
u32 jmp_idx;
u32 jmp_idx, skip_idx;
int ret = 0;
/* nop, patched to skip this prog when it is detached */
skip_idx = ctx->idx;
EMIT(PPC_RAW_NOP());
/* Save cookie */
if (IS_ENABLED(CONFIG_PPC64)) {
PPC_LI64(_R3, n->cookie);
@ -679,13 +683,17 @@ static int invoke_bpf_prog(u32 *image, u32 *ro_image, struct codegen_context *ct
EMIT(PPC_RAW_ADDI(_R5, _R1, run_ctx_off));
ret = bpf_jit_emit_func_call_rel(image, ro_image, ctx,
(unsigned long)bpf_trampoline_exit(p));
if (ret)
return ret;
return ret;
if (ro_image) /* image is NULL for dummy pass */
bpf_tramp_image_add_skip(im, p, &ro_image[skip_idx], &ro_image[ctx->idx]);
return 0;
}
static int invoke_bpf_mod_ret(u32 *image, u32 *ro_image, struct codegen_context *ctx,
struct bpf_tramp_nodes *tn, int regs_off, int retval_off,
int run_ctx_off, u32 *branches)
struct bpf_tramp_image *im, struct bpf_tramp_nodes *tn,
int regs_off, int retval_off, int run_ctx_off, u32 *branches)
{
int i;
@ -696,8 +704,8 @@ static int invoke_bpf_mod_ret(u32 *image, u32 *ro_image, struct codegen_context
EMIT(PPC_RAW_LI(_R3, 0));
EMIT(PPC_RAW_STL(_R3, _R1, retval_off));
for (i = 0; i < tn->nr_nodes; i++) {
if (invoke_bpf_prog(image, ro_image, ctx, tn->nodes[i], regs_off, retval_off,
run_ctx_off, true))
if (invoke_bpf_prog(image, ro_image, ctx, im, tn->nodes[i], regs_off,
retval_off, run_ctx_off, true))
return -EINVAL;
/*
@ -1043,8 +1051,8 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im
cookie_ctx_off--;
}
if (invoke_bpf_prog(image, ro_image, ctx, fentry->nodes[i], regs_off, retval_off,
run_ctx_off, flags & BPF_TRAMP_F_RET_FENTRY_RET))
if (invoke_bpf_prog(image, ro_image, ctx, im, fentry->nodes[i], regs_off,
retval_off, run_ctx_off, flags & BPF_TRAMP_F_RET_FENTRY_RET))
return -EINVAL;
}
@ -1053,7 +1061,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im
if (!branches)
return -ENOMEM;
if (invoke_bpf_mod_ret(image, ro_image, ctx, fmod_ret, regs_off, retval_off,
if (invoke_bpf_mod_ret(image, ro_image, ctx, im, fmod_ret, regs_off, retval_off,
run_ctx_off, branches)) {
ret = -EINVAL;
goto cleanup;
@ -1090,11 +1098,6 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im
/* Restore updated tail_call_cnt */
if (flags & BPF_TRAMP_F_TAIL_CALL_CTX)
bpf_trampoline_restore_tail_call_cnt(image, ctx, bpf_frame_size, r4_off);
/* Reserve space to patch branch instruction to skip fexit progs */
if (ro_image) /* image is NULL for dummy pass */
im->ip_after_call = &((u32 *)ro_image)[ctx->idx];
EMIT(PPC_RAW_NOP());
}
/* Update branches saved in invoke_bpf_mod_ret with address of do_fexit */
@ -1123,16 +1126,14 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im
cookie_ctx_off--;
}
if (invoke_bpf_prog(image, ro_image, ctx, fexit->nodes[i], regs_off, retval_off,
run_ctx_off, false)) {
if (invoke_bpf_prog(image, ro_image, ctx, im, fexit->nodes[i], regs_off,
retval_off, run_ctx_off, false)) {
ret = -EINVAL;
goto cleanup;
}
}
if (flags & BPF_TRAMP_F_CALL_ORIG) {
if (ro_image) /* image is NULL for dummy pass */
im->ip_epilogue = &((u32 *)ro_image)[ctx->idx];
PPC_LI_ADDR(_R3, im);
ret = bpf_jit_emit_func_call_rel(image, ro_image, ctx,
(unsigned long)__bpf_tramp_exit);
@ -1192,7 +1193,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im
int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags,
struct bpf_tramp_nodes *tnodes, void *func_addr)
{
struct bpf_tramp_image im;
struct bpf_tramp_image im = {};
int ret;
ret = __arch_prepare_bpf_trampoline(&im, NULL, NULL, NULL, m, flags, tnodes, func_addr);
@ -1320,7 +1321,7 @@ int bpf_arch_text_poke(void *ip, enum bpf_text_poke_type old_t,
/*
* If we are not poking at bpf prog entry, then we are simply patching in/out
* an unconditional branch instruction at im->ip_after_call
* an unconditional branch instruction in a trampoline image
*/
if (offset) {
if (old_t == BPF_MOD_CALL || new_t == BPF_MOD_CALL) {

23
arch/powerpc/perf/imc-pmu.c
View File

@ -1275,6 +1275,8 @@ static int trace_imc_prepare_sample(struct trace_imc_data *mem,
struct perf_event_header *header,
struct perf_event *event)
{
u16 misc = 0;
/* Sanity checks for a valid record */
if (be64_to_cpu(READ_ONCE(mem->tb1)) > *prev_tb)
*prev_tb = be64_to_cpu(READ_ONCE(mem->tb1));
@ -1289,23 +1291,19 @@ static int trace_imc_prepare_sample(struct trace_imc_data *mem,
data->ip = be64_to_cpu(READ_ONCE(mem->ip));
data->period = event->hw.last_period;
header->type = PERF_RECORD_SAMPLE;
header->size = sizeof(*header) + event->header_size;
header->misc = 0;
if (cpu_has_feature(CPU_FTR_ARCH_31)) {
switch (IMC_TRACE_RECORD_VAL_HVPR(be64_to_cpu(READ_ONCE(mem->val)))) {
case 0:/* when MSR HV and PR not set in the trace-record */
header->misc |= PERF_RECORD_MISC_GUEST_KERNEL;
misc |= PERF_RECORD_MISC_GUEST_KERNEL;
break;
case 1: /* MSR HV is 0 and PR is 1 */
header->misc |= PERF_RECORD_MISC_GUEST_USER;
misc |= PERF_RECORD_MISC_GUEST_USER;
break;
case 2: /* MSR HV is 1 and PR is 0 */
header->misc |= PERF_RECORD_MISC_KERNEL;
misc |= PERF_RECORD_MISC_KERNEL;
break;
case 3: /* MSR HV is 1 and PR is 1 */
header->misc |= PERF_RECORD_MISC_USER;
misc |= PERF_RECORD_MISC_USER;
break;
default:
pr_info("IMC: Unable to set the flag based on MSR bits\n");
@ -1313,11 +1311,14 @@ static int trace_imc_prepare_sample(struct trace_imc_data *mem,
}
} else {
if (is_kernel_addr(data->ip))
header->misc |= PERF_RECORD_MISC_KERNEL;
misc |= PERF_RECORD_MISC_KERNEL;
else
header->misc |= PERF_RECORD_MISC_USER;
misc |= PERF_RECORD_MISC_USER;
}
perf_event_header__init_id(header, data, event);
perf_event_header__init(header, data,
PERF_RECORD_SAMPLE, misc,
sizeof(*header) + event->header_size,
event);
return 0;
}

1
arch/riscv/kernel/vdso/Makefile
View File

@ -36,6 +36,7 @@ endif
ccflags-y := -fno-stack-protector
ccflags-y += -DDISABLE_BRANCH_PROFILING
ccflags-y += -fno-builtin
ccflags-y += $(CONFIG_CC_OPT_INLINE_MEMSET)
ccflags-y += $(KBUILD_BASE_ISA)$(CFI_MARCH)
ccflags-y += $(CFI_FULL)
asflags-y += $(KBUILD_BASE_ISA)$(CFI_MARCH)

2
arch/riscv/kvm/aia_device.c
View File

@ -237,7 +237,7 @@ static int aia_init(struct kvm *kvm)
return -EBUSY;
/* We might be in the middle of creating a VCPU? */
if (kvm->created_vcpus != atomic_read(&kvm->online_vcpus))
if (kvm_is_vcpu_creation_in_progress(kvm))
return -EBUSY;
/* Number of sources should be less than or equals number of IDs */

56
arch/riscv/net/bpf_jit_comp64.c
View File

@ -822,6 +822,24 @@ static int gen_jump_or_nops(void *target, void *ip, u32 *insns, bool is_call)
return emit_jump_and_link(is_call ? RV_REG_T0 : RV_REG_ZERO, rvoff, false, &ctx);
}
int arch_bpf_trampoline_skip(void *nop, void *target)
{
u32 old_insn = rv_nop();
u32 new_insn = rv_jal(RV_REG_ZERO, (target - nop) >> 1);
int ret;
if (memcmp(nop, &old_insn, sizeof(old_insn)))
return -EFAULT;
cpus_read_lock();
mutex_lock(&text_mutex);
ret = patch_text(nop, &new_insn, sizeof(new_insn));
mutex_unlock(&text_mutex);
cpus_read_unlock();
return ret;
}
int bpf_arch_text_poke(void *ip, enum bpf_text_poke_type old_t,
enum bpf_text_poke_type new_t, void *old_addr,
void *new_addr)
@ -904,12 +922,17 @@ static void emit_store_stack_imm64(u8 reg, int stack_off, u64 imm64,
emit_sd(RV_REG_FP, stack_off, reg, ctx);
}
static int invoke_bpf_prog(struct bpf_tramp_node *node, int args_off, int retval_off,
int run_ctx_off, bool save_ret, struct rv_jit_context *ctx)
static int invoke_bpf_prog(struct bpf_tramp_image *im, struct bpf_tramp_node *node,
int args_off, int retval_off, int run_ctx_off, bool save_ret,
struct rv_jit_context *ctx)
{
int ret, branch_off;
struct bpf_prog *p = node->link->prog;
int cookie_off = offsetof(struct bpf_tramp_run_ctx, bpf_cookie);
void *skip = ctx->ro_insns + ctx->ninsns;
/* nop, patched to a jal over this prog when it is detached */
emit(rv_nop(), ctx);
if (node->cookie)
emit_store_stack_imm64(RV_REG_T1, -run_ctx_off + cookie_off, node->cookie, ctx);
@ -962,13 +985,17 @@ static int invoke_bpf_prog(struct bpf_tramp_node *node, int args_off, int retval
/* arg3: &run_ctx */
emit_addi(RV_REG_A2, RV_REG_FP, -run_ctx_off, ctx);
ret = emit_call((const u64)bpf_trampoline_exit(p), true, ctx);
if (ret)
return ret;
return ret;
bpf_tramp_image_add_skip(im, p, skip, ctx->ro_insns + ctx->ninsns);
return 0;
}
static int invoke_bpf(struct bpf_tramp_nodes *tn, int args_off, int retval_off,
int run_ctx_off, int func_meta_off, bool save_ret, u64 func_meta,
int cookie_off, struct rv_jit_context *ctx)
static int invoke_bpf(struct bpf_tramp_image *im, struct bpf_tramp_nodes *tn,
int args_off, int retval_off, int run_ctx_off, int func_meta_off,
bool save_ret, u64 func_meta, int cookie_off,
struct rv_jit_context *ctx)
{
int i, cur_cookie = (cookie_off - args_off) / 8;
@ -981,8 +1008,8 @@ static int invoke_bpf(struct bpf_tramp_nodes *tn, int args_off, int retval_off,
emit_store_stack_imm64(RV_REG_T1, -func_meta_off, meta, ctx);
cur_cookie--;
}
err = invoke_bpf_prog(tn->nodes[i], args_off, retval_off, run_ctx_off,
save_ret, ctx);
err = invoke_bpf_prog(im, tn->nodes[i], args_off, retval_off,
run_ctx_off, save_ret, ctx);
if (err)
return err;
}
@ -1170,7 +1197,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im,
}
if (fentry->nr_nodes) {
ret = invoke_bpf(fentry, args_off, retval_off, run_ctx_off, func_meta_off,
ret = invoke_bpf(im, fentry, args_off, retval_off, run_ctx_off, func_meta_off,
flags & BPF_TRAMP_F_RET_FENTRY_RET, func_meta, cookie_off, ctx);
if (ret)
return ret;
@ -1184,7 +1211,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im,
/* cleanup to avoid garbage return value confusion */
emit_sd(RV_REG_FP, -retval_off, RV_REG_ZERO, ctx);
for (i = 0; i < fmod_ret->nr_nodes; i++) {
ret = invoke_bpf_prog(fmod_ret->nodes[i], args_off, retval_off,
ret = invoke_bpf_prog(im, fmod_ret->nodes[i], args_off, retval_off,
run_ctx_off, true, ctx);
if (ret)
goto out;
@ -1211,10 +1238,6 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im,
emit_sd(RV_REG_FP, -tcc_off, RV_REG_TCC, ctx);
emit_sd(RV_REG_FP, -retval_off, RV_REG_A0, ctx);
emit_sd(RV_REG_FP, -(retval_off - 8), regmap[BPF_REG_0], ctx);
im->ip_after_call = ctx->ro_insns + ctx->ninsns;
/* 2 nops reserved for auipc+jalr pair */
emit(rv_nop(), ctx);
emit(rv_nop(), ctx);
}
/* update branches saved in invoke_bpf_mod_ret with bnez */
@ -1230,14 +1253,13 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im,
emit_store_stack_imm64(RV_REG_T1, -func_meta_off, func_meta, ctx);
if (fexit->nr_nodes) {
ret = invoke_bpf(fexit, args_off, retval_off, run_ctx_off, func_meta_off,
ret = invoke_bpf(im, fexit, args_off, retval_off, run_ctx_off, func_meta_off,
false, func_meta, cookie_off, ctx);
if (ret)
goto out;
}
if (flags & BPF_TRAMP_F_CALL_ORIG) {
im->ip_epilogue = ctx->ro_insns + ctx->ninsns;
emit_imm(RV_REG_A0, ctx->insns ? (const s64)im : RV_MAX_COUNT_IMM, ctx);
ret = emit_call((const u64)__bpf_tramp_exit, true, ctx);
if (ret)
@ -1299,7 +1321,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im,
int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags,
struct bpf_tramp_nodes *tnodes, void *func_addr)
{
struct bpf_tramp_image im;
struct bpf_tramp_image im = {};
struct rv_jit_context ctx;
int ret;

1
arch/s390/kernel/vdso/Makefile
View File

@ -29,6 +29,7 @@ KBUILD_CFLAGS_VDSO := $(filter-out -munaligned-symbols,$(KBUILD_CFLAGS_VDSO))
KBUILD_CFLAGS_VDSO := $(filter-out -fno-asynchronous-unwind-tables,$(KBUILD_CFLAGS_VDSO))
KBUILD_CFLAGS_VDSO += -fPIC -fno-common -fno-builtin -fasynchronous-unwind-tables
KBUILD_CFLAGS_VDSO += -fno-stack-protector $(DISABLE_KSTACK_ERASE)
KBUILD_CFLAGS_VDSO += $(CONFIG_CC_OPT_INLINE_MEMSET)
ldflags-y := -shared -soname=linux-vdso.so.1 \
--hash-style=both --build-id=sha1 \
$(call ld-option, --eh-frame-hdr) -T

5
arch/s390/kvm/s390/s390.c
View File

@ -3579,12 +3579,11 @@ void kvm_arch_vcpu_put(struct kvm_vcpu *vcpu)
void kvm_arch_vcpu_postcreate(struct kvm_vcpu *vcpu)
{
mutex_lock(&vcpu->kvm->lock);
preempt_disable();
vcpu->arch.sie_block->epoch = vcpu->kvm->arch.epoch;
vcpu->arch.sie_block->epdx = vcpu->kvm->arch.epdx;
preempt_enable();
mutex_unlock(&vcpu->kvm->lock);
if (!kvm_is_ucontrol(vcpu->kvm)) {
vcpu->arch.gmap = vcpu->kvm->arch.gmap;
sca_add_vcpu(vcpu);
@ -3757,13 +3756,11 @@ static int kvm_s390_vcpu_setup(struct kvm_vcpu *vcpu)
kvm_s390_vcpu_pci_setup(vcpu);
mutex_lock(&vcpu->kvm->lock);
if (kvm_s390_pv_is_protected(vcpu->kvm)) {
rc = kvm_s390_pv_create_cpu(vcpu, &uvrc, &uvrrc);
if (rc)
kvm_s390_vcpu_unsetup_cmma(vcpu);
}
mutex_unlock(&vcpu->kvm->lock);
return rc;
}

46
arch/s390/net/bpf_jit_comp.c
View File

@ -2566,6 +2566,8 @@ struct bpf_tramp_jit {
int r14_off; /* Offset of saved %r14, has to be at the
* bottom */
int do_fexit; /* do_fexit: label */
int skip[BPF_MAX_TRAMP_LINKS]; /* skip: labels after each prog */
int nr_progs;
};
static void load_imm64(struct bpf_jit *jit, int dst_reg, u64 val)
@ -2584,6 +2586,7 @@ static void emit_store_stack_imm64(struct bpf_jit *jit, int tmp_reg, int stack_o
}
static int invoke_bpf_prog(struct bpf_tramp_jit *tjit,
struct bpf_tramp_image *im,
const struct btf_func_model *m,
struct bpf_tramp_node *node, bool save_ret)
{
@ -2591,8 +2594,20 @@ static int invoke_bpf_prog(struct bpf_tramp_jit *tjit,
int cookie_off = tjit->run_ctx_off +
offsetof(struct bpf_tramp_run_ctx, bpf_cookie);
struct bpf_prog *p = node->link->prog;
void *skip = jit->prg_buf + jit->prg;
int idx = tjit->nr_progs++;
int patch;
if (idx >= ARRAY_SIZE(tjit->skip))
return -E2BIG;
/*
* nop, patched to skip this prog when it is detached
*/
/* brcl 0,skip */
EMIT6_PCREL_RILC(0xc0040000, 0, tjit->skip[idx]);
/*
* run_ctx.cookie = node->cookie;
*/
@ -2652,10 +2667,15 @@ static int invoke_bpf_prog(struct bpf_tramp_jit *tjit,
/* brasl %r14,__bpf_prog_exit */
EMIT6_PCREL_RILB_PTR(0xc0050000, REG_14, bpf_trampoline_exit(p));
/* skip: */
tjit->skip[idx] = jit->prg;
bpf_tramp_image_add_skip(im, p, skip, jit->prg_buf + jit->prg);
return 0;
}
static int invoke_bpf(struct bpf_tramp_jit *tjit,
struct bpf_tramp_image *im,
const struct btf_func_model *m,
struct bpf_tramp_nodes *tn, bool save_ret,
u64 func_meta, int cookie_off)
@ -2670,7 +2690,7 @@ static int invoke_bpf(struct bpf_tramp_jit *tjit,
emit_store_stack_imm64(jit, REG_0, tjit->func_meta_off, meta);
cur_cookie--;
}
if (invoke_bpf_prog(tjit, m, tn->nodes[i], save_ret))
if (invoke_bpf_prog(tjit, im, m, tn->nodes[i], save_ret))
return -EINVAL;
}
@ -2712,6 +2732,11 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im,
u64 func_meta;
int i, j;
/* The skip labels are taken from the previous pass. */
tjit->nr_progs = 0;
if (im)
im->nr_skips = 0;
/* Support as many stack arguments as "mvc" instruction can handle. */
nr_reg_args = min_t(int, m->nr_args, MAX_NR_REG_ARGS);
nr_stack_args = m->nr_args - nr_reg_args;
@ -2875,7 +2900,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im,
emit_store_stack_imm64(jit, REG_0, tjit->retval_off, 0);
}
if (invoke_bpf(tjit, m, fentry, flags & BPF_TRAMP_F_RET_FENTRY_RET,
if (invoke_bpf(tjit, im, m, fentry, flags & BPF_TRAMP_F_RET_FENTRY_RET,
func_meta, cookie_off))
return -EINVAL;
@ -2889,7 +2914,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im,
0xf000 | tjit->retval_off);
for (i = 0; i < fmod_ret->nr_nodes; i++) {
if (invoke_bpf_prog(tjit, m, fmod_ret->nodes[i], true))
if (invoke_bpf_prog(tjit, im, m, fmod_ret->nodes[i], true))
return -EINVAL;
/*
@ -2943,15 +2968,6 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im,
/* mvc tccnt_off(%r15),tail_call_cnt(4,%r15) */
_EMIT6(0xd203f000 | tjit->tccnt_off,
0xf000 | offsetof(struct prog_frame, tail_call_cnt));
im->ip_after_call = jit->prg_buf + jit->prg;
/*
* The following nop will be patched by bpf_tramp_image_put().
*/
/* brcl 0,im->ip_epilogue */
EMIT6_PCREL_RILC(0xc0040000, 0, (u64)im->ip_epilogue);
}
/* Set the "is_return" flag for fsession. */
@ -2962,12 +2978,10 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im,
/* do_fexit: */
tjit->do_fexit = jit->prg;
if (invoke_bpf(tjit, m, fexit, false, func_meta, cookie_off))
if (invoke_bpf(tjit, im, m, fexit, false, func_meta, cookie_off))
return -EINVAL;
if (flags & BPF_TRAMP_F_CALL_ORIG) {
im->ip_epilogue = jit->prg_buf + jit->prg;
/*
* __bpf_tramp_exit(im);
*/
@ -3016,7 +3030,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im,
int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags,
struct bpf_tramp_nodes *tnodes, void *orig_call)
{
struct bpf_tramp_image im;
struct bpf_tramp_image im = {};
struct bpf_tramp_jit tjit;
int ret;

2
arch/x86/crypto/aesni-intel_glue.c
View File

@ -1293,6 +1293,7 @@ static void gcm_process_assoc(const struct aes_gcm_key *key, u8 ghash_acc[16],
unsigned int len;
const u8 *src = walk.addr;
assoclen -= orig_len_this_step;
if (unlikely(pos)) {
len = min(len_this_step, 16 - pos);
memcpy(&buf[pos], src, len);
@ -1320,7 +1321,6 @@ static void gcm_process_assoc(const struct aes_gcm_key *key, u8 ghash_acc[16],
kernel_fpu_end();
kernel_fpu_begin();
}
assoclen -= orig_len_this_step;
}
if (unlikely(pos))
aes_gcm_aad_update(key, ghash_acc, buf, pos, flags);

2
arch/x86/entry/vdso/vdso64/Makefile
View File

@ -14,7 +14,7 @@ vobjs-$(CONFIG_X86_SGX) += vsgx.o
vobjs-$(CONFIG_FUTEX_ROBUST_UNLOCK) += vfutex.o
# Compilation flags
flags-y := -DBUILD_VDSO64 -m64 -mcmodel=small
flags-y := -DBUILD_VDSO64 -m64 -mcmodel=small $(CONFIG_CC_OPT_INLINE_MEMSET)
# The location of this include matters!
include $(src)/../common/Makefile.include

4
arch/x86/include/uapi/asm/ptrace-abi.h
View File

@ -2,6 +2,7 @@
#ifndef _ASM_X86_PTRACE_ABI_H
#define _ASM_X86_PTRACE_ABI_H
#if defined(__ASSEMBLER__) || defined(__FRAME_OFFSETS)
#ifdef __i386__
#define EBX 0
@ -25,7 +26,6 @@
#else /* __i386__ */
#if defined(__ASSEMBLER__) || defined(__FRAME_OFFSETS)
/*
* C ABI says these regs are callee-preserved. They aren't saved on kernel entry
* unless syscall needs a complete, fully filled "struct pt_regs".
@ -57,12 +57,12 @@
#define EFLAGS 144
#define RSP 152
#define SS 160
#endif /* __ASSEMBLER__ */
/* top of stack page */
#define FRAME_SIZE 168
#endif /* !__i386__ */
#endif /* defined(__ASSEMBLER__) || defined(__FRAME_OFFSETS) */
/* Arbitrarily choose the same ptrace numbers as used by the Sparc code. */
#define PTRACE_GETREGS 12

15
arch/x86/kernel/sys_x86_64.c
View File

@ -50,9 +50,16 @@ static unsigned long get_align_mask(struct file *filp)
* value before calling vm_unmapped_area() or ORed directly to the
* address.
*/
static unsigned long get_align_bits(void)
static unsigned long get_align_bits(struct file *filp)
{
return va_align.bits & get_align_mask(NULL);
/*
* va_align.bits is smaller than the huge page size and will
* lead to misaligned huge pages. Ignore it for huge mappings.
*/
if (is_file_hugepages(filp))
return 0;
return va_align.bits & get_align_mask(filp);
}
static int __init control_va_addr_alignment(char *str)
@ -157,7 +164,7 @@ arch_get_unmapped_area(struct file *filp, unsigned long addr, unsigned long len,
}
if (filp) {
info.align_mask = get_align_mask(filp);
info.align_offset += get_align_bits();
info.align_offset += get_align_bits(filp);
}
return vm_unmapped_area(&info);
@ -222,7 +229,7 @@ arch_get_unmapped_area_topdown(struct file *filp, unsigned long addr0,
if (filp) {
info.align_mask = get_align_mask(filp);
info.align_offset += get_align_bits();
info.align_offset += get_align_bits(filp);
}
addr = vm_unmapped_area(&info);
if (!(addr & ~PAGE_MASK))

6
arch/x86/kvm/mmu/mmu.c
View File

@ -2512,6 +2512,12 @@ static void shadow_walk_init_using_root(struct kvm_shadow_walk_iterator *iterato
iterator->addr = addr;
iterator->shadow_addr = root;
if (WARN_ON_ONCE(!VALID_PAGE(root)) || kvm_mmu_is_dummy_root(root)) {
iterator->level = 0;
return;
}
iterator->level = vcpu->arch.mmu->root_role.level;
if (iterator->level >= PT64_ROOT_4LEVEL &&

57
arch/x86/kvm/svm/sev.c
View File

@ -488,6 +488,20 @@ static void snp_guest_req_cleanup(struct kvm *kvm)
sev->guest_resp_buf = NULL;
}
static int sev_alloc_have_run_cpus(struct kvm_sev_info *sev)
{
if (!zalloc_cpumask_var(&sev->have_run_cpus, GFP_KERNEL_ACCOUNT))
return -ENOMEM;
return 0;
}
static void sev_free_have_run_cpus(struct kvm_sev_info *sev)
{
free_cpumask_var(sev->have_run_cpus);
memset(&sev->have_run_cpus, 0, sizeof(sev->have_run_cpus));
}
static int __sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp,
struct kvm_sev_init *data,
unsigned long vm_type)
@ -545,10 +559,9 @@ static int __sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp,
if (ret)
goto e_free_asid;
if (!zalloc_cpumask_var(&sev->have_run_cpus, GFP_KERNEL_ACCOUNT)) {
ret = -ENOMEM;
ret = sev_alloc_have_run_cpus(sev);
if (ret)
goto e_free_asid;
}
/* This needs to happen after SEV/SNP firmware initialization. */
if (snp_active) {
@ -566,7 +579,7 @@ static int __sev_guest_init(struct kvm *kvm, struct kvm_sev_cmd *argp,
return 0;
e_free:
free_cpumask_var(sev->have_run_cpus);
sev_free_have_run_cpus(sev);
e_free_asid:
argp->error = init_args.error;
sev_asid_free(sev);
@ -1125,9 +1138,6 @@ static int sev_launch_update_vmsa(struct kvm *kvm, struct kvm_sev_cmd *argp)
if (!sev_es_guest(kvm))
return -ENOTTY;
if (kvm_is_vcpu_creation_in_progress(kvm))
return -EBUSY;
ret = kvm_lock_all_vcpus(kvm);
if (ret)
return ret;
@ -2035,6 +2045,13 @@ static void sev_migrate_from(struct kvm *dst_kvm, struct kvm *src_kvm)
struct kvm_sev_info *mirror;
unsigned long i;
/*
* Do cache maintenance on the source VM *before* clearing "SEV active",
* as memory reclaim flows won't trigger cache maintenance on the VM
* once it's no longer an SEV VM.
*/
sev_writeback_caches(src_kvm);
dst->active = true;
dst->asid = src->asid;
dst->handle = src->handle;
@ -2048,12 +2065,6 @@ static void sev_migrate_from(struct kvm *dst_kvm, struct kvm *src_kvm)
src->pages_locked = 0;
src->es_active = false;
/*
* Do cache maintenance on the source VM as it is no longer an SEV VM,
* i.e. memory reclaim flows won't trigger cache maintenance on the VM.
*/
sev_writeback_caches(src_kvm);
list_cut_before(&dst->regions_list, &src->regions_list, &src->regions_list);
mutex_lock(&sev_mirror_lock);
@ -2121,10 +2132,6 @@ static int sev_check_source_vcpus(struct kvm *dst, struct kvm *src)
struct kvm_vcpu *src_vcpu;
unsigned long i;
if (kvm_is_vcpu_creation_in_progress(src) ||
kvm_is_vcpu_creation_in_progress(dst))
return -EBUSY;
if (!sev_es_guest(src))
return 0;
@ -2198,10 +2205,9 @@ int sev_vm_move_enc_context_from(struct kvm *kvm, unsigned int source_fd)
* does not, i.e. KVM could skip flushes if memory is reclaimed from
* the old VM but not the new VM.
*/
if (!zalloc_cpumask_var(&dst_sev->have_run_cpus, GFP_KERNEL_ACCOUNT)) {
ret = -ENOMEM;
ret = sev_alloc_have_run_cpus(dst_sev);
if (ret)
goto out_source_vcpu;
}
sev_migrate_from(kvm, source_kvm);
kvm_vm_dead(source_kvm);
@ -2520,9 +2526,6 @@ static int snp_launch_update_vmsa(struct kvm *kvm, struct kvm_sev_cmd *argp)
unsigned long i;
int ret;
if (kvm_is_vcpu_creation_in_progress(kvm))
return -EBUSY;
ret = kvm_lock_all_vcpus(kvm);
if (ret)
return ret;
@ -2898,10 +2901,9 @@ int sev_vm_copy_enc_context_from(struct kvm *kvm, unsigned int source_fd)
}
mirror_sev = to_kvm_sev_info(kvm);
if (!zalloc_cpumask_var(&mirror_sev->have_run_cpus, GFP_KERNEL_ACCOUNT)) {
ret = -ENOMEM;
ret = sev_alloc_have_run_cpus(mirror_sev);
if (ret)
goto e_unlock;
}
/*
* The mirror kvm holds an enc_context_owner ref so its asid can't
@ -2994,7 +2996,7 @@ void sev_vm_destroy(struct kvm *kvm)
* Free the mask even if the VM is not *currently* an SEV VM, as it may
* have been an SEV VM prior to intra-host migration.
*/
free_cpumask_var(sev->have_run_cpus);
sev_free_have_run_cpus(sev);
if (!sev_guest(kvm))
return;
@ -3630,7 +3632,6 @@ int pre_sev_run(struct vcpu_svm *svm, int cpu)
sd->sev_vmcbs[asid] = svm->vmcb;
svm->vmcb->control.tlb_ctl = TLB_CONTROL_FLUSH_ASID;
vmcb_mark_dirty(svm->vmcb, VMCB_ASID);
return 0;
}

43
arch/x86/kvm/svm/svm.c
View File

@ -673,16 +673,7 @@ static void clr_dr_intercepts(struct vcpu_svm *svm)
static bool msr_write_intercepted(struct vcpu_svm *svm, u32 msr)
{
/*
* For non-nested case:
* If the L01 MSR bitmap does not intercept the MSR, then we need to
* save it.
*
* For nested case:
* If the L02 MSR bitmap does not intercept the MSR, then we need to
* save it.
*/
void *msrpm = is_guest_mode(&svm->vcpu) ? svm->nested.msrpm : svm->msrpm;
void *msrpm = __va(__sme_clr(svm->vmcb->control.msrpm_base_pa));
return svm_test_msr_bitmap_write(msrpm, msr);
}
@ -1902,8 +1893,7 @@ static void new_asid(struct vcpu_svm *svm, struct svm_cpu_data *sd)
if (sd->next_asid > sd->max_asid) {
++sd->asid_generation;
sd->next_asid = sd->min_asid;
svm->vmcb->control.tlb_ctl = TLB_CONTROL_FLUSH_ALL_ASID;
vmcb_mark_dirty(svm->vmcb, VMCB_ASID);
sd->flush_all_asids = true;
}
svm->current_vmcb->asid_generation = sd->asid_generation;
@ -4528,6 +4518,9 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu, u64 run_flags)
svm->vmcb->control.asid = svm->asid;
vmcb_mark_dirty(svm->vmcb, VMCB_ASID);
}
if (this_cpu_ptr(&svm_data)->flush_all_asids)
svm->vmcb->control.tlb_ctl = TLB_CONTROL_FLUSH_ALL_ASID;
svm->vmcb->save.cr2 = vcpu->arch.cr2;
if (guest_cpu_cap_has(vcpu, X86_FEATURE_ERAPS) &&
@ -4618,16 +4611,23 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu, u64 run_flags)
vcpu->arch.nested_run_pending = 0;
}
svm->vmcb->control.tlb_ctl = TLB_CONTROL_DO_NOTHING;
if (!svm_is_vmrun_failure(svm->vmcb->control.exit_code)) {
this_cpu_ptr(&svm_data)->flush_all_asids = false;
svm->vmcb->control.tlb_ctl = TLB_CONTROL_DO_NOTHING;
/*
* Unconditionally mask off the CLEAR_RAP bit, the AND is just as cheap
* as the TEST+Jcc to avoid it.
*/
if (cpu_feature_enabled(X86_FEATURE_ERAPS))
svm->vmcb->control.erap_ctl &= ~ERAP_CONTROL_CLEAR_RAP;
/*
* Unconditionally mask off the CLEAR_RAP bit, the AND is just
* as cheap as the TEST+Jcc to avoid it.
*/
if (cpu_feature_enabled(X86_FEATURE_ERAPS))
svm->vmcb->control.erap_ctl &= ~ERAP_CONTROL_CLEAR_RAP;
vmcb_mark_all_clean(svm->vmcb);
vmcb_mark_all_clean(svm->vmcb);
if (!msr_write_intercepted(svm, MSR_AMD64_PERF_CNTR_GLOBAL_CTL))
rdmsrq(MSR_AMD64_PERF_CNTR_GLOBAL_CTL,
vcpu_to_pmu(vcpu)->global_ctrl);
}
/* if exit due to PF check for async PF */
if (svm->vmcb->control.exit_code == SVM_EXIT_EXCP_BASE + PF_VECTOR)
@ -4636,9 +4636,6 @@ static __no_kcsan fastpath_t svm_vcpu_run(struct kvm_vcpu *vcpu, u64 run_flags)
kvm_clear_available_registers(vcpu, SVM_REGS_LAZY_LOAD_SET);
if (!msr_write_intercepted(svm, MSR_AMD64_PERF_CNTR_GLOBAL_CTL))
rdmsrq(MSR_AMD64_PERF_CNTR_GLOBAL_CTL, vcpu_to_pmu(vcpu)->global_ctrl);
trace_kvm_exit(vcpu, KVM_ISA_SVM);
svm_complete_interrupts(vcpu);

1
arch/x86/kvm/svm/svm.h
View File

@ -376,6 +376,7 @@ struct svm_cpu_data {
u32 next_asid;
u32 min_asid;
bool flush_all_asids;
bool bp_spec_reduce_set;
struct vmcb *save_area;

6
arch/x86/kvm/vmx/nested.c
View File

@ -1754,6 +1754,9 @@ static void copy_vmcs12_to_shadow(struct vcpu_vmx *vmx)
static void copy_enlightened_to_vmcs12(struct vcpu_vmx *vmx, u32 hv_clean_fields)
{
#ifdef CONFIG_KVM_HYPERV
const u64 runtime_controls = HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_GRP1 |
HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_GRP2 |
HV_VMX_ENLIGHTENED_CLEAN_FIELD_CONTROL_PROC;
struct vmcs12 *vmcs12 = vmx->nested.cached_vmcs12;
struct hv_enlightened_vmcs *evmcs = nested_vmx_evmcs(vmx);
struct kvm_vcpu_hv *hv_vcpu = to_hv_vcpu(&vmx->vcpu);
@ -1762,6 +1765,9 @@ static void copy_enlightened_to_vmcs12(struct vcpu_vmx *vmx, u32 hv_clean_fields
vmcs12->tpr_threshold = evmcs->tpr_threshold;
vmcs12->guest_rip = evmcs->guest_rip;
if ((hv_clean_fields & runtime_controls) != runtime_controls)
vmx->nested.force_msr_bitmap_recalc = true;
if (unlikely(!(hv_clean_fields &
HV_VMX_ENLIGHTENED_CLEAN_FIELD_ENLIGHTENMENTSCONTROL))) {
hv_vcpu->nested.pa_page_gpa = evmcs->partition_assist_page;

5
arch/x86/kvm/vmx/tdx.c
View File

@ -2728,11 +2728,6 @@ static tdx_vm_state_guard_t tdx_acquire_vm_state_locks(struct kvm *kvm)
mutex_lock(&kvm->lock);
if (kvm->created_vcpus != atomic_read(&kvm->online_vcpus)) {
r = -EBUSY;
goto out_err;
}
r = kvm_lock_all_vcpus(kvm);
if (r)
goto out_err;

38
arch/x86/mm/pgtable.c
View File

@ -705,47 +705,37 @@ int pmd_clear_huge(pmd_t *pmd)
}
#ifdef CONFIG_X86_64
/**
* pud_free_pmd_page - Clear PUD entry and free PMD page
* @pud: Pointer to a PUD
* @addr: Virtual address associated with PUD
*
* Context: The PUD range has been unmapped and TLB purged.
* Return: 1 if clearing the entry succeeded. 0 otherwise.
*
* NOTE: Callers must allow a single page allocation.
/*
* Given a PUD poitner, detach and free the pointed-to
* PMD page and any PTE page children. The entire range
* under the PUD must not have any valid translations
* and the TLB must have already been flushed.
*/
int pud_free_pmd_page(pud_t *pud, unsigned long addr)
{
pmd_t *pmd, *pmd_sv;
struct ptdesc *pt;
pmd_t *pmd;
int i;
pmd = pud_pgtable(*pud);
pmd_sv = (pmd_t *)__get_free_page(GFP_KERNEL);
if (!pmd_sv)
return 0;
for (i = 0; i < PTRS_PER_PMD; i++) {
pmd_sv[i] = pmd[i];
if (!pmd_none(pmd[i]))
pmd_clear(&pmd[i]);
}
/* Detach the PMD page: */
pud_clear(pud);
/* INVLPG to clear all paging-structure caches */
/*
* PMD and all its descendents are unreachable
* via normal page walks. Make them unreachable
* in cached mid-level walks too:
*/
flush_tlb_kernel_range(addr, addr + PAGE_SIZE-1);
for (i = 0; i < PTRS_PER_PMD; i++) {
if (!pmd_none(pmd_sv[i])) {
pt = page_ptdesc(pmd_page(pmd_sv[i]));
if (!pmd_none(pmd[i])) {
pt = page_ptdesc(pmd_page(pmd[i]));
pagetable_dtor_free(pt);
}
}
free_page((unsigned long)pmd_sv);
pmd_free(&init_mm, pmd);
return 1;

26
arch/x86/net/bpf_jit_comp.c
View File

@ -3217,16 +3217,21 @@ static void restore_regs(const struct btf_func_model *m, u8 **prog,
}
static int invoke_bpf_prog(const struct btf_func_model *m, u8 **pprog,
struct bpf_tramp_image *im,
struct bpf_tramp_node *node, int stack_size,
int run_ctx_off, bool save_ret,
void *image, void *rw_image)
{
u8 *prog = *pprog;
u8 *jmp_insn;
u8 *jmp_insn, *skip;
int ctx_cookie_off = offsetof(struct bpf_tramp_run_ctx, bpf_cookie);
struct bpf_prog *p = node->link->prog;
u64 cookie = node->cookie;
/* nop, patched to skip this prog when it is detached */
skip = image + (prog - (u8 *)rw_image);
emit_nops(&prog, X86_PATCH_SIZE);
/* mov rdi, cookie */
emit_mov_imm64(&prog, BPF_REG_1, (long) cookie >> 32, (u32) (long) cookie);
@ -3301,6 +3306,8 @@ static int invoke_bpf_prog(const struct btf_func_model *m, u8 **pprog,
if (emit_rsb_call(&prog, bpf_trampoline_exit(p), image + (prog - (u8 *)rw_image)))
return -EINVAL;
bpf_tramp_image_add_skip(im, p, skip, image + (prog - (u8 *)rw_image));
*pprog = prog;
return 0;
}
@ -3332,6 +3339,7 @@ static int emit_cond_near_jump(u8 **pprog, void *func, void *ip, u8 jmp_cond)
}
static int invoke_bpf(const struct btf_func_model *m, u8 **pprog,
struct bpf_tramp_image *im,
struct bpf_tramp_nodes *tl, int stack_size,
int run_ctx_off, int func_meta_off, bool save_ret,
void *image, void *rw_image, u64 func_meta,
@ -3346,7 +3354,7 @@ static int invoke_bpf(const struct btf_func_model *m, u8 **pprog,
func_meta | (cur_cookie << BPF_TRAMP_COOKIE_INDEX_SHIFT));
cur_cookie--;
}
if (invoke_bpf_prog(m, &prog, tl->nodes[i], stack_size,
if (invoke_bpf_prog(m, &prog, im, tl->nodes[i], stack_size,
run_ctx_off, save_ret, image, rw_image))
return -EINVAL;
}
@ -3355,6 +3363,7 @@ static int invoke_bpf(const struct btf_func_model *m, u8 **pprog,
}
static int invoke_bpf_mod_ret(const struct btf_func_model *m, u8 **pprog,
struct bpf_tramp_image *im,
struct bpf_tramp_nodes *tl, int stack_size,
int run_ctx_off, u8 **branches,
void *image, void *rw_image)
@ -3368,7 +3377,7 @@ static int invoke_bpf_mod_ret(const struct btf_func_model *m, u8 **pprog,
emit_mov_imm32(&prog, false, BPF_REG_0, 0);
emit_stx(&prog, BPF_DW, BPF_REG_FP, BPF_REG_0, -8);
for (i = 0; i < tl->nr_nodes; i++) {
if (invoke_bpf_prog(m, &prog, tl->nodes[i], stack_size, run_ctx_off, true,
if (invoke_bpf_prog(m, &prog, im, tl->nodes[i], stack_size, run_ctx_off, true,
image, rw_image))
return -EINVAL;
@ -3640,7 +3649,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im
}
if (fentry->nr_nodes) {
if (invoke_bpf(m, &prog, fentry, regs_off, run_ctx_off, func_meta_off,
if (invoke_bpf(m, &prog, im, fentry, regs_off, run_ctx_off, func_meta_off,
flags & BPF_TRAMP_F_RET_FENTRY_RET, image, rw_image,
func_meta, cookie_off))
return -EINVAL;
@ -3652,7 +3661,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im
if (!branches)
return -ENOMEM;
if (invoke_bpf_mod_ret(m, &prog, fmod_ret, regs_off,
if (invoke_bpf_mod_ret(m, &prog, im, fmod_ret, regs_off,
run_ctx_off, branches, image, rw_image)) {
ret = -EINVAL;
goto cleanup;
@ -3682,8 +3691,6 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im
}
/* remember return value in a stack for bpf prog to access */
emit_stx(&prog, BPF_DW, BPF_REG_FP, BPF_REG_0, -8);
im->ip_after_call = image + (prog - (u8 *)rw_image);
emit_nops(&prog, X86_PATCH_SIZE);
}
if (fmod_ret->nr_nodes) {
@ -3708,7 +3715,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im
emit_store_stack_imm64(&prog, BPF_REG_0, -func_meta_off, func_meta);
if (fexit->nr_nodes) {
if (invoke_bpf(m, &prog, fexit, regs_off, run_ctx_off, func_meta_off,
if (invoke_bpf(m, &prog, im, fexit, regs_off, run_ctx_off, func_meta_off,
false, image, rw_image, func_meta, cookie_off)) {
ret = -EINVAL;
goto cleanup;
@ -3723,7 +3730,6 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im
* restored to R0.
*/
if (flags & BPF_TRAMP_F_CALL_ORIG) {
im->ip_epilogue = image + (prog - (u8 *)rw_image);
/* arg1: mov rdi, im */
emit_mov_imm64(&prog, BPF_REG_1, (long) im >> 32, (u32) (long) im);
if (emit_rsb_call(&prog, __bpf_tramp_exit, image + (prog - (u8 *)rw_image))) {
@ -3811,7 +3817,7 @@ int arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *image, void *i
int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags,
struct bpf_tramp_nodes *tnodes, void *func_addr)
{
struct bpf_tramp_image im;
struct bpf_tramp_image im = {};
void *image;
int ret;

4
arch/x86/um/asm/ptrace.h
View File

@ -13,9 +13,7 @@ enum {
};
#include <linux/compiler.h>
#ifndef CONFIG_X86_32
#define __FRAME_OFFSETS /* Needed to get the R* macros */
#endif
#define __FRAME_OFFSETS /* Needed to get the register macros */
#include <asm/ptrace-generic.h>
#define user_mode(r) UPT_IS_USER(&(r)->regs)

1
arch/x86/um/ptrace_32.c
View File

@ -7,6 +7,7 @@
#include <linux/sched.h>
#include <linux/uaccess.h>
#include <linux/regset.h>
#define __FRAME_OFFSETS
#include <asm/ptrace-abi.h>
#include <registers.h>
#include <skas.h>

5
block/blk-cgroup.c
View File

@ -2091,6 +2091,7 @@ static inline struct blkcg_gq *blkg_tryget_closest(struct bio *bio,
struct request_queue *q = bio->bi_bdev->bd_queue;
struct blkcg *blkcg = css_to_blkcg(css);
struct blkcg_gq *blkg;
unsigned long flags;
rcu_read_lock();
blkg = blkg_lookup(blkcg, q);
@ -2105,11 +2106,11 @@ static inline struct blkcg_gq *blkg_tryget_closest(struct bio *bio,
* Fast path failed, we're probably issuing IO in this cgroup the first
* time, hold lock to create new blkg.
*/
spin_lock_irq(&q->queue_lock);
spin_lock_irqsave(&q->queue_lock, flags);
blkg = blkg_lookup_create(blkcg, bio->bi_bdev->bd_disk);
if (blkg)
blkg = blkg_lookup_tryget(blkg);
spin_unlock_irq(&q->queue_lock);
spin_unlock_irqrestore(&q->queue_lock, flags);
return blkg;
}

63
block/blk-mq.c
View File

@ -447,16 +447,6 @@ static struct request *blk_mq_rq_ctx_init(struct blk_mq_alloc_data *data,
WRITE_ONCE(rq->deadline, 0);
req_ref_set(rq, 1);
if (rq->rq_flags & RQF_USE_SCHED) {
struct elevator_queue *e = data->q->elevator;
INIT_HLIST_NODE(&rq->hash);
RB_CLEAR_NODE(&rq->rb_node);
if (e->type->ops.prepare_request)
e->type->ops.prepare_request(rq);
}
return rq;
}
@ -498,6 +488,12 @@ __blk_mq_alloc_requests_batch(struct blk_mq_alloc_data *data)
return rq_list_pop(data->cached_rqs);
}
static bool blk_op_bypass_sched(blk_opf_t opf)
{
return (opf & REQ_OP_MASK) == REQ_OP_FLUSH ||
blk_op_is_passthrough(opf);
}
static void blk_mq_limit_depth(struct blk_mq_alloc_data *data)
{
struct elevator_mq_ops *ops;
@ -518,12 +514,10 @@ static void blk_mq_limit_depth(struct blk_mq_alloc_data *data)
* Flush/passthrough requests are special and go directly to the
* dispatch list, they are not subject to the async_depth limit.
*/
if ((data->cmd_flags & REQ_OP_MASK) == REQ_OP_FLUSH ||
blk_op_is_passthrough(data->cmd_flags))
if (blk_op_bypass_sched(data->cmd_flags))
return;
WARN_ON_ONCE(data->flags & BLK_MQ_REQ_RESERVED);
data->rq_flags |= RQF_USE_SCHED;
/*
* By default, sync requests have no limit, and async requests are
@ -534,6 +528,29 @@ static void blk_mq_limit_depth(struct blk_mq_alloc_data *data)
ops->limit_depth(data->cmd_flags, data);
}
/*
* Finish initializing a request once it has been claimed for an operation.
* Cached requests are allocated before that operation is known.
*/
static void blk_mq_rq_late_init(struct request *rq, u64 alloc_time_ns)
{
struct elevator_queue *e;
blk_mq_rq_time_init(rq, alloc_time_ns);
if (!(rq->rq_flags & RQF_SCHED_TAGS) || (rq->rq_flags & RQF_RESV) ||
blk_op_bypass_sched(rq->cmd_flags))
return;
rq->rq_flags |= RQF_USE_SCHED;
INIT_HLIST_NODE(&rq->hash);
RB_CLEAR_NODE(&rq->rb_node);
e = rq->q->elevator;
if (e->type->ops.prepare_request)
e->type->ops.prepare_request(rq);
}
static struct request *__blk_mq_alloc_requests(struct blk_mq_alloc_data *data)
{
struct request_queue *q = data->q;
@ -562,7 +579,7 @@ static struct request *__blk_mq_alloc_requests(struct blk_mq_alloc_data *data)
if (data->nr_tags > 1) {
rq = __blk_mq_alloc_requests_batch(data);
if (rq) {
blk_mq_rq_time_init(rq, alloc_time_ns);
blk_mq_rq_late_init(rq, alloc_time_ns);
return rq;
}
data->nr_tags = 1;
@ -590,7 +607,7 @@ static struct request *__blk_mq_alloc_requests(struct blk_mq_alloc_data *data)
if (!(data->rq_flags & RQF_SCHED_TAGS))
blk_mq_inc_active_requests(data->hctx);
rq = blk_mq_rq_ctx_init(data, blk_mq_tags_from_data(data), tag);
blk_mq_rq_time_init(rq, alloc_time_ns);
blk_mq_rq_late_init(rq, alloc_time_ns);
return rq;
}
@ -637,8 +654,6 @@ static struct request *blk_mq_alloc_cached_request(struct request_queue *q,
if (plug->nr_ios == 1)
return NULL;
rq = blk_mq_rq_cache_fill(q, plug, opf, flags);
if (!rq)
return NULL;
} else {
rq = rq_list_peek(&plug->cached_rqs);
if (!rq || rq->q != q)
@ -646,15 +661,13 @@ static struct request *blk_mq_alloc_cached_request(struct request_queue *q,
if (blk_mq_get_hctx_type(opf) != rq->mq_hctx->type)
return NULL;
if (op_is_flush(rq->cmd_flags) != op_is_flush(opf))
return NULL;
rq_list_pop(&plug->cached_rqs);
blk_mq_rq_time_init(rq, blk_time_get_ns());
rq->cmd_flags = opf;
INIT_LIST_HEAD(&rq->queuelist);
blk_mq_rq_late_init(rq, blk_time_get_ns());
}
rq->cmd_flags = opf;
INIT_LIST_HEAD(&rq->queuelist);
return rq;
}
@ -766,7 +779,7 @@ struct request *blk_mq_alloc_request_hctx(struct request_queue *q,
if (!(data.rq_flags & RQF_SCHED_TAGS))
blk_mq_inc_active_requests(data.hctx);
rq = blk_mq_rq_ctx_init(&data, blk_mq_tags_from_data(&data), tag);
blk_mq_rq_time_init(rq, alloc_time_ns);
blk_mq_rq_late_init(rq, alloc_time_ns);
rq->__data_len = 0;
rq->phys_gap_bit = 0;
rq->__sector = (sector_t) -1;
@ -3060,8 +3073,6 @@ static struct request *blk_mq_get_cached_request(struct blk_plug *plug,
if (type != rq->mq_hctx->type &&
(type != HCTX_TYPE_READ || rq->mq_hctx->type != HCTX_TYPE_DEFAULT))
return NULL;
if (op_is_flush(rq->cmd_flags) != op_is_flush(opf))
return NULL;
rq_list_pop(&plug->cached_rqs);
return rq;
}
@ -3163,9 +3174,9 @@ void blk_mq_submit_bio(struct bio *bio)
new_request:
if (rq) {
rq_qos_throttle(rq->q, bio);
blk_mq_rq_time_init(rq, blk_time_get_ns());
rq->cmd_flags = bio->bi_opf;
INIT_LIST_HEAD(&rq->queuelist);
blk_mq_rq_late_init(rq, blk_time_get_ns());
} else {
rq = blk_mq_get_new_requests(q, plug, bio);
if (unlikely(!rq)) {

5
block/fops.c
View File

@ -402,6 +402,11 @@ static ssize_t blkdev_direct_IO(struct kiocb *iocb, struct iov_iter *iter)
if (blkdev_dio_invalid(bdev, iocb, iter))
return -EINVAL;
/* HIPRI needs private as bio; HAS_METADATA keeps it as uio_meta */
if ((iocb->ki_flags & IOCB_HIPRI) &&
(iocb->ki_flags & IOCB_HAS_METADATA))
return -EOPNOTSUPP;
if (iov_iter_rw(iter) == WRITE) {
u16 max_write_streams = bdev_max_write_streams(bdev);

2
block/mq-deadline.c
View File

@ -685,7 +685,7 @@ static void dd_insert_requests(struct blk_mq_hw_ctx *hctx,
blk_mq_free_requests(&free);
}
/* Callback from inside blk_mq_rq_ctx_init(). */
/* Callback from inside blk_mq_rq_late_init(). */
static void dd_prepare_request(struct request *rq)
{
rq->elv.priv[0] = NULL;

51
crypto/aes.c
View File

@ -637,7 +637,18 @@ static struct skcipher_alg skcipher_algs[] = {
.decrypt = crypto_aes_cbc_decrypt,
},
#endif
#if IS_ENABLED(CONFIG_CRYPTO_CTS)
/*
* Don't register library-based "cts(cbc(aes))" on architectures where
* it might block a "better" implementation from being instantiated via
* the "cts" template. These exclusions are temporary and will go away
* as the arch-optimized AES code is migrated into the library.
*/
#if IS_ENABLED(CONFIG_CRYPTO_CTS) && \
!(IS_ENABLED(CONFIG_ARM) || \
IS_ENABLED(CONFIG_ARM64) || \
IS_ENABLED(CONFIG_PPC) || \
IS_ENABLED(CONFIG_S390) || \
IS_ENABLED(CONFIG_SPARC))
{
.base.cra_name = "cts(cbc(aes))",
.base.cra_driver_name = "cts-cbc-aes-lib",
@ -687,7 +698,13 @@ static struct skcipher_alg skcipher_algs[] = {
.decrypt = crypto_aes_xctr_crypt,
},
#endif
#if IS_ENABLED(CONFIG_CRYPTO_XTS)
/*
* Don't register library-based "xts(aes)" on architectures where it
* might block a "better" implementation from being instantiated via the
* "xts" template. This exclusion is temporary and will go away when
* the library AES-XTS is optimized for SPARC.
*/
#if IS_ENABLED(CONFIG_CRYPTO_XTS) && !IS_ENABLED(CONFIG_SPARC)
{
.base.cra_name = "xts(aes)",
.base.cra_driver_name = "xts-aes-lib",
@ -980,7 +997,20 @@ static __maybe_unused int crypto_aes_ccm_decrypt(struct aead_request *req)
}
static struct aead_alg aead_algs[] = {
#if IS_ENABLED(CONFIG_CRYPTO_GCM)
/*
* Don't register library-based "gcm(aes)" and "rfc4106(gcm(aes))" on
* architectures where they might block a "better" implementation from
* being instantiated via the "gcm" and "rfc4106" templates. These
* exclusions are temporary and will go away as the arch-optimized AES
* code is migrated into the library.
*/
#if IS_ENABLED(CONFIG_CRYPTO_GCM) && \
!(IS_ENABLED(CONFIG_ARM) || \
IS_ENABLED(CONFIG_ARM64) || \
IS_ENABLED(CONFIG_PPC) || \
IS_ENABLED(CONFIG_RISCV) || \
IS_ENABLED(CONFIG_S390) || \
IS_ENABLED(CONFIG_SPARC))
{
.base.cra_name = "gcm(aes)",
.base.cra_driver_name = "gcm-aes-lib",
@ -1012,7 +1042,20 @@ static struct aead_alg aead_algs[] = {
.chunksize = AES_BLOCK_SIZE,
},
#endif /* CONFIG_CRYPTO_GCM */
#if IS_ENABLED(CONFIG_CRYPTO_CCM)
/*
* Don't register library-based "ccm(aes)" on architectures where it
* might block a "better" implementation from being instantiated via the
* "ccm" template. These exclusions are temporary and will go away as
* the arch-optimized AES code is migrated into the library.
*/
#if IS_ENABLED(CONFIG_CRYPTO_CCM) && \
!(IS_ENABLED(CONFIG_ARM) || \
IS_ENABLED(CONFIG_ARM64) || \
IS_ENABLED(CONFIG_PPC) || \
IS_ENABLED(CONFIG_RISCV) || \
IS_ENABLED(CONFIG_S390) || \
IS_ENABLED(CONFIG_SPARC) || \
IS_ENABLED(CONFIG_X86))
{
.base.cra_name = "ccm(aes)",
.base.cra_driver_name = "ccm-aes-lib",

3
drivers/android/binder.c
View File

@ -2930,8 +2930,9 @@ static int binder_proc_transaction(struct binder_transaction *t,
t_outdated->buffer = NULL;
buffer->transaction = NULL;
trace_binder_transaction_update_buffer_release(buffer);
binder_release_entire_buffer(proc, NULL, buffer, false);
binder_release_entire_buffer(proc, NULL, buffer, true);
binder_alloc_free_buf(&proc->alloc, buffer);
binder_free_txn_fixups(t_outdated);
kfree(t_outdated);
binder_stats_deleted(BINDER_STAT_TRANSACTION);
}

20
drivers/android/binder/node.rs
View File

@ -51,9 +51,9 @@
/// about to drop the weak reference, then the strong increment could be processed after the
/// other thread has already exited, which would be too late.
///
/// Note that trying to create a `ListArc` to the node can succeed even if `has_normal_push` is
/// Note that trying to create a `ListArc` to the node can succeed even if `has_pushed_node` is
/// set. This is because another thread might just have popped the node from a todo list, but not
/// yet called `do_work`. However, if `has_normal_push` is false, then creating a `ListArc` should
/// yet called `do_work`. However, if `has_pushed_node` is false, then creating a `ListArc` should
/// always succeed.
///
/// Like the other fields in `NodeInner`, the delivery state is protected by the process lock.
@ -738,7 +738,21 @@ fn do_work(
self.do_work_locked(writer, owner_inner)
}
fn cancel(self: DArc<Self>) {}
fn cancel(self: DArc<Self>) {
let _drop_outside_lock;
let mut owner_inner = self.owner.inner.lock();
// We only do something on BINDER_THREAD_EXIT, not process exit.
if owner_inner.is_dead {
return;
}
// If BINDER_THREAD_EXIT is invoked on a thread with a pending node refcount update, we
// should move ourselves to ensure the refcount update is still delivered.
if let Some(node) = ListArc::try_from_arc_borrow(self.as_arc_borrow()) {
_drop_outside_lock = owner_inner.push_work(&self.owner, node);
}
}
fn should_sync_wakeup(&self) -> bool {
false

34
drivers/android/binder/node/wrapper.rs
View File

@ -57,7 +57,39 @@ fn do_work(
node.do_work_locked(writer, owner_inner)
}
fn cancel(self: DArc<Self>) {}
fn cancel(self: DArc<Self>) {
let _drop_outside_lock;
let node = &self.node;
let mut owner_inner = node.owner.inner.lock();
// We only do something on BINDER_THREAD_EXIT, not process exit.
if owner_inner.is_dead {
return;
}
// We transfer the responsibility of the node refcount update to the scheduled Node because
// NodeWrapper has no way to re-create the ListArc.
let inner = node.inner.access_mut(&mut owner_inner);
let ds = &mut inner.delivery_state;
assert!(ds.has_pushed_wrapper);
assert!(ds.has_strong_zero2one);
ds.has_pushed_wrapper = false;
// We are changing the state to one where the Node is the strong zero2one update instead of
// the wrapper.
ds.has_weak_zero2one = false;
if !ds.has_pushed_node {
if let Some(node2) = ListArc::try_from_arc_borrow(node.as_arc_borrow()) {
ds.has_pushed_node = true;
_drop_outside_lock = owner_inner.push_work(&node.owner, node2);
} else {
// This can't actually happen.
ds.has_strong_zero2one = false;
}
}
}
fn should_sync_wakeup(&self) -> bool {
false

8
drivers/android/binder/rust_binderfs.c
View File

@ -72,6 +72,7 @@ struct binder_features {
bool oneway_spam_detection;
bool extended_error;
bool freeze_notification;
bool transaction_report;
};
static const struct constant_table binderfs_param_stats[] = {
@ -89,6 +90,7 @@ static struct binder_features binder_features = {
.oneway_spam_detection = true,
.extended_error = true,
.freeze_notification = true,
.transaction_report = true,
};
static inline struct binderfs_info *BINDERFS_SB(const struct super_block *sb)
@ -566,6 +568,12 @@ static int init_binder_features(struct super_block *sb)
if (IS_ERR(dentry))
return PTR_ERR(dentry);
dentry = rust_binderfs_create_file(dir, "transaction_report",
&binder_features_fops,
&binder_features.transaction_report);
if (IS_ERR(dentry))
return PTR_ERR(dentry);
return 0;
}

8
drivers/android/binder/thread.rs
View File

@ -686,6 +686,12 @@ pub(crate) fn push_return_work(&self, reply: u32) {
self.inner.lock().push_return_work(reply);
}
pub(crate) fn pop_work_even_if_deferred(&self) -> Option<DLArc<dyn DeliverToRead>> {
let mut thread_inner = self.inner.lock();
thread_inner.process_work_list = true;
thread_inner.pop_work()
}
fn translate_object(
&self,
obj_index: usize,
@ -1678,7 +1684,7 @@ pub(crate) fn release(self: &Arc<Self>) {
self.unwind_transaction_stack();
// Cancel all pending work items.
while let Ok(Some(work)) = self.get_work_local(false) {
while let Some(work) = self.pop_work_even_if_deferred() {
work.into_arc().cancel();
}
}

3
drivers/android/binderfs.c
View File

@ -189,12 +189,11 @@ static int binderfs_binder_device_create(struct inode *ref_inode,
goto err;
}
inode->i_private = device;
binder_add_device(device);
d_make_persistent(dentry, inode);
fsnotify_create(root->d_inode, dentry);
simple_done_creating(dentry);
binder_add_device(device);
return 0;
err:

4
drivers/block/drbd/drbd_nl_gen.c
View File

@ -638,10 +638,6 @@ const struct genl_split_ops drbd_nl_ops[32] = {
},
};
static const struct genl_multicast_group drbd_nl_mcgrps[] = {
[DRBD_NLGRP_EVENTS] = { "events", },
};
static int __drbd_cfg_context_from_attrs(struct drbd_cfg_context *s,
struct nlattr ***ret_nested_attribute_table,
struct genl_info *info)

1
drivers/block/virtio_blk.c
View File

@ -743,6 +743,7 @@ static int virtblk_read_zoned_limits(struct virtio_blk *vblk,
dev_warn(&vdev->dev, "zero write granularity reported\n");
return -ENODEV;
}
lim->zone_write_granularity = wg;
lim->physical_block_size = wg;
lim->io_min = wg;

15
drivers/bluetooth/btintel.c
View File

@ -289,7 +289,8 @@ void btintel_hw_error(struct hci_dev *hdev, u8 code)
goto unlock;
}
bt_dev_err(hdev, "Exception info %s", (char *)(skb->data + 1));
bt_dev_err(hdev, "Exception info %.*s", (int)(skb->len - 1),
(char *)(skb->data + 1));
kfree_skb(skb);
@ -408,8 +409,16 @@ int btintel_load_ddc_config(struct hci_dev *hdev, const char *ddc_name)
/* DDC file contains one or more DDC structure which has
* Length (1 byte), DDC ID (2 bytes), and DDC value (Length - 2).
*/
while (fw->size > fw_ptr - fw->data) {
u8 cmd_plen = fw_ptr[0] + sizeof(u8);
while (fw->size > (size_t)(fw_ptr - fw->data)) {
size_t remaining = fw->size - (fw_ptr - fw->data);
unsigned int cmd_plen = fw_ptr[0] + 1U;
if (cmd_plen < 3 || cmd_plen > U8_MAX || cmd_plen > remaining) {
bt_dev_err(hdev, "Malformed DDC record (plen=%u, remaining=%zu)",
cmd_plen, remaining);
release_firmware(fw);
return -EINVAL;
}
skb = __hci_cmd_sync(hdev, 0xfc8b, cmd_plen, fw_ptr,
HCI_INIT_TIMEOUT);

12
drivers/bluetooth/btintel_pcie.c
View File

@ -1197,7 +1197,7 @@ static int btintel_pcie_recv_frame(struct btintel_pcie_data *data,
{
int ret;
u8 pkt_type;
u16 plen;
u32 plen;
u32 pcie_pkt_type;
void *pdata;
struct hci_dev *hdev = data->hdev;
@ -2219,6 +2219,16 @@ static int btintel_pcie_send_frame(struct hci_dev *hdev,
if (test_bit(BTINTEL_PCIE_RECOVERY_IN_PROGRESS, &data->flags))
return -ENODEV;
/* Account for the 4-byte PCIe type header prepended before the
* DMA copy. Written as a subtraction to avoid wrap-around on
* attacker-controlled skb->len.
*/
if (skb->len > BTINTEL_PCIE_BUFFER_SIZE - BTINTEL_PCIE_HCI_TYPE_LEN) {
bt_dev_err(hdev, "Packet too large: %u > %u", skb->len,
BTINTEL_PCIE_BUFFER_SIZE - BTINTEL_PCIE_HCI_TYPE_LEN);
return -EMSGSIZE;
}
/* Due to the fw limitation, the type header of the packet should be
* 4 bytes unlike 1 byte for UART. In UART, the firmware can read
* the first byte to get the packet type and redirect the rest of data

2
drivers/char/tpm/tpm-interface.c
View File

@ -268,7 +268,7 @@ ssize_t tpm_transmit_cmd(struct tpm_chip *chip, struct tpm_buf *buf,
int err;
ssize_t len;
len = tpm_transmit(chip, buf->data, PAGE_SIZE);
len = tpm_transmit(chip, buf->data, buf->capacity);
if (len < 0)
return len;

6
drivers/char/tpm/tpm2-cmd.c
View File

@ -281,8 +281,10 @@ int tpm2_get_random(struct tpm_chip *chip, u8 *dest, size_t max)
}
tpm_buf_append_u16(buf, num_bytes);
err = tpm_buf_fill_hmac_session(chip, buf);
if (err)
if (err) {
tpm2_end_auth_session(chip);
return err;
}
err = tpm_transmit_cmd(chip, buf,
offsetof(struct tpm2_get_random_out,
@ -305,7 +307,7 @@ int tpm2_get_random(struct tpm_chip *chip, u8 *dest, size_t max)
out = (struct tpm2_get_random_out *)&buf->data[offset];
recd = min_t(u32, be16_to_cpu(out->size), num_bytes);
if (tpm_buf_length(buf) <
TPM_HEADER_SIZE +
offset +
offsetof(struct tpm2_get_random_out, buffer) +
recd) {
tpm2_end_auth_session(chip);

1
drivers/char/tpm/tpm2-sessions.c
View File

@ -975,6 +975,7 @@ static int tpm2_load_null(struct tpm_chip *chip, u32 *null_key)
/* Deduce from the name change TPM interference: */
dev_err(&chip->dev, "null key integrity check failed\n");
tpm2_flush_context(chip, tmp_null_key);
rc = -ENODEV;
err:
if (rc) {

92
drivers/clk/spacemit/ccu-k3.c
View File

@ -30,14 +30,87 @@ static const struct ccu_pll_rate_tbl pll2_rate_tbl[] = {
};
static const struct ccu_pll_rate_tbl pll3_rate_tbl[] = {
CCU_PLLA_RATE(1050000000UL, 0x0b2b3000, 0x00000000, 0xa0458080),
CCU_PLLA_RATE(1100000000UL, 0x0b2d3555, 0x00005500, 0xa0458181),
CCU_PLLA_RATE(1150000000UL, 0x0b2f3aaa, 0x0000ab00, 0xa0458181),
CCU_PLLA_RATE(1200000000UL, 0x0b320000, 0x00000000, 0xa0458181),
CCU_PLLA_RATE(1250000000UL, 0x0b340555, 0x00005500, 0xa0458181),
CCU_PLLA_RATE(1300000000UL, 0x0b360aaa, 0x0000ab00, 0xa0458282),
CCU_PLLA_RATE(1350000000UL, 0x0b381000, 0x00000000, 0xa0458282),
CCU_PLLA_RATE(1400000000UL, 0x0b3a1555, 0x00005500, 0xa0458282),
CCU_PLLA_RATE(1450000000UL, 0x0b3c1aaa, 0x0000ab00, 0xa0458383),
CCU_PLLA_RATE(1500000000UL, 0x0b3e2000, 0x00000000, 0xa0458383),
CCU_PLLA_RATE(1550000000UL, 0x0b402555, 0x00005500, 0xa0458383),
CCU_PLLA_RATE(1600000000UL, 0x0b422aaa, 0x0000ab00, 0xa0458484),
CCU_PLLA_RATE(1650000000UL, 0x0b443000, 0x00000000, 0xa0458484),
CCU_PLLA_RATE(1700000000UL, 0x0b463555, 0x00005500, 0xa0458484),
CCU_PLLA_RATE(1750000000UL, 0x0b483aaa, 0x0000ab00, 0xa0458585),
CCU_PLLA_RATE(1800000000UL, 0x0b4b0000, 0x00000000, 0xa0458585),
CCU_PLLA_RATE(1850000000UL, 0x0b4d0555, 0x00005500, 0xa0458585),
CCU_PLLA_RATE(1900000000UL, 0x0b4f0aaa, 0x0000ab00, 0xa0458686),
CCU_PLLA_RATE(1950000000UL, 0x0b511000, 0x00000000, 0xa0458686),
CCU_PLLA_RATE(2000000000UL, 0x0b292aaa, 0x0000ab00, 0xa0558686),
CCU_PLLA_RATE(2050000000UL, 0x0b2a2d55, 0x00005500, 0xa0558686),
CCU_PLLA_RATE(2100000000UL, 0x0b2b3000, 0x00000000, 0xa0558787),
CCU_PLLA_RATE(2150000000UL, 0x0b2c32aa, 0x0000ab00, 0xa0558787),
CCU_PLLA_RATE(2200000000UL, 0x0b2d3555, 0x00005500, 0xa0558787),
CCU_PLLA_RATE(2250000000UL, 0x0b2e3800, 0x00000000, 0xa0558888),
CCU_PLLA_RATE(2300000000UL, 0x0b2f3aaa, 0x0000ab00, 0xa0558888),
CCU_PLLA_RATE(2350000000UL, 0x0b303d55, 0x00005500, 0xa0558888),
CCU_PLLA_RATE(2400000000UL, 0x0b320000, 0x00000000, 0xa0558989),
};
static const struct ccu_pll_rate_tbl pll4_rate_tbl[] = {
CCU_PLLA_RATE(1050000000UL, 0x0b2b3000, 0x00000000, 0xa0458080),
CCU_PLLA_RATE(1100000000UL, 0x0b2d3555, 0x00005500, 0xa0458181),
CCU_PLLA_RATE(1150000000UL, 0x0b2f3aaa, 0x0000ab00, 0xa0458181),
CCU_PLLA_RATE(1200000000UL, 0x0b320000, 0x00000000, 0xa0458181),
CCU_PLLA_RATE(1250000000UL, 0x0b340555, 0x00005500, 0xa0458181),
CCU_PLLA_RATE(1300000000UL, 0x0b360aaa, 0x0000ab00, 0xa0458282),
CCU_PLLA_RATE(1350000000UL, 0x0b381000, 0x00000000, 0xa0458282),
CCU_PLLA_RATE(1400000000UL, 0x0b3a1555, 0x00005500, 0xa0458282),
CCU_PLLA_RATE(1450000000UL, 0x0b3c1aaa, 0x0000ab00, 0xa0458383),
CCU_PLLA_RATE(1500000000UL, 0x0b3e2000, 0x00000000, 0xa0458383),
CCU_PLLA_RATE(1550000000UL, 0x0b402555, 0x00005500, 0xa0458383),
CCU_PLLA_RATE(1600000000UL, 0x0b422aaa, 0x0000ab00, 0xa0458484),
CCU_PLLA_RATE(1650000000UL, 0x0b443000, 0x00000000, 0xa0458484),
CCU_PLLA_RATE(1700000000UL, 0x0b463555, 0x00005500, 0xa0458484),
CCU_PLLA_RATE(1750000000UL, 0x0b483aaa, 0x0000ab00, 0xa0458585),
CCU_PLLA_RATE(1800000000UL, 0x0b4b0000, 0x00000000, 0xa0458585),
CCU_PLLA_RATE(1850000000UL, 0x0b4d0555, 0x00005500, 0xa0458585),
CCU_PLLA_RATE(1900000000UL, 0x0b4f0aaa, 0x0000ab00, 0xa0458686),
CCU_PLLA_RATE(1950000000UL, 0x0b511000, 0x00000000, 0xa0458686),
CCU_PLLA_RATE(2000000000UL, 0x0b292aaa, 0x0000ab00, 0xa0558686),
CCU_PLLA_RATE(2050000000UL, 0x0b2a2d55, 0x00005500, 0xa0558686),
CCU_PLLA_RATE(2100000000UL, 0x0b2b3000, 0x00000000, 0xa0558787),
CCU_PLLA_RATE(2150000000UL, 0x0b2c32aa, 0x0000ab00, 0xa0558787),
CCU_PLLA_RATE(2200000000UL, 0x0b2d3555, 0x00005500, 0xa0558787),
CCU_PLLA_RATE(2250000000UL, 0x0b2e3800, 0x00000000, 0xa0558888),
CCU_PLLA_RATE(2300000000UL, 0x0b2f3aaa, 0x0000ab00, 0xa0558888),
CCU_PLLA_RATE(2350000000UL, 0x0b303d55, 0x00005500, 0xa0558888),
CCU_PLLA_RATE(2400000000UL, 0x0b320000, 0x00000000, 0xa0558989),
};
static const struct ccu_pll_rate_tbl pll5_rate_tbl[] = {
CCU_PLLA_RATE(1050000000UL, 0x0b2b3000, 0x00000000, 0xa0458080),
CCU_PLLA_RATE(1100000000UL, 0x0b2d3555, 0x00005500, 0xa0458181),
CCU_PLLA_RATE(1150000000UL, 0x0b2f3aaa, 0x0000ab00, 0xa0458181),
CCU_PLLA_RATE(1200000000UL, 0x0b320000, 0x00000000, 0xa0458181),
CCU_PLLA_RATE(1250000000UL, 0x0b340555, 0x00005500, 0xa0458181),
CCU_PLLA_RATE(1300000000UL, 0x0b360aaa, 0x0000ab00, 0xa0458282),
CCU_PLLA_RATE(1350000000UL, 0x0b381000, 0x00000000, 0xa0458282),
CCU_PLLA_RATE(1400000000UL, 0x0b3a1555, 0x00005500, 0xa0458282),
CCU_PLLA_RATE(1450000000UL, 0x0b3c1aaa, 0x0000ab00, 0xa0458383),
CCU_PLLA_RATE(1500000000UL, 0x0b3e2000, 0x00000000, 0xa0458383),
CCU_PLLA_RATE(1550000000UL, 0x0b402555, 0x00005500, 0xa0458383),
CCU_PLLA_RATE(1600000000UL, 0x0b422aaa, 0x0000ab00, 0xa0458484),
CCU_PLLA_RATE(1650000000UL, 0x0b443000, 0x00000000, 0xa0458484),
CCU_PLLA_RATE(1700000000UL, 0x0b463555, 0x00005500, 0xa0458484),
CCU_PLLA_RATE(1750000000UL, 0x0b483aaa, 0x0000ab00, 0xa0458585),
CCU_PLLA_RATE(1800000000UL, 0x0b4b0000, 0x00000000, 0xa0458585),
CCU_PLLA_RATE(1850000000UL, 0x0b4d0555, 0x00005500, 0xa0458585),
CCU_PLLA_RATE(1900000000UL, 0x0b4f0aaa, 0x0000ab00, 0xa0458686),
CCU_PLLA_RATE(1950000000UL, 0x0b511000, 0x00000000, 0xa0458686),
CCU_PLLA_RATE(2000000000UL, 0x0b292aaa, 0x0000ab00, 0xa0558686),
};
@ -50,6 +123,25 @@ static const struct ccu_pll_rate_tbl pll7_rate_tbl[] = {
};
static const struct ccu_pll_rate_tbl pll8_rate_tbl[] = {
CCU_PLLA_RATE(1050000000UL, 0x0b2b3000, 0x00000000, 0xa0458080),
CCU_PLLA_RATE(1100000000UL, 0x0b2d3555, 0x00005500, 0xa0458181),
CCU_PLLA_RATE(1150000000UL, 0x0b2f3aaa, 0x0000ab00, 0xa0458181),
CCU_PLLA_RATE(1200000000UL, 0x0b320000, 0x00000000, 0xa0458181),
CCU_PLLA_RATE(1250000000UL, 0x0b340555, 0x00005500, 0xa0458181),
CCU_PLLA_RATE(1300000000UL, 0x0b360aaa, 0x0000ab00, 0xa0458282),
CCU_PLLA_RATE(1350000000UL, 0x0b381000, 0x00000000, 0xa0458282),
CCU_PLLA_RATE(1400000000UL, 0x0b3a1555, 0x00005500, 0xa0458282),
CCU_PLLA_RATE(1450000000UL, 0x0b3c1aaa, 0x0000ab00, 0xa0458383),
CCU_PLLA_RATE(1500000000UL, 0x0b3e2000, 0x00000000, 0xa0458383),
CCU_PLLA_RATE(1550000000UL, 0x0b402555, 0x00005500, 0xa0458383),
CCU_PLLA_RATE(1600000000UL, 0x0b422aaa, 0x0000ab00, 0xa0458484),
CCU_PLLA_RATE(1650000000UL, 0x0b443000, 0x00000000, 0xa0458484),
CCU_PLLA_RATE(1700000000UL, 0x0b463555, 0x00005500, 0xa0458484),
CCU_PLLA_RATE(1750000000UL, 0x0b483aaa, 0x0000ab00, 0xa0458585),
CCU_PLLA_RATE(1800000000UL, 0x0b4b0000, 0x00000000, 0xa0458585),
CCU_PLLA_RATE(1850000000UL, 0x0b4d0555, 0x00005500, 0xa0458585),
CCU_PLLA_RATE(1900000000UL, 0x0b4f0aaa, 0x0000ab00, 0xa0458686),
CCU_PLLA_RATE(1950000000UL, 0x0b511000, 0x00000000, 0xa0458686),
CCU_PLLA_RATE(2000000000UL, 0x0b292aaa, 0x0000ab00, 0xa0558686),
};

26
drivers/clk/ti/composite.c
View File

@ -52,7 +52,7 @@ static const struct clk_ops ti_composite_gate_ops = {
struct component_clk {
int num_parents;
struct clk_parent_data *parent_data;
const char **parent_names;
struct device_node *node;
int type;
struct clk_hw *hw;
@ -116,7 +116,7 @@ static void __init _register_composite(void *user,
struct clk_hw_omap_comp *cclk = to_clk_hw_comp(hw);
struct component_clk *comp;
int num_parents = 0;
struct clk_parent_data *parent_data = NULL;
const char **parent_names = NULL;
const char *name;
int i;
int ret;
@ -155,7 +155,7 @@ static void __init _register_composite(void *user,
continue;
if (comp->num_parents) {
num_parents = comp->num_parents;
parent_data = comp->parent_data;
parent_names = comp->parent_names;
break;
}
}
@ -166,8 +166,8 @@ static void __init _register_composite(void *user,
}
name = ti_dt_clk_name(node);
clk = clk_register_composite_pdata(NULL, name,
parent_data, num_parents,
clk = clk_register_composite(NULL, name,
parent_names, num_parents,
_get_hw(cclk, CLK_COMPONENT_TYPE_MUX),
&ti_clk_mux_ops,
_get_hw(cclk, CLK_COMPONENT_TYPE_DIVIDER),
@ -190,7 +190,7 @@ static void __init _register_composite(void *user,
if (!cclk->comp_clks[i])
continue;
list_del(&cclk->comp_clks[i]->link);
kfree(cclk->comp_clks[i]->parent_data);
kfree(cclk->comp_clks[i]->parent_names);
kfree(cclk->comp_clks[i]);
}
@ -237,9 +237,8 @@ int __init ti_clk_add_component(struct device_node *node, struct clk_hw *hw,
int type)
{
unsigned int num_parents;
struct clk_parent_data *parent_data;
const char **parent_names;
struct component_clk *clk;
unsigned int i;
num_parents = of_clk_get_parent_count(node);
@ -248,21 +247,20 @@ int __init ti_clk_add_component(struct device_node *node, struct clk_hw *hw,
return -EINVAL;
}
parent_data = kzalloc_objs(*parent_data, num_parents);
if (!parent_data)
parent_names = kcalloc(num_parents, sizeof(char *), GFP_KERNEL);
if (!parent_names)
return -ENOMEM;
for (i = 0; i < num_parents; i++)
parent_data[i].index = i;
of_clk_parent_fill(node, parent_names, num_parents);
clk = kzalloc_obj(*clk);
if (!clk) {
kfree(parent_data);
kfree(parent_names);
return -ENOMEM;
}
clk->num_parents = num_parents;
clk->parent_data = parent_data;
clk->parent_names = parent_names;
clk->hw = hw;
clk->node = node;
clk->type = type;

4
drivers/cpufreq/cpufreq.c
View File

@ -2590,8 +2590,8 @@ static void cpufreq_update_pressure(struct cpufreq_policy *policy)
cpu = cpumask_first(policy->related_cpus);
max_freq = arch_scale_freq_ref(cpu);
if (!max_freq)
max_freq = policy->cpuinfo.max_freq;
if (!max_freq && cpufreq_driver->scale_freq_ref)
max_freq = cpufreq_driver->scale_freq_ref(policy);
capped_freq = policy->max;

10
drivers/cpufreq/intel_pstate.c
View File

@ -1135,6 +1135,14 @@ static bool hybrid_clear_max_perf_cpu(void)
return ret;
}
static unsigned int intel_pstate_scale_freq_ref(struct cpufreq_policy *policy)
{
if (READ_ONCE(all_cpu_data[policy->cpu]->capacity_perf))
return policy->cpuinfo.max_freq;
return 0;
}
static void intel_pstate_update_freq_limits(struct cpudata *cpu)
{
int scaling = cpu->pstate.scaling;
@ -3088,6 +3096,7 @@ static struct cpufreq_driver intel_pstate = {
.offline = intel_pstate_cpu_offline,
.online = intel_pstate_cpu_online,
.update_limits = intel_pstate_update_limits,
.scale_freq_ref = intel_pstate_scale_freq_ref,
.name = "intel_pstate",
};
@ -3411,6 +3420,7 @@ static struct cpufreq_driver intel_cpufreq = {
.suspend = intel_cpufreq_suspend,
.resume = intel_pstate_resume,
.update_limits = intel_pstate_update_limits,
.scale_freq_ref = intel_pstate_scale_freq_ref,
.name = "intel_cpufreq",
};

67
drivers/edac/altera_edac.c
View File

@ -453,15 +453,6 @@ static int altr_sdram_probe(struct platform_device *pdev)
return res;
}
static void altr_sdram_remove(struct platform_device *pdev)
{
struct mem_ctl_info *mci = platform_get_drvdata(pdev);
edac_mc_del_mc(&pdev->dev);
edac_mc_free(mci);
platform_set_drvdata(pdev, NULL);
}
/*
* If you want to suspend, need to disable EDAC by removing it
* from the device tree or defconfig.
@ -481,13 +472,13 @@ static const struct dev_pm_ops altr_sdram_pm_ops = {
static struct platform_driver altr_sdram_edac_driver = {
.probe = altr_sdram_probe,
.remove = altr_sdram_remove,
.driver = {
.name = "altr_sdram_edac",
#ifdef CONFIG_PM
.pm = &altr_sdram_pm_ops,
#endif
.of_match_table = altr_sdram_ctrl_of_match,
.suppress_bind_attrs = true,
},
};
@ -517,6 +508,7 @@ static struct platform_driver altr_edac_driver = {
.driver = {
.name = "socfpga_ecc_manager",
.of_match_table = altr_edac_of_match,
.suppress_bind_attrs = true,
},
};
module_platform_driver(altr_edac_driver);
@ -803,22 +795,12 @@ static int altr_edac_device_probe(struct platform_device *pdev)
return res;
}
static void altr_edac_device_remove(struct platform_device *pdev)
{
struct edac_device_ctl_info *dci = platform_get_drvdata(pdev);
struct altr_edac_device_dev *drvdata = dci->pvt_info;
debugfs_remove_recursive(drvdata->debugfs_dir);
edac_device_del_device(&pdev->dev);
edac_device_free_ctl_info(dci);
}
static struct platform_driver altr_edac_device_driver = {
.probe = altr_edac_device_probe,
.remove = altr_edac_device_remove,
.driver = {
.name = "altr_edac_device",
.of_match_table = altr_edac_device_of_match,
.suppress_bind_attrs = true,
},
};
module_platform_driver(altr_edac_device_driver);
@ -944,7 +926,7 @@ static int __maybe_unused altr_init_memory_port(void __iomem *ioaddr, int port)
return ret;
}
static __init int __maybe_unused
static int __maybe_unused
altr_init_a10_ecc_block(struct device_node *np, u32 irq_mask,
u32 ecc_ctrl_en_mask, bool dual_port)
{
@ -1019,7 +1001,7 @@ altr_init_a10_ecc_block(struct device_node *np, u32 irq_mask,
static int validate_parent_available(struct device_node *np);
static const struct of_device_id altr_edac_a10_device_of_match[];
static int __init __maybe_unused altr_init_a10_ecc_device_type(char *compat)
static int __maybe_unused altr_init_a10_ecc_device_type(char *compat)
{
int irq;
struct device_node *child, *np;
@ -1348,7 +1330,7 @@ static const struct edac_device_prv_data a10_l2ecc_data = {
#ifdef CONFIG_EDAC_ALTERA_ETHERNET
static int __init socfpga_init_ethernet_ecc(struct altr_edac_device_dev *dev)
static int socfpga_init_ethernet_ecc(struct altr_edac_device_dev *dev)
{
int ret;
@ -1378,7 +1360,7 @@ static const struct edac_device_prv_data a10_enetecc_data = {
#ifdef CONFIG_EDAC_ALTERA_NAND
static int __init socfpga_init_nand_ecc(struct altr_edac_device_dev *device)
static int socfpga_init_nand_ecc(struct altr_edac_device_dev *device)
{
int ret;
@ -1408,7 +1390,7 @@ static const struct edac_device_prv_data a10_nandecc_data = {
#ifdef CONFIG_EDAC_ALTERA_DMA
static int __init socfpga_init_dma_ecc(struct altr_edac_device_dev *device)
static int socfpga_init_dma_ecc(struct altr_edac_device_dev *device)
{
int ret;
@ -1438,7 +1420,7 @@ static const struct edac_device_prv_data a10_dmaecc_data = {
#ifdef CONFIG_EDAC_ALTERA_USB
static int __init socfpga_init_usb_ecc(struct altr_edac_device_dev *device)
static int socfpga_init_usb_ecc(struct altr_edac_device_dev *device)
{
int ret;
@ -1468,7 +1450,7 @@ static const struct edac_device_prv_data a10_usbecc_data = {
#ifdef CONFIG_EDAC_ALTERA_QSPI
static int __init socfpga_init_qspi_ecc(struct altr_edac_device_dev *device)
static int socfpga_init_qspi_ecc(struct altr_edac_device_dev *device)
{
int ret;
@ -1524,6 +1506,7 @@ static int altr_portb_setup(struct altr_edac_device_dev *device)
dci = edac_device_alloc_ctl_info(sizeof(*altdev), ecc_name, 1,
ecc_name, 1, 0, edac_idx);
if (!dci) {
of_node_put(np);
edac_printk(KERN_ERR, EDAC_DEVICE,
"%s: Unable to allocate PortB EDAC device\n",
ecc_name);
@ -1534,8 +1517,11 @@ static int altr_portb_setup(struct altr_edac_device_dev *device)
altdev = dci->pvt_info;
*altdev = *device;
if (!devres_open_group(device->edac->dev, altr_portb_setup, GFP_KERNEL))
if (!devres_open_group(device->edac->dev, altr_portb_setup, GFP_KERNEL)) {
edac_device_free_ctl_info(dci);
of_node_put(np);
return -ENOMEM;
}
/* Update PortB specific values */
altdev->edac_dev_name = ecc_name;
@ -1600,6 +1586,8 @@ static int altr_portb_setup(struct altr_edac_device_dev *device)
rc = -ENOMEM;
goto err_release_group_1;
}
of_node_put(np);
altr_create_edacdev_dbgfs(dci, prv);
list_add(&altdev->next, &altdev->edac->a10_ecc_devices);
@ -1609,14 +1597,20 @@ static int altr_portb_setup(struct altr_edac_device_dev *device)
return 0;
err_release_group_1:
edac_device_free_ctl_info(dci);
/*
* Release the devres group first so the managed IRQs are
* unregistered before dci (which contains the IRQ handler's
* data via dci->pvt_info) is freed, avoiding a use-after-free.
*/
devres_release_group(device->edac->dev, altr_portb_setup);
edac_device_free_ctl_info(dci);
of_node_put(np);
edac_printk(KERN_ERR, EDAC_DEVICE,
"%s:Error setting up EDAC device: %d\n", ecc_name, rc);
return rc;
}
static int __init socfpga_init_sdmmc_ecc(struct altr_edac_device_dev *device)
static int socfpga_init_sdmmc_ecc(struct altr_edac_device_dev *device)
{
int rc = -ENODEV;
struct device_node *child;
@ -1638,7 +1632,7 @@ static int __init socfpga_init_sdmmc_ecc(struct altr_edac_device_dev *device)
goto exit;
/* Setup portB */
return altr_portb_setup(device);
rc = altr_portb_setup(device);
exit:
of_node_put(child);
@ -2013,9 +2007,17 @@ static int altr_edac_a10_device_add(struct altr_arria10_edac *edac,
return 0;
err_release_group1:
/*
* Release the devres group first so the managed IRQs are
* unregistered before dci (which contains the IRQ handler's
* data via dci->pvt_info) is freed, avoiding a use-after-free.
*/
devres_release_group(edac->dev, NULL);
edac_device_free_ctl_info(dci);
goto err_print;
err_release_group:
devres_release_group(edac->dev, NULL);
err_print:
edac_printk(KERN_ERR, EDAC_DEVICE,
"%s:Error setting up EDAC device: %d\n", ecc_name, rc);
@ -2214,6 +2216,7 @@ static struct platform_driver altr_edac_a10_driver = {
.driver = {
.name = "socfpga_a10_ecc_manager",
.of_match_table = altr_edac_a10_of_match,
.suppress_bind_attrs = true,
},
};
module_platform_driver(altr_edac_a10_driver);

1
drivers/edac/versalnet_edac.c
View File

@ -953,6 +953,7 @@ static void mc_remove(struct platform_device *pdev)
remove_versalnet(priv);
rproc_shutdown(priv->mcdi->r5_rproc);
cdx_mcdi_finish(priv->mcdi);
rproc_put(priv->mcdi->r5_rproc);
kfree(priv->mcdi);
}

13
drivers/gpu/drm/amd/amdgpu/amdgpu_acp.c
View File

@ -318,7 +318,7 @@ static int acp_hw_init(struct amdgpu_ip_block *ip_block)
r = device_for_each_child(adev->acp.parent, &adev->acp.acp_genpd->gpd,
acp_genpd_add_device);
if (r)
goto failure;
goto failure_remove_mfd;
break;
}
default:
@ -443,7 +443,7 @@ static int acp_hw_init(struct amdgpu_ip_block *ip_block)
r = device_for_each_child(adev->acp.parent, &adev->acp.acp_genpd->gpd,
acp_genpd_add_device);
if (r)
goto failure;
goto failure_remove_mfd;
}
/* Assert Soft reset of ACP */
@ -461,7 +461,7 @@ static int acp_hw_init(struct amdgpu_ip_block *ip_block)
if (--count == 0) {
dev_err(&adev->pdev->dev, "Failed to reset ACP\n");
r = -ETIMEDOUT;
goto failure;
goto failure_remove_mfd;
}
udelay(100);
}
@ -479,7 +479,7 @@ static int acp_hw_init(struct amdgpu_ip_block *ip_block)
if (--count == 0) {
dev_err(&adev->pdev->dev, "Failed to reset ACP\n");
r = -ETIMEDOUT;
goto failure;
goto failure_remove_mfd;
}
udelay(100);
}
@ -489,6 +489,11 @@ static int acp_hw_init(struct amdgpu_ip_block *ip_block)
cgs_write_register(adev->acp.cgs_device, mmACP_SOFT_RESET, val);
return 0;
failure_remove_mfd:
device_for_each_child(adev->acp.parent, NULL,
acp_genpd_remove_device);
mfd_remove_devices(adev->acp.parent);
failure:
kfree(adev->acp.i2s_pdata);
kfree(adev->acp.acp_res);

2
drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c
View File

@ -1325,6 +1325,8 @@ static int amdgpu_discovery_sysfs_ips(struct amdgpu_device *adev,
ip_hw_instance->kobj.kset = &ip_hw_id->hw_id_kset;
res = kobject_add(&ip_hw_instance->kobj, NULL,
"%d", ip_hw_instance->num_instance);
if (res)
kobject_put(&ip_hw_instance->kobj);
next_ip:
if (reg_base_64)
ip_offset += struct_size(ip, base_address_64,

8
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
View File

@ -2664,6 +2664,14 @@ static int amdgpu_pmops_suspend_noirq(struct device *dev)
struct amdgpu_device *adev = drm_to_adev(drm_dev);
int r;
/*
* A GPU parked by vga_switcheroo has no power and no PCIe link, so the
* ASIC reset below would fail and abort the whole noirq suspend phase.
* Bail out like amdgpu_device_prepare/suspend/resume() already do.
*/
if (drm_dev->switch_power_state == DRM_SWITCH_POWER_OFF)
return 0;
if (amdgpu_acpi_should_gpu_reset(adev)) {
amdgpu_device_lock_reset_domain(adev->reset_domain);
r = amdgpu_asic_reset(adev);

37
drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
View File

@ -724,8 +724,14 @@ void amdgpu_ring_set_fence_errors_and_reemit(struct amdgpu_ring *ring,
bool is_guilty_fence;
bool is_guilty_context;
last_seq = amdgpu_fence_read(ring) & ring->fence_drv.num_fences_mask;
seq = ring->fence_drv.sync_seq & ring->fence_drv.num_fences_mask;
last_seq = amdgpu_fence_read(ring);
seq = ring->fence_drv.sync_seq;
/* Hardware completion can precede the interrupt which signals fences. */
if (last_seq == seq) {
amdgpu_fence_process(ring);
return;
}
/* If there is nothing to reemit, return early and set an error on the fence
* if applicable. If all of the fences are siganlled, this will be a nop.
@ -733,7 +739,9 @@ void amdgpu_ring_set_fence_errors_and_reemit(struct amdgpu_ring *ring,
* we are skipping it on purpose.
*/
if (!ring->ring_backup_entries_to_copy) {
amdgpu_fence_driver_force_completion(ring, &guilty_fence->base);
if (guilty_fence)
amdgpu_fence_driver_force_completion(ring,
&guilty_fence->base);
return;
}
ring->reemit = true;
@ -741,9 +749,8 @@ void amdgpu_ring_set_fence_errors_and_reemit(struct amdgpu_ring *ring,
spin_lock_irqsave(&ring->fence_drv.lock, flags);
do {
last_seq++;
last_seq &= ring->fence_drv.num_fences_mask;
ptr = &ring->fence_drv.fences[last_seq];
ptr = &ring->fence_drv.fences[last_seq & ring->fence_drv.num_fences_mask];
rcu_read_lock();
unprocessed = rcu_dereference(*ptr);
@ -800,26 +807,24 @@ void amdgpu_ring_backup_unprocessed_commands(struct amdgpu_ring *ring,
struct amdgpu_fence *fence;
u32 seq, last_seq;
last_seq = amdgpu_fence_read(ring) & ring->fence_drv.num_fences_mask;
seq = ring->fence_drv.sync_seq & ring->fence_drv.num_fences_mask;
last_seq = amdgpu_fence_read(ring);
seq = ring->fence_drv.sync_seq;
ring->ring_backup_entries_to_copy = 0;
/* if we've already seen this fence, return early.
* ring->ring_backup_entries_to_copy is set to 0 so
* the reemit helper will return early as well to
* avoid getting stuck in a reemit loop.
/*
* Avoid replay loops for a repeated guilty fence. NULL identifies a
* collateral ring, whose pending commands still need backup.
*/
if (ring->guilty_fence == guilty_fence) {
if (guilty_fence && ring->guilty_fence == guilty_fence) {
ring->guilty_fence = NULL;
return;
}
ring->guilty_fence = guilty_fence;
do {
while (last_seq != seq) {
last_seq++;
last_seq &= ring->fence_drv.num_fences_mask;
ptr = &ring->fence_drv.fences[last_seq];
ptr = &ring->fence_drv.fences[last_seq & ring->fence_drv.num_fences_mask];
rcu_read_lock();
unprocessed = rcu_dereference(*ptr);
@ -829,7 +834,7 @@ void amdgpu_ring_backup_unprocessed_commands(struct amdgpu_ring *ring,
amdgpu_ring_backup_unprocessed_command(ring, fence);
}
rcu_read_unlock();
} while (last_seq != seq);
}
}
struct amdgpu_fence *

8
drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c
View File

@ -2342,7 +2342,7 @@ void amdgpu_ttm_fini(struct amdgpu_device *adev)
void amdgpu_ttm_enable_buffer_funcs(struct amdgpu_device *adev)
{
struct ttm_resource_manager *man = ttm_manager_type(&adev->mman.bdev, TTM_PL_VRAM);
u32 num_clear_entities, num_move_entities;
u32 num_clear_entities, num_move_entities, sdma_ip_version;
int r, i, j;
if (!adev->mman.initialized || amdgpu_in_reset(adev) ||
@ -2367,6 +2367,12 @@ void amdgpu_ttm_enable_buffer_funcs(struct amdgpu_device *adev)
num_clear_entities = MIN(adev->mman.num_buffer_funcs_scheds, TTM_NUM_MOVE_FENCES);
num_move_entities = MIN(adev->mman.num_buffer_funcs_scheds, TTM_NUM_MOVE_FENCES);
/* TODO: workaround for DCC corruption when moving BOs from multiple queues at
* the same time: use a single queue until the root cause is identified and fixed.
*/
sdma_ip_version = amdgpu_ip_version(adev, SDMA0_HWIP, 0);
if (sdma_ip_version == IP_VERSION(7, 0, 0) || sdma_ip_version == IP_VERSION(7, 0, 1))
num_move_entities = 1;
adev->mman.clear_entities = kzalloc_objs(struct amdgpu_ttm_buffer_entity,
num_clear_entities);

2
drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c
View File

@ -3808,6 +3808,7 @@ static int gfx_v11_0_cp_gfx_resume(struct amdgpu_device *adev)
rb_bufsz = order_base_2(ring->ring_size / 8);
tmp = REG_SET_FIELD(0, CP_RB0_CNTL, RB_BUFSZ, rb_bufsz);
tmp = REG_SET_FIELD(tmp, CP_RB0_CNTL, RB_BLKSZ, rb_bufsz - 2);
tmp = REG_SET_FIELD(tmp, CP_RB0_CNTL, KMD_QUEUE, 1);
WREG32_SOC15(GC, 0, regCP_RB0_CNTL, tmp);
/* Initialize the ring buffer's write pointers */
@ -4216,6 +4217,7 @@ static int gfx_v11_0_gfx_mqd_init(struct amdgpu_device *adev, void *m,
tmp = regCP_GFX_HQD_CNTL_DEFAULT;
tmp = REG_SET_FIELD(tmp, CP_GFX_HQD_CNTL, RB_BUFSZ, rb_bufsz);
tmp = REG_SET_FIELD(tmp, CP_GFX_HQD_CNTL, RB_BLKSZ, rb_bufsz - 2);
tmp = REG_SET_FIELD(tmp, CP_GFX_HQD_CNTL, KMD_QUEUE, prop->kernel_queue);
#ifdef __BIG_ENDIAN
tmp = REG_SET_FIELD(tmp, CP_GFX_HQD_CNTL, BUF_SWAP, 1);
#endif

2
drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c
View File

@ -2767,6 +2767,7 @@ static int gfx_v12_0_cp_gfx_resume(struct amdgpu_device *adev)
rb_bufsz = order_base_2(ring->ring_size / 8);
tmp = REG_SET_FIELD(0, CP_RB0_CNTL, RB_BUFSZ, rb_bufsz);
tmp = REG_SET_FIELD(tmp, CP_RB0_CNTL, RB_BLKSZ, rb_bufsz - 2);
tmp = REG_SET_FIELD(tmp, CP_RB0_CNTL, KMD_QUEUE, 1);
WREG32_SOC15(GC, 0, regCP_RB0_CNTL, tmp);
/* Initialize the ring buffer's write pointers */
@ -3059,6 +3060,7 @@ static int gfx_v12_0_gfx_mqd_init(struct amdgpu_device *adev, void *m,
tmp = regCP_GFX_HQD_CNTL_DEFAULT;
tmp = REG_SET_FIELD(tmp, CP_GFX_HQD_CNTL, RB_BUFSZ, rb_bufsz);
tmp = REG_SET_FIELD(tmp, CP_GFX_HQD_CNTL, RB_BLKSZ, rb_bufsz - 2);
tmp = REG_SET_FIELD(tmp, CP_GFX_HQD_CNTL, KMD_QUEUE, prop->kernel_queue);
#ifdef __BIG_ENDIAN
tmp = REG_SET_FIELD(tmp, CP_GFX_HQD_CNTL, BUF_SWAP, 1);
#endif

9
drivers/gpu/drm/amd/amdgpu/gfx_v6_0.c
View File

@ -322,6 +322,14 @@ static const u32 verde_rlc_save_restore_register_list[] =
0x00000000
};
static void gfx_v6_0_free_microcode(struct amdgpu_device *adev)
{
amdgpu_ucode_release(&adev->gfx.pfp_fw);
amdgpu_ucode_release(&adev->gfx.me_fw);
amdgpu_ucode_release(&adev->gfx.ce_fw);
amdgpu_ucode_release(&adev->gfx.rlc_fw);
}
static int gfx_v6_0_init_microcode(struct amdgpu_device *adev)
{
const char *chip_name;
@ -3249,6 +3257,7 @@ static int gfx_v6_0_sw_fini(struct amdgpu_ip_block *ip_block)
amdgpu_ring_fini(&adev->gfx.compute_ring[i]);
amdgpu_gfx_rlc_fini(adev);
gfx_v6_0_free_microcode(adev);
return 0;
}

2
drivers/gpu/drm/amd/amdgpu/gmc_v12_0.c
View File

@ -327,7 +327,7 @@ static void gmc_v12_0_flush_gpu_tlb(struct amdgpu_device *adev, uint32_t vmid,
return;
}
gmc_v12_0_flush_vm_hub(adev, vmid, vmhub, 0);
gmc_v12_0_flush_vm_hub(adev, vmid, vmhub, flush_type);
return;
}

2
drivers/gpu/drm/amd/amdgpu/gmc_v12_1.c
View File

@ -389,7 +389,7 @@ static void gmc_v12_1_flush_gpu_tlb(struct amdgpu_device *adev, uint32_t vmid,
return;
}
gmc_v12_1_flush_vm_hub(adev, vmid, vmhub, 0);
gmc_v12_1_flush_vm_hub(adev, vmid, vmhub, flush_type);
return;
}

4
drivers/gpu/drm/amd/amdgpu/sdma_v7_1.c
View File

@ -1339,10 +1339,6 @@ static int sdma_v7_1_sw_init(struct amdgpu_ip_block *ip_block)
else
DRM_ERROR("Failed to allocated memory for SDMA IP Dump\n");
#ifdef CONFIG_DRM_AMDGPU_NAVI3X_USERQ
adev->userq_funcs[AMDGPU_HW_IP_DMA] = &userq_mes_funcs;
#endif
return r;
}

1
drivers/gpu/drm/amd/amdgpu/vi.c
View File

@ -1397,6 +1397,7 @@ struct vi_reset_quirk {
static const struct vi_reset_quirk vi_reset_quirks[] = {
{ 0x67ef, PCI_VENDOR_ID_APPLE, 0x0190, 0xe3 }, /* Radeon Pro 555X */
{ 0x67ef, PCI_VENDOR_ID_APPLE, 0x018f, 0xc2 }, /* Radeon Pro 560X */
{ 0x67ef, PCI_VENDOR_ID_APPLE, 0x017a, 0xc7 }, /* Radeon Pro 555 */
};
static bool vi_need_reset_on_init(struct amdgpu_device *adev)

13
drivers/gpu/drm/amd/amdkfd/kfd_svm.c
View File

@ -1752,12 +1752,15 @@ static int svm_range_validate_and_map(struct mm_struct *mm,
}
/*
* If prange with always mapped flag, update mapping on GPUs with
* ACCESS attribute
* If prange with always mapped flag is not mapped or migrated
* yet, update mapping on GPUs with ACCESS attribute. Don't add
* them otherwise, ACCESS GPUs may be on different XGMI hives,
* that would force hmm_range_fault to migrate VRAM back to ram.
*/
if (prange->flags & KFD_IOCTL_SVM_FLAG_GPU_ALWAYS_MAPPED)
bitmap_or(ctx->bitmap, ctx->bitmap, prange->bitmap_access,
MAX_GPU_INSTANCE);
if (bitmap_empty(ctx->bitmap, MAX_GPU_INSTANCE) &&
prange->flags & KFD_IOCTL_SVM_FLAG_GPU_ALWAYS_MAPPED)
bitmap_copy(ctx->bitmap, prange->bitmap_access,
MAX_GPU_INSTANCE);
} else {
bitmap_or(ctx->bitmap, prange->bitmap_access,
prange->bitmap_aip, MAX_GPU_INSTANCE);

11
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c
View File

@ -426,6 +426,8 @@ static int amdgpu_dm_init_power_module(struct amdgpu_display_manager *dm)
init_data[i].use_custom_backlight_caps = false;
init_data[i].custom_backlight_caps_config_no = 0;
init_data[i].use_linear_backlight_curve = false;
if (amdgpu_dc_debug_mask & DC_DISABLE_CUSTOM_BRIGHTNESS_CURVE)
init_data[i].use_linear_backlight_curve = true;
init_data[i].def_varibright_enable = 0;
init_data[i].varibright_level = 0;
/*
@ -1582,9 +1584,14 @@ static int dm_suspend(struct amdgpu_ip_block *ip_block)
dc_allow_idle_optimizations(adev->dm.dc, false);
dm->cached_dc_state = dc_state_create_copy(dm->dc->current_state);
if (!dm->cached_dc_state) {
drm_err(adev_to_drm(adev),
"Failed to allocate cached DC state during suspend\n");
mutex_unlock(&dm->dc_lock);
return -ENOMEM;
}
if (dm->cached_dc_state)
dm_gpureset_toggle_interrupts(adev, dm->cached_dc_state, false);
dm_gpureset_toggle_interrupts(adev, dm->cached_dc_state, false);
res = amdgpu_dm_commit_zero_streams(dm->dc);
if (res != DC_OK) {

92
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_backlight.c
View File

@ -39,6 +39,7 @@
#include "modules/inc/mod_power.h"
#include <linux/backlight.h>
#include <linux/minmax.h>
#include <linux/power_supply.h>
#include <drm/drm_edid.h>
#include <drm/drm_utils.h>
@ -118,6 +119,68 @@ static inline u32 scale_fw_to_input(int min, int max, u64 input)
return min + DIV_ROUND_CLOSEST_ULL(input * (max - min), AMDGPU_MAX_BL_LEVEL);
}
static u32 backlight_signal_to_luminance(const struct amdgpu_dm_backlight_caps *caps,
u32 signal)
{
u32 lower_signal, upper_signal;
u32 lower_lum, upper_lum;
int i;
if (!caps->data_points)
return DIV_ROUND_CLOSEST(signal * 100, AMDGPU_MAX_BL_LEVEL);
if (signal < caps->luminance_data[0].input_signal) {
lower_signal = 0;
lower_lum = 0;
upper_signal = caps->luminance_data[0].input_signal;
upper_lum = caps->luminance_data[0].luminance;
} else if (signal >=
caps->luminance_data[caps->data_points - 1].input_signal) {
lower_signal = caps->luminance_data[caps->data_points - 1].input_signal;
lower_lum = caps->luminance_data[caps->data_points - 1].luminance;
upper_signal = caps->max_input_signal;
upper_lum = 100;
} else {
for (i = 1;
i < caps->data_points &&
signal > caps->luminance_data[i].input_signal;
i++)
;
lower_signal = caps->luminance_data[i - 1].input_signal;
upper_signal = caps->luminance_data[i].input_signal;
lower_lum = caps->luminance_data[i - 1].luminance;
upper_lum = caps->luminance_data[i].luminance;
}
if (upper_signal == lower_signal)
return upper_lum;
return lower_lum + DIV_ROUND_CLOSEST((upper_lum - lower_lum) *
(signal - lower_signal),
upper_signal - lower_signal);
}
static u32 convert_brightness_to_millipercent(const struct amdgpu_dm_backlight_caps *caps,
u32 user_brightness)
{
unsigned int min, max;
u32 signal, luminance;
if (!get_brightness_range(caps, &min, &max))
return DIV_ROUND_CLOSEST_ULL((u64)user_brightness * 100000,
MAX_BACKLIGHT_LEVEL);
user_brightness = clamp(user_brightness, min, max);
if (amdgpu_dc_debug_mask & DC_DISABLE_CUSTOM_BRIGHTNESS_CURVE)
return DIV_ROUND_CLOSEST_ULL((u64)user_brightness * 100000, max);
signal = scale_input_to_fw(max, user_brightness);
luminance = backlight_signal_to_luminance(caps, signal);
return min_t(u32, luminance * 1000, 100000);
}
STATIC_IFN_KUNIT
void convert_custom_brightness(const struct amdgpu_dm_backlight_caps *caps,
unsigned int min, unsigned int max,
@ -260,10 +323,10 @@ void amdgpu_dm_backlight_set_level(struct amdgpu_display_manager *dm,
struct amdgpu_dm_backlight_caps *caps;
struct dc_link *link;
u32 brightness = 0;
u32 millipercent = 0;
bool rc = false, reallow_idle = false;
struct drm_connector *connector;
struct dc_stream_state *stream;
unsigned int min, max;
list_for_each_entry(connector, &dm->ddev->mode_config.connector_list, head) {
struct amdgpu_dm_connector *aconnector = to_amdgpu_dm_connector(connector);
@ -286,12 +349,21 @@ void amdgpu_dm_backlight_set_level(struct amdgpu_display_manager *dm,
/* update scratch register */
if (bl_idx == 0)
amdgpu_atombios_scratch_regs_set_backlight_level(dm->adev, dm->brightness[bl_idx]);
brightness = convert_brightness_from_user(caps, dm->brightness[bl_idx]);
link = (struct dc_link *)dm->backlight_link[bl_idx];
/* Apply brightness quirk */
if (caps->brightness_mask)
brightness |= caps->brightness_mask;
link = (struct dc_link *)dm->backlight_link[bl_idx];
stream = dm_find_stream_with_link(dm, link);
if (!stream)
return;
if (caps->aux_support) {
brightness = convert_brightness_from_user(caps, dm->brightness[bl_idx]);
/* Apply brightness quirk */
if (caps->brightness_mask)
brightness |= caps->brightness_mask;
} else {
millipercent = convert_brightness_to_millipercent(caps,
user_brightness);
}
if (trace_amdgpu_dm_brightness_enabled()) {
trace_amdgpu_dm_brightness(__builtin_return_address(0),
@ -301,10 +373,6 @@ void amdgpu_dm_backlight_set_level(struct amdgpu_display_manager *dm,
power_supply_is_system_supplied() > 0);
}
stream = dm_find_stream_with_link(dm, link);
if (!stream)
return;
mutex_lock(&dm->dc_lock);
if (dm->dc->caps.ips_support && dm->dc->ctx->dmub_srv->idle_allowed) {
dc_allow_idle_optimizations(dm->dc, false);
@ -316,10 +384,8 @@ void amdgpu_dm_backlight_set_level(struct amdgpu_display_manager *dm,
AUX_BL_DEFAULT_TRANSITION_TIME_MS, false, true);
} else {
/* power module uses millipercent */
get_brightness_range(caps, &min, &max);
brightness = DIV_ROUND_CLOSEST(brightness * 100, (max - min)) * 1000;
rc = mod_power_set_backlight_percent(dm->power_module, stream,
brightness, 0, false);
millipercent, 0, false);
}
/*

24
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_connector.c
View File

@ -732,6 +732,24 @@ get_aspect_ratio(const struct drm_display_mode *mode_in)
}
EXPORT_IF_KUNIT(get_aspect_ratio);
/*
* CTA-861 5.1: RGB video formats default to limited range except 640x480 (VIC 1).
* With Broadcast RGB left at Automatic follow that default, as i915 does.
*/
static bool rgb_output_is_limited_range(const struct dc_crtc_timing *dc_crtc_timing,
const struct drm_connector_state *connector_state)
{
switch (connector_state->hdmi.broadcast_rgb) {
case DRM_HDMI_BROADCAST_RGB_FULL:
return false;
case DRM_HDMI_BROADCAST_RGB_LIMITED:
return true;
default:
return connector_state->connector->display_info.is_hdmi &&
dc_crtc_timing->vic > 1;
}
}
enum dc_color_space
amdgpu_dm_get_output_color_space(const struct dc_crtc_timing *dc_crtc_timing,
const struct drm_connector_state *connector_state)
@ -757,7 +775,7 @@ amdgpu_dm_get_output_color_space(const struct dc_crtc_timing *dc_crtc_timing,
case DRM_MODE_COLORIMETRY_BT2020_RGB:
case DRM_MODE_COLORIMETRY_BT2020_YCC:
if (dc_crtc_timing->pixel_encoding == PIXEL_ENCODING_RGB) {
if (connector_state->hdmi.broadcast_rgb == DRM_HDMI_BROADCAST_RGB_LIMITED)
if (rgb_output_is_limited_range(dc_crtc_timing, connector_state))
color_space = COLOR_SPACE_2020_RGB_LIMITEDRANGE;
else
color_space = COLOR_SPACE_2020_RGB_FULLRANGE;
@ -769,7 +787,7 @@ amdgpu_dm_get_output_color_space(const struct dc_crtc_timing *dc_crtc_timing,
default:
if (dc_crtc_timing->pixel_encoding == PIXEL_ENCODING_RGB) {
color_space = COLOR_SPACE_SRGB;
if (connector_state->hdmi.broadcast_rgb == DRM_HDMI_BROADCAST_RGB_LIMITED)
if (rgb_output_is_limited_range(dc_crtc_timing, connector_state))
color_space = COLOR_SPACE_SRGB_LIMITED;
/*
* 27030khz is the separation point between HDTV and SDTV
@ -2256,8 +2274,8 @@ amdgpu_dm_create_validate_stream_for_sink(struct drm_connector *connector,
* below gate which of these entries are actually attempted.
*/
static const enum dc_pixel_encoding encoding_order[] = {
PIXEL_ENCODING_YCBCR444,
PIXEL_ENCODING_RGB,
PIXEL_ENCODING_YCBCR444,
PIXEL_ENCODING_YCBCR422,
PIXEL_ENCODING_YCBCR420,
};

6
drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_mst_types.c
View File

@ -1231,7 +1231,8 @@ static bool get_conv_frl_bw(struct amdgpu_dm_connector *aconnector,
unsigned int max_sink_bw_in_kbps = 0;
unsigned int dsc_max_sink_bw_in_kbps = 0;
if (aconnector->dc_link->dc->caps.dp_hdmi21_pcon_support &&
if (aconnector->dc_sink &&
aconnector->dc_link->dc->caps.dp_hdmi21_pcon_support &&
aconnector->mst_downstream_port_caps.bytes.byte0.bits.DWN_STRM_PORTX_TYPE == DOWN_STREAM_DETAILED_HDMI) {
max_conv_bw_in_kbps = dc_link_bw_kbps_from_raw_frl_link_rate_data(
aconnector->dc_link->dc,
@ -1996,6 +1997,9 @@ enum dc_status dm_dp_mst_is_port_support_mode(
struct dc_dsc_config_options dsc_options = {0};
uint32_t stream_kbps;
if (!aconnector->dc_sink)
return DC_FAIL_BANDWIDTH_VALIDATE;
/* DSC unnecessary case
* Check if timing could be supported within end-to-end BW
*/

117
drivers/gpu/drm/amd/display/amdgpu_dm/tests/amdgpu_dm_connector_test.c
View File

@ -409,7 +409,9 @@ static void dm_test_output_color_space_default_rgb_full(struct kunit *test)
{
struct dc_crtc_timing timing = {};
struct drm_connector_state state = {};
struct drm_connector *connector = kunit_kzalloc(test, sizeof(*connector), GFP_KERNEL);
state.connector = connector;
timing.pixel_encoding = PIXEL_ENCODING_RGB;
state.colorspace = DRM_MODE_COLORIMETRY_DEFAULT;
state.hdmi.broadcast_rgb = DRM_HDMI_BROADCAST_RGB_AUTO;
@ -559,7 +561,9 @@ static void dm_test_output_color_space_bt2020_rgb(struct kunit *test)
{
struct dc_crtc_timing timing = {};
struct drm_connector_state state = {};
struct drm_connector *connector = kunit_kzalloc(test, sizeof(*connector), GFP_KERNEL);
state.connector = connector;
timing.pixel_encoding = PIXEL_ENCODING_RGB;
state.colorspace = DRM_MODE_COLORIMETRY_BT2020_RGB;
@ -584,6 +588,110 @@ static void dm_test_output_color_space_bt2020_rgb_limited(struct kunit *test)
(int)COLOR_SPACE_2020_RGB_LIMITEDRANGE);
}
/**
* dm_test_output_color_space_default_rgb_auto_hdmi_cta - Automatic follows the CTA-861 default
* @test: The KUnit test context
*/
static void dm_test_output_color_space_default_rgb_auto_hdmi_cta(struct kunit *test)
{
struct dc_crtc_timing timing = {};
struct drm_connector_state state = {};
struct drm_connector *connector = kunit_kzalloc(test, sizeof(*connector), GFP_KERNEL);
connector->display_info.is_hdmi = true;
timing.pixel_encoding = PIXEL_ENCODING_RGB;
timing.vic = 32;
state.connector = connector;
state.colorspace = DRM_MODE_COLORIMETRY_DEFAULT;
state.hdmi.broadcast_rgb = DRM_HDMI_BROADCAST_RGB_AUTO;
KUNIT_EXPECT_EQ(test, (int)amdgpu_dm_get_output_color_space(&timing, &state),
(int)COLOR_SPACE_SRGB_LIMITED);
}
/**
* dm_test_output_color_space_default_rgb_auto_hdmi_vga - 640x480 (VIC 1) stays full range
* @test: The KUnit test context
*/
static void dm_test_output_color_space_default_rgb_auto_hdmi_vga(struct kunit *test)
{
struct dc_crtc_timing timing = {};
struct drm_connector_state state = {};
struct drm_connector *connector = kunit_kzalloc(test, sizeof(*connector), GFP_KERNEL);
connector->display_info.is_hdmi = true;
timing.pixel_encoding = PIXEL_ENCODING_RGB;
timing.vic = 1;
state.connector = connector;
state.colorspace = DRM_MODE_COLORIMETRY_DEFAULT;
state.hdmi.broadcast_rgb = DRM_HDMI_BROADCAST_RGB_AUTO;
KUNIT_EXPECT_EQ(test, (int)amdgpu_dm_get_output_color_space(&timing, &state),
(int)COLOR_SPACE_SRGB);
}
/**
* dm_test_output_color_space_default_rgb_auto_dp_cta - Automatic on a non-HDMI sink is full range
* @test: The KUnit test context
*/
static void dm_test_output_color_space_default_rgb_auto_dp_cta(struct kunit *test)
{
struct dc_crtc_timing timing = {};
struct drm_connector_state state = {};
struct drm_connector *connector = kunit_kzalloc(test, sizeof(*connector), GFP_KERNEL);
timing.pixel_encoding = PIXEL_ENCODING_RGB;
timing.vic = 32;
state.connector = connector;
state.colorspace = DRM_MODE_COLORIMETRY_DEFAULT;
state.hdmi.broadcast_rgb = DRM_HDMI_BROADCAST_RGB_AUTO;
KUNIT_EXPECT_EQ(test, (int)amdgpu_dm_get_output_color_space(&timing, &state),
(int)COLOR_SPACE_SRGB);
}
/**
* dm_test_output_color_space_default_rgb_full_hdmi_cta - Full overrides the CTA-861 default
* @test: The KUnit test context
*/
static void dm_test_output_color_space_default_rgb_full_hdmi_cta(struct kunit *test)
{
struct dc_crtc_timing timing = {};
struct drm_connector_state state = {};
struct drm_connector *connector = kunit_kzalloc(test, sizeof(*connector), GFP_KERNEL);
connector->display_info.is_hdmi = true;
timing.pixel_encoding = PIXEL_ENCODING_RGB;
timing.vic = 32;
state.connector = connector;
state.colorspace = DRM_MODE_COLORIMETRY_DEFAULT;
state.hdmi.broadcast_rgb = DRM_HDMI_BROADCAST_RGB_FULL;
KUNIT_EXPECT_EQ(test, (int)amdgpu_dm_get_output_color_space(&timing, &state),
(int)COLOR_SPACE_SRGB);
}
/**
* dm_test_output_color_space_bt2020_rgb_auto_hdmi_cta - BT.2020 RGB follows the same default
* @test: The KUnit test context
*/
static void dm_test_output_color_space_bt2020_rgb_auto_hdmi_cta(struct kunit *test)
{
struct dc_crtc_timing timing = {};
struct drm_connector_state state = {};
struct drm_connector *connector = kunit_kzalloc(test, sizeof(*connector), GFP_KERNEL);
connector->display_info.is_hdmi = true;
timing.pixel_encoding = PIXEL_ENCODING_RGB;
timing.vic = 32;
state.connector = connector;
state.colorspace = DRM_MODE_COLORIMETRY_BT2020_RGB;
state.hdmi.broadcast_rgb = DRM_HDMI_BROADCAST_RGB_AUTO;
KUNIT_EXPECT_EQ(test, (int)amdgpu_dm_get_output_color_space(&timing, &state),
(int)COLOR_SPACE_2020_RGB_LIMITEDRANGE);
}
/**
* dm_test_output_color_space_bt2020_ycc - Test Output color space bt2020 ycc
* @test: The KUnit test context
@ -647,7 +755,9 @@ static void dm_test_output_color_space_bt2020_ycc_rgb_encoding(struct kunit *tes
{
struct dc_crtc_timing timing = {};
struct drm_connector_state state = {};
struct drm_connector *connector = kunit_kzalloc(test, sizeof(*connector), GFP_KERNEL);
state.connector = connector;
timing.pixel_encoding = PIXEL_ENCODING_RGB;
state.colorspace = DRM_MODE_COLORIMETRY_BT2020_YCC;
@ -3425,6 +3535,7 @@ static struct dm_test_fill_ctx *dm_test_fill_ctx_alloc(struct kunit *test)
ctx->conn_state = drmm_kzalloc(ctx->drm, sizeof(*ctx->conn_state), GFP_KERNEL);
KUNIT_ASSERT_NOT_NULL(test, ctx->conn_state);
ctx->conn_state->connector = &ctx->aconnector->base;
ctx->stream = kunit_kzalloc(test, sizeof(*ctx->stream), GFP_KERNEL);
KUNIT_ASSERT_NOT_NULL(test, ctx->stream);
ctx->mode = kunit_kzalloc(test, sizeof(*ctx->mode), GFP_KERNEL);
@ -3818,6 +3929,7 @@ static struct dm_test_stream_ctx *dm_test_stream_ctx_alloc(struct kunit *test)
ctx->dm_state = kunit_kzalloc(test, sizeof(*ctx->dm_state), GFP_KERNEL);
KUNIT_ASSERT_NOT_NULL(test, ctx->dm_state);
ctx->dm_state->base.connector = &ctx->aconnector->base;
ctx->dm_state->scaling = RMX_OFF;
ctx->mode = kunit_kzalloc(test, sizeof(*ctx->mode), GFP_KERNEL);
@ -5458,6 +5570,11 @@ static struct kunit_case amdgpu_dm_connector_tests[] = {
KUNIT_CASE(dm_test_output_color_space_oprgb),
KUNIT_CASE(dm_test_output_color_space_bt2020_rgb),
KUNIT_CASE(dm_test_output_color_space_bt2020_rgb_limited),
KUNIT_CASE(dm_test_output_color_space_default_rgb_auto_hdmi_cta),
KUNIT_CASE(dm_test_output_color_space_default_rgb_auto_hdmi_vga),
KUNIT_CASE(dm_test_output_color_space_default_rgb_auto_dp_cta),
KUNIT_CASE(dm_test_output_color_space_default_rgb_full_hdmi_cta),
KUNIT_CASE(dm_test_output_color_space_bt2020_rgb_auto_hdmi_cta),
KUNIT_CASE(dm_test_output_color_space_bt2020_ycc),
KUNIT_CASE(dm_test_output_color_space_default_ycbcr709_y_only),
KUNIT_CASE(dm_test_output_color_space_default_ycbcr601),

1
drivers/gpu/drm/amd/display/dc/dc.h
View File

@ -1905,6 +1905,7 @@ struct dc_scratch_space {
struct ddc_service *ddc;
enum dp_panel_mode panel_mode;
bool panel_mode_initialized;
bool aux_mode;
/* Private to DC core */

2
drivers/gpu/drm/amd/display/dc/dml/Makefile
View File

@ -29,7 +29,7 @@ dml_ccflags := $(CC_FLAGS_FPU)
dml_rcflags := $(CC_FLAGS_NO_FPU)
ifneq ($(CONFIG_FRAME_WARN),0)
ifeq ($(filter y,$(CONFIG_KASAN)$(CONFIG_KCSAN)$(CONFIG_UBSAN)),y)
ifneq ($(filter y,$(CONFIG_KASAN) $(CONFIG_KCSAN) $(CONFIG_UBSAN)),)
ifeq ($(CONFIG_CC_IS_CLANG)$(CONFIG_COMPILE_TEST),yy)
frame_warn_limit := 4096
else

Some files were not shown because too many files have changed in this diff Show More