ASoC: cs-amp-lib: Some bug and typo fixes

Richard Fitzgerald <rf@opensource.cirrus.com> says:

This series fixes bugs and a typo in cs-amp-lib.

Link: https://patch.msgid.link/20260521122511.987322-1-rf@opensource.cirrus.com
This commit is contained in:
Mark Brown 2026-05-21 15:13:52 +01:00
commit 1129c0011b
No known key found for this signature in database
GPG Key ID: 24D68B725D5487D0
373 changed files with 4763 additions and 2397 deletions

View File

@ -857,6 +857,7 @@ Tobias Klauser <tklauser@distanz.ch> <klto@zhaw.ch>
Tobias Klauser <tklauser@distanz.ch> <tklauser@nuerscht.ch>
Tobias Klauser <tklauser@distanz.ch> <tklauser@xenon.tklauser.home>
Todor Tomov <todor.too@gmail.com> <todor.tomov@linaro.org>
Tomasz Jeznach <tomasz.jeznach@linux.dev> <tjeznach@rivosinc.com>
Tony Luck <tony.luck@intel.com>
Trilok Soni <quic_tsoni@quicinc.com> <tsoni@codeaurora.org>
TripleX Chung <xxx.phy@gmail.com> <triplex@zh-kernel.org>

View File

@ -21,13 +21,13 @@ call at each patchable function entry, and patches it dynamically at runtime to
enable or disable the redirection. In the case of RISC-V, 2 instructions,
AUIPC + JALR, are required to compose a function call. However, it is impossible
to patch 2 instructions and expect that a concurrent read-side executes them
without a race condition. This series makes atmoic code patching possible in
without a race condition. This series makes atomic code patching possible in
RISC-V ftrace. Kernel preemption makes things even worse as it allows the old
state to persist across the patching process with stop_machine().
In order to get rid of stop_machine() and run dynamic ftrace with full kernel
preemption, we partially initialize each patchable function entry at boot-time,
setting the first instruction to AUIPC, and the second to NOP. Now, atmoic
setting the first instruction to AUIPC, and the second to NOP. Now, atomic
patching is possible because the kernel only has to update one instruction.
According to Ziccif, as long as an instruction is naturally aligned, the ISA
guarantee an atomic update.
@ -36,8 +36,8 @@ By fixing down the first instruction, AUIPC, the range of the ftrace trampoline
is limited to +-2K from the predetermined target, ftrace_caller, due to the lack
of immediate encoding space in RISC-V. To address the issue, we introduce
CALL_OPS, where an 8B naturally align metadata is added in front of each
pacthable function. The metadata is resolved at the first trampoline, then the
execution can be derect to another custom trampoline.
patchable function. The metadata is resolved at the first trampoline, then the
execution can be directed to another custom trampoline.
CMODX in the User Space
-----------------------

View File

@ -78,7 +78,7 @@ the program.
Per-task indirect branch tracking state can be monitored and
controlled via the :c:macro:`PR_GET_CFI` and :c:macro:`PR_SET_CFI`
``prctl()` arguments (respectively), by supplying
``prctl()`` arguments (respectively), by supplying
:c:macro:`PR_CFI_BRANCH_LANDING_PADS` as the second argument. These
are architecture-agnostic, and will return -EINVAL if the underlying
functionality is not supported.

View File

@ -22,5 +22,5 @@ The following sensors are supported
sysfs-Interface
---------------
temp0_input
temp1_input
- Temperature of external NTC (milli-degree C)

View File

@ -69,6 +69,15 @@ properties:
header:
description: For C-compatible languages, header which already defines this value.
type: string
scope:
description: |
Visibility of this definition. "uapi" (default) renders into
the uAPI header, "kernel" renders into the kernel-side
generated header, "user" renders into the user-side
generated header. When combined with `header:`, the
definition is not rendered, and the named header is
included only by code matching the scope.
enum: [ uapi, kernel, user ]
type:
enum: [ const, enum, flags ]
doc:

View File

@ -83,6 +83,15 @@ properties:
header:
description: For C-compatible languages, header which already defines this value.
type: string
scope:
description: |
Visibility of this definition. "uapi" (default) renders into
the uAPI header, "kernel" renders into the kernel-side
generated header, "user" renders into the user-side
generated header. When combined with `header:`, the
definition is not rendered, and the named header is
included only by code matching the scope.
enum: [ uapi, kernel, user ]
type:
enum: [ const, enum, flags, struct ] # Trim
doc:

View File

@ -55,6 +55,15 @@ properties:
header:
description: For C-compatible languages, header which already defines this value.
type: string
scope:
description: |
Visibility of this definition. "uapi" (default) renders into
the uAPI header, "kernel" renders into the kernel-side
generated header, "user" renders into the user-side
generated header. When combined with `header:`, the
definition is not rendered, and the named header is
included only by code matching the scope.
enum: [ uapi, kernel, user ]
type:
enum: [ const, enum, flags ]
doc:

View File

@ -87,6 +87,15 @@ properties:
header:
description: For C-compatible languages, header which already defines this value.
type: string
scope:
description: |
Visibility of this definition. "uapi" (default) renders into
the uAPI header, "kernel" renders into the kernel-side
generated header, "user" renders into the user-side
generated header. When combined with `header:`, the
definition is not rendered, and the named header is
included only by code matching the scope.
enum: [ uapi, kernel, user ]
type:
enum: [ const, enum, flags, struct ] # Trim
doc:

View File

@ -33,6 +33,11 @@ doc: |
@cap-get operation.
definitions:
-
type: const
name: max-handle-id
value: 0x3fffffe
scope: kernel
-
type: enum
name: scope
@ -140,6 +145,8 @@ attribute-sets:
-
name: id
type: u32
checks:
max: max-handle-id
doc: |
Numeric identifier of a shaper. The id semantic depends on
the scope. For @queue scope it's the queue id and for @node

View File

@ -86,6 +86,7 @@ regressions and security problems.
debugging/index
handling-regressions
security-bugs
threat-model
cve
embargoed-hardware-issues

View File

@ -66,6 +66,42 @@ In addition, the following information are highly desirable:
the issue appear. It is useful to share them, as they can be helpful to
keep end users protected during the time it takes them to apply the fix.
What qualifies as a security bug
--------------------------------
It is important that most bugs are handled publicly so as to involve the widest
possible audience and find the best solution. By nature, bugs that are handled
in closed discussions between a small set of participants are less likely to
produce the best possible fix (e.g., risk of missing valid use cases, limited
testing abilities).
It turns out that the majority of the bugs reported via the security team are
just regular bugs that have been improperly qualified as security bugs due to
a lack of awareness of the Linux kernel's threat model, as described in
Documentation/process/threat-model.rst, and ought to have been sent through
the normal channels described in Documentation/admin-guide/reporting-issues.rst
instead.
The security list exists for urgent bugs that grant an attacker a capability
they are not supposed to have on a correctly configured production system, and
can be easily exploited, representing an imminent threat to many users. Before
reporting, consider whether the issue actually crosses a trust boundary on such
a system.
**If you resorted to AI assistance to identify a bug, you must treat it as
public**. While you may have valid reasons to believe it is not, the security
team's experience shows that bugs discovered this way systematically surface
simultaneously across multiple researchers, often on the same day. In this
case, do not publicly share a reproducer, as this could cause unintended harm;
just mention that one is available and maintainers might ask for it privately
if they need it.
If you are unsure whether an issue qualifies, err on the side of reporting
privately: the security team would rather triage a borderline report than miss
a real vulnerability. Reporting ordinary bugs to the security list, however,
does not make them move faster and consumes triage capacity that other reports
need.
Identifying contacts
--------------------
@ -74,7 +110,7 @@ affected subsystem's maintainers and Cc: the Linux kernel security team. Do
not send it to a public list at this stage, unless you have good reasons to
consider the issue as being public or trivial to discover (e.g. result of a
widely available automated vulnerability scanning tool that can be repeated by
anyone).
anyone, or use of AI-based tools).
If you're sending a report for issues affecting multiple parts in the kernel,
even if they're fairly similar issues, please send individual messages (think
@ -131,6 +167,64 @@ the Linux kernel security team only. Your message will be triaged, and you
will receive instructions about whom to contact, if needed. Your message may
equally be forwarded as-is to the relevant maintainers.
Responsible use of AI to find bugs
----------------------------------
A significant fraction of bug reports submitted to the security team are
actually the result of code reviews assisted by AI tools. While this can be an
efficient means to find bugs in rarely explored areas, it causes an overload on
maintainers, who are sometimes forced to ignore such reports due to their poor
quality or accuracy. As such, reporters must be particularly cautious about a
number of points which tend to make these reports needlessly difficult to
handle:
* **Length**: AI-generated reports tend to be excessively long, containing
multiple sections and excessive detail. This makes it difficult to spot
important information such as affected files, versions, and impact. Please
ensure that a clear summary of the problem and all critical details are
presented first. Do not require triage engineers to scan multiple pages of
text. Configure your tools to produce concise, human-style reports.
* **Formatting**: Most AI-generated reports are littered with Markdown tags.
These decorations complicate the search for important information and do
not survive the quoting processes involved in forwarding or replying.
Please **always convert your report to plain text** without any formatting
decorations before sending it.
* **Impact Evaluation**: Many AI-generated reports lack an understanding
of the kernel's threat model (see Documentation/process/threat-model.rst)
and go to great lengths inventing theoretical consequences. This adds
noise and complicates triage. Please stick to verifiable facts (e.g.,
"this bug permits any user to gain CAP_NET_ADMIN") without enumerating
speculative implications. Have your tool read this documentation as
part of the evaluation process.
* **Reproducer**: AI-based tools are often capable of generating reproducers.
Please always ensure your tool provides one and **test it thoroughly**. If
the reproducer does not work, or if the tool cannot produce one, the
validity of the report should be seriously questioned. Note that since the
report will be posted to a public list, the reproducer should only be
shared upon maintainers' request.
* **Propose a Fix**: Many AI tools are actually better at writing code than
evaluating it. Please ask your tool to propose a fix and **test it** before
reporting the problem. If the fix cannot be tested because it relies on
rare hardware or almost extinct network protocols, the issue is likely not
a security bug. In any case, if a fix is proposed, it must adhere to
Documentation/process/submitting-patches.rst and include a 'Fixes:' tag
designating the commit that introduced the bug.
Failure to consider these points exposes your report to the risk of being
ignored.
Use common sense when evaluating the report. If the affected file has not been
touched for more than one year and is maintained by a single individual, it is
likely that usage has declined and exposed users are virtually non-existent
(e.g., drivers for very old hardware, obsolete filesystems). In such cases,
there is no need to consume a maintainer's time with an unimportant report. If
the issue is clearly trivial and publicly discoverable, you should report it
directly to the public mailing lists.
Sending the report
------------------
@ -148,7 +242,15 @@ run additional tests. Reports where the reporter does not respond promptly
or cannot effectively discuss their findings may be abandoned if the
communication does not quickly improve.
The report must be sent to maintainers, with the security team in ``Cc:``.
The report must be sent to maintainers. If there are two or fewer
recipients in your message, you must also always Cc: the Linux kernel
security team who will ensure the message is delivered to the proper
people, and will be able to assist small maintainer teams with processes
they may not be familiar with. For larger teams, Cc: the Linux kernel
security team for your first few reports or when seeking specific help,
such as when resending a message which got no response within a week.
Once you have become comfortable with the process for a few reports, it is
no longer necessary to Cc: the security list when sending to large teams.
The Linux kernel security team can be contacted by email at
<security@kernel.org>. This is a private list of security officers
who will help verify the bug report and assist developers working on a fix.

View File

@ -0,0 +1,235 @@
The Linux Kernel threat model
=============================
There are a lot of assumptions regarding what the kernel does and does not
protect against. These assumptions tend to cause confusion for bug reports
(:doc:`security-related ones <security-bugs>` vs :doc:`non-security ones
<../admin-guide/reporting-issues>`), and can complicate security enforcement
when the responsibilities for some boundaries is not clear between the kernel,
distros, administrators and users.
This document tries to clarify the responsibilities of the kernel in this
domain.
The kernel's responsibilities
-----------------------------
The kernel abstracts access to local hardware resources and to remote systems
in a way that allows multiple local users to get a fair share of the available
resources granted to them, and, when the underlying hardware permits, to assign
a level of confidentiality to their communications and to the data they are
processing or storing.
The kernel assumes that the underlying hardware behaves according to its
specifications. This includes the integrity of the CPU's instruction set, the
transparency of the branch prediction unit and the cache units, the consistency
of the Memory Management Unit (MMU), the isolation of DMA-capable peripherals
(e.g., via IOMMU), state transitions in controllers, ranges of values read from
registers, the respect of documented hardware limitations, etc.
When hardware fails to maintain its specified isolation (e.g., CPU bugs,
side-channels, hardware response to unexpected inputs), the kernel will usually
attempt to implement reasonable mitigations. These are best-effort measures
intended to reduce the attack surface or elevate the cost of an attack within
the limits of the hardware's facilities; they do not constitute a
kernel-provided safety guarantee.
Users always perform their activities under the authority of an administrator
who is able to grant or deny various types of permissions that may affect how
users benefit from available resources, or the level of confidentiality of
their activities. Administrators may also delegate all or part of their own
permissions to some users, particularly via capabilities but not only. All this
is performed via configuration (sysctl, file-system permissions etc).
The Linux Kernel applies a certain collection of default settings that match
its threat model. Distros have their own threat model and will come with their
own configuration presets, that the administrator may have to adjust to better
suit their expectations (relax or restrict).
By default, the Linux Kernel guarantees the following protections when running
on common processors featuring privilege levels and memory management units:
* **User-based isolation**: an unprivileged user may restrict access to their
own data from other unprivileged users running on the same system. This
includes:
* stored data, via file system permissions
* in-memory data (pages are not accessible by default to other users)
* process activity (ptrace is not permitted to other users)
* inter-process communication (other users may not observe data exchanged via
UNIX domain sockets or other IPC mechanisms).
* network communications within the same or with other systems
* **Capability-based protection**:
* users not having elevated capabilities (including but not limited to
CAP_SYS_ADMIN) may not alter the
kernel's configuration, memory nor state, change other users' view of the
file system layout, grant any user capabilities they do not have, nor
affect the system's availability (shutdown, reboot, panic, hang, or making
the system unresponsive via unbounded resource exhaustion).
* users not having the ``CAP_NET_ADMIN`` capability may not alter the network
configuration, intercept nor spoof network communications from other users
nor systems.
* users not having ``CAP_SYS_PTRACE`` may not observe other users' processes
activities.
When ``CONFIG_USER_NS`` is set, the kernel also permits unprivileged users to
create their own user namespace in which they have all capabilities, but with a
number of restrictions (they may not perform actions that have impacts on the
initial user namespace, such as changing time, loading modules or mounting
block devices). Please refer to ``user_namespaces(7)`` for more details, the
possibilities of user namespaces are not covered in this document.
The kernel also offers a lot of troubleshooting and debugging facilities, which
can constitute attack vectors when placed in wrong hands. While some of them
are designed to be accessible to regular local users with a low risk (e.g.
kernel logs via ``/proc/kmsg``), some would expose enough information to
represent a risk in most places and the decision to expose them is under the
administrator's responsibility (perf events, traces), and others are not
designed to be accessed by non-privileged users (e.g. debugfs). Access to these
facilities by a user who has been explicitly granted permission by an
administrator does not constitute a security breach.
Bugs that permit to violate the principles above constitute security breaches.
However, bugs that permit one violation only once another one was already
achieved are only weaknesses. The kernel applies a number of self-protection
measures whose purpose is to avoid crossing a security boundary when certain
classes of bugs are found, but a failure of these extra protections do not
constitute a vulnerability alone.
What does not constitute a security bug
---------------------------------------
In the Linux kernel's threat model, the following classes of problems are
**NOT** considered as Linux Kernel security bugs. However, when it is believed
that the kernel could do better, they should be reported, so that they can be
reviewed and fixed where reasonably possible, but they will be handled as any
regular bug:
* **Configuration**:
* outdated kernels and particularly end-of-life branches are out of the scope
of the kernel's threat model: administrators are responsible for keeping
their system up to date. For a bug to qualify as a security bug, it must be
demonstrated that it affects actively maintained versions.
* build-level: changes to the kernel configuration that are explicitly
documented as lowering the security level (e.g. ``CONFIG_NOMMU``), or
targeted at developers only.
* OS-level: changes to command line parameters, sysctls, filesystem
permissions, user capabilities, exposure of privileged interfaces, that
explicitly increase exposure by either offering non-default access to
unprivileged users, or reduce the kernel's ability to enforce some
protections or mitigations. Example: write access to procfs or debugfs.
* issues triggered only when using features intended for development or
debugging (e.g., LOCKDEP, KASAN, FAULT_INJECTION): these features are known
to introduce overhead and potential instability and are not intended for
production use.
* issues affecting drivers exposed under CONFIG_STAGING, as well as features
marked EXPERIMENTAL in the configuration.
* loading of explicitly insecure/broken/staging modules, and generally any
using any subsystem marked as experimental or not intended for production
use.
* running out-of-tree modules or unofficial kernel forks; these should be
reported to the relevant vendor.
* **Excess of initial privileges**:
* actions performed by a user already possessing the privileges required to
perform that action or modify that state (e.g. ``CAP_SYS_ADMIN``,
``CAP_NET_ADMIN``, ``CAP_SYS_RAWIO``, ``CAP_SYS_MODULE`` with no further
boundary being crossed).
* actions performed in user namespace that do not bypass the restrictions
imposed to the initial user (e.g. ptrace usage, signal delivery, resource
usage, access to FS/device/sysctl/memory, network binding, system/network
configuration etc).
* anything performed by the root user in the initial namespace (e.g. kernel
oops when writing to a privileged device).
* **Out of production use**:
This covers theoretical/probabilistic attacks that rely on laboratory
conditions with zero system noise, or those requiring an unrealistic number
of attempts (e.g., billions of trials) that would be detected by standard
system monitoring long before success, such as:
* prediction of random numbers that only works in a totally silent
environment (such as IP ID, TCP ports or sequence numbers that can only be
guessed in a lab).
* activity observation and information leaks based on probabilistic
approaches that are prone to measurement noise and not realistically
reproducible on a production system.
* issues that can only be triggered by heavy attacks (e.g. brute force) whose
impact on the system makes it unlikely or impossible to remain undetected
before they succeed (e.g. consuming all memory before succeeding).
* problems seen only under development simulators, emulators, or combinations
that do not exist on real systems at the time of reporting (issues
involving tens of millions of threads, tens of thousands of CPUs,
unrealistic CPU frequencies, RAM sizes or disk capacities, network speeds.
* issues whose reproduction requires hardware modification or emulation,
including fake USB devices that pretend to be another one.
* as well as issues that can be triggered at a cost that is orders of
magnitude higher than the expected benefits (e.g. fully functional keyboard
emulator only to retrieve 7 uninitialized bytes in a structure, or
brute-force method involving millions of connection attempts to guess a
port number).
* **Hardening failures**:
* ability to bypass some of the kernel's hardening measures with no
demonstrable exploit path (e.g. ASLR bypass, events timing or probing with
no demonstrable consequence). These are just weaknesses, not
vulnerabilities.
* missing argument checks and failure to report certain errors with no
immediate consequence.
* **Random information leaks**:
This concerns information leaks of small data parts that happen to be there
and that cannot be chosen by the attacker, or face access restrictions:
* structure padding reported by syscalls or other interfaces.
* identifiers, partial data, non-terminated strings reported in error
messages.
* Leaks of kernel memory addresses/pointers do not constitute an immediately
exploitable vector and are not security bugs, though they must be reported
and fixed.
* **Crafted file system images**:
* bugs triggered by mounting a corrupted or maliciously crafted file system
image are generally not security bugs, as the kernel assumes the underlying
storage media is under the administrator's control, unless the filesystem
driver is specifically documented as being hardened against untrusted media.
* issues that are resolved, mitigated, or detected by running a filesystem
consistency check (fsck) on the image prior to mounting.
* **Physical access**:
Issues that require physical access to the machine, hardware modification, or
the use of specialized hardware (e.g., logic analyzers, DMA-attack tools over
PCI-E/Thunderbolt) are out of scope unless the system is explicitly
configured with technologies meant to defend against such attacks
(e.g. IOMMU).
* **Functional and performance regressions**:
Any issue that can be mitigated by setting proper permissions and limits
doesn't qualify as a security bug.

View File

@ -40,7 +40,7 @@ There are two drivers in the kernel
*For systems using SoundWire*: sound/soc/codecs/cs35l56.c and associated files
*For systems using HDA*: sound/pci/hda/cs35l56_hda.c
*For systems using HDA*: sound/hda/codecs/side-codecs/cs35l56_hda.c
Firmware
========

View File

@ -656,8 +656,8 @@ References
See [white-paper]_, [api-spec]_, [amd-apm]_, [kvm-forum]_, and [snp-fw-abi]_
for more info.
.. [white-paper] https://developer.amd.com/wordpress/media/2013/12/AMD_Memory_Encryption_Whitepaper_v7-Public.pdf
.. [api-spec] https://support.amd.com/TechDocs/55766_SEV-KM_API_Specification.pdf
.. [amd-apm] https://support.amd.com/TechDocs/24593.pdf (section 15.34)
.. [white-paper] https://docs.amd.com/v/u/en-US/memory-encryption-white-paper
.. [api-spec] https://docs.amd.com/v/u/en-US/55766_PUB_3.24_SEV_API
.. [amd-apm] https://docs.amd.com/v/u/en-US/24593_3.44_APM_Vol2 (section 15.34)
.. [kvm-forum] https://www.linux-kvm.org/images/7/74/02x08A-Thomas_Lendacky-AMDs_Virtualizatoin_Memory_Encryption_Technology.pdf
.. [snp-fw-abi] https://www.amd.com/system/files/TechDocs/56860.pdf
.. [snp-fw-abi] https://www.amd.com/content/dam/amd/en/documents/developer/56860.pdf

View File

@ -68,6 +68,12 @@ Maintainers List
first. When adding to this list, please keep the entries in
alphabetical order.
3C509 NETWORK DRIVER
M: "Maciej W. Rozycki" <macro@orcam.me.uk>
L: netdev@vger.kernel.org
S: Maintained
F: drivers/net/ethernet/3com/3c509.c
3C59X NETWORK DRIVER
M: Steffen Klassert <klassert@kernel.org>
L: netdev@vger.kernel.org
@ -2015,7 +2021,7 @@ F: Documentation/hwmon/aquacomputer_d5next.rst
F: drivers/hwmon/aquacomputer_d5next.c
AQUANTIA ETHERNET DRIVER (atlantic)
M: Igor Russkikh <irusskikh@marvell.com>
M: Sukhdeep Singh <sukhdeeps@marvell.com>
L: netdev@vger.kernel.org
S: Maintained
W: https://www.marvell.com/
@ -2024,7 +2030,7 @@ F: Documentation/networking/device_drivers/ethernet/aquantia/atlantic.rst
F: drivers/net/ethernet/aquantia/atlantic/
AQUANTIA ETHERNET DRIVER PTP SUBSYSTEM
M: Egor Pomozov <epomozov@marvell.com>
M: Sukhdeep Singh <sukhdeeps@marvell.com>
L: netdev@vger.kernel.org
S: Maintained
W: http://www.aquantia.com
@ -4181,8 +4187,8 @@ F: include/uapi/linux/sonet.h
F: net/atm/
ATMEL MACB ETHERNET DRIVER
M: Nicolas Ferre <nicolas.ferre@microchip.com>
M: Claudiu Beznea <claudiu.beznea@tuxon.dev>
M: Théo Lebrun <theo.lebrun@bootlin.com>
R: Conor Dooley <conor.dooley@microchip.com>
S: Maintained
F: drivers/net/ethernet/cadence/
@ -14054,6 +14060,7 @@ KERNEL VIRTUAL MACHINE FOR ARM64 (KVM/arm64)
M: Marc Zyngier <maz@kernel.org>
M: Oliver Upton <oupton@kernel.org>
R: Joey Gouly <joey.gouly@arm.com>
R: Steffen Eiden <seiden@linux.ibm.com>
R: Suzuki K Poulose <suzuki.poulose@arm.com>
R: Zenghui Yu <yuzenghui@huawei.com>
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
@ -22942,7 +22949,7 @@ N: riscv
K: riscv
RISC-V IOMMU
M: Tomasz Jeznach <tjeznach@rivosinc.com>
M: Tomasz Jeznach <tomasz.jeznach@linux.dev>
L: iommu@lists.linux.dev
L: linux-riscv@lists.infradead.org
S: Maintained

View File

@ -2,7 +2,7 @@
VERSION = 7
PATCHLEVEL = 1
SUBLEVEL = 0
EXTRAVERSION = -rc3
EXTRAVERSION = -rc4
NAME = Baby Opossum Posse
# *DOCUMENTATION*

View File

@ -23,6 +23,7 @@ static inline u64 tcr_el2_ps_to_tcr_el1_ips(u64 tcr_el2)
static inline u64 translate_tcr_el2_to_tcr_el1(u64 tcr)
{
return TCR_EPD1_MASK | /* disable TTBR1_EL1 */
((tcr & TCR_EL2_DS) ? TCR_DS : 0) |
((tcr & TCR_EL2_TBI) ? TCR_TBI0 : 0) |
tcr_el2_ps_to_tcr_el1_ips(tcr) |
(tcr & TCR_EL2_TG0_MASK) |

View File

@ -844,7 +844,7 @@
#define INIT_SCTLR_EL2_MMU_ON \
(SCTLR_ELx_M | SCTLR_ELx_C | SCTLR_ELx_SA | SCTLR_ELx_I | \
SCTLR_ELx_IESB | SCTLR_ELx_WXN | ENDIAN_SET_EL2 | \
SCTLR_ELx_ITFSB | SCTLR_EL2_RES1)
SCTLR_ELx_ITFSB | SCTLR_ELx_EIS | SCTLR_ELx_EOS | SCTLR_EL2_RES1)
#define INIT_SCTLR_EL2_MMU_OFF \
(SCTLR_EL2_RES1 | ENDIAN_SET_EL2)

View File

@ -62,6 +62,13 @@ static void noinstr arm64_exit_to_kernel_mode(struct pt_regs *regs,
irqentry_exit_to_kernel_mode_after_preempt(regs, state);
}
static __always_inline void arm64_syscall_enter_from_user_mode(struct pt_regs *regs)
{
enter_from_user_mode(regs);
mte_disable_tco_entry(current);
sme_enter_from_user_mode();
}
/*
* Handle IRQ/context state management when entering from user mode.
* Before this function is called it is not safe to call regular kernel code,
@ -70,20 +77,30 @@ static void noinstr arm64_exit_to_kernel_mode(struct pt_regs *regs,
static __always_inline void arm64_enter_from_user_mode(struct pt_regs *regs)
{
enter_from_user_mode(regs);
rseq_note_user_irq_entry();
mte_disable_tco_entry(current);
sme_enter_from_user_mode();
}
static __always_inline void arm64_syscall_exit_to_user_mode(struct pt_regs *regs)
{
local_irq_disable();
syscall_exit_to_user_mode_prepare(regs);
local_daif_mask();
sme_exit_to_user_mode();
mte_check_tfsr_exit();
exit_to_user_mode();
}
/*
* Handle IRQ/context state management when exiting to user mode.
* After this function returns it is not safe to call regular kernel code,
* instrumentable code, or any code which may trigger an exception.
*/
static __always_inline void arm64_exit_to_user_mode(struct pt_regs *regs)
{
local_irq_disable();
exit_to_user_mode_prepare_legacy(regs);
irqentry_exit_to_user_mode_prepare(regs);
local_daif_mask();
sme_exit_to_user_mode();
mte_check_tfsr_exit();
@ -92,7 +109,7 @@ static __always_inline void arm64_exit_to_user_mode(struct pt_regs *regs)
asmlinkage void noinstr asm_exit_to_user_mode(struct pt_regs *regs)
{
arm64_exit_to_user_mode(regs);
arm64_syscall_exit_to_user_mode(regs);
}
/*
@ -716,12 +733,12 @@ static void noinstr el0_brk64(struct pt_regs *regs, unsigned long esr)
static void noinstr el0_svc(struct pt_regs *regs)
{
arm64_enter_from_user_mode(regs);
arm64_syscall_enter_from_user_mode(regs);
cortex_a76_erratum_1463225_svc_handler();
fpsimd_syscall_enter();
local_daif_restore(DAIF_PROCCTX);
do_el0_svc(regs);
arm64_exit_to_user_mode(regs);
arm64_syscall_exit_to_user_mode(regs);
fpsimd_syscall_exit();
}
@ -868,11 +885,11 @@ static void noinstr el0_cp15(struct pt_regs *regs, unsigned long esr)
static void noinstr el0_svc_compat(struct pt_regs *regs)
{
arm64_enter_from_user_mode(regs);
arm64_syscall_enter_from_user_mode(regs);
cortex_a76_erratum_1463225_svc_handler();
local_daif_restore(DAIF_PROCCTX);
do_el0_svc_compat(regs);
arm64_exit_to_user_mode(regs);
arm64_syscall_exit_to_user_mode(regs);
}
static void noinstr el0_bkpt32(struct pt_regs *regs, unsigned long esr)

View File

@ -4,6 +4,7 @@
* Author: Christoffer Dall <c.dall@virtualopensystems.com>
*/
#include <linux/arm-smccc.h>
#include <linux/bug.h>
#include <linux/cpu_pm.h>
#include <linux/errno.h>
@ -2638,6 +2639,22 @@ static int init_pkvm_host_sve_state(void)
return 0;
}
static int pkvm_check_sme_dvmsync_fw_call(void)
{
struct arm_smccc_res res;
if (!cpus_have_final_cap(ARM64_WORKAROUND_4193714))
return 0;
arm_smccc_1_1_smc(ARM_SMCCC_CPU_WORKAROUND_4193714, &res);
if (res.a0) {
kvm_err("pKVM requires firmware support for C1-Pro erratum 4193714\n");
return -ENODEV;
}
return 0;
}
/*
* Finalizes the initialization of hyp mode, once everything else is initialized
* and the initialziation process cannot fail.
@ -2838,6 +2855,10 @@ static int __init init_hyp_mode(void)
if (err)
goto out_err;
err = pkvm_check_sme_dvmsync_fw_call();
if (err)
goto out_err;
err = kvm_hyp_init_protection(hyp_va_bits);
if (err) {
kvm_err("Failed to init hyp memory protection\n");

View File

@ -245,7 +245,7 @@ static inline void __activate_traps_ich_hfgxtr(struct kvm_vcpu *vcpu)
__activate_fgt(hctxt, vcpu, ICH_HFGITR_EL2);
}
#define __deactivate_fgt(htcxt, vcpu, reg) \
#define __deactivate_fgt(hctxt, vcpu, reg) \
do { \
write_sysreg_s(ctxt_sys_reg(hctxt, reg), \
SYS_ ## reg); \

View File

@ -35,6 +35,9 @@ void trace_clock_update(u32 mult, u32 shift, u64 epoch_ns, u64 epoch_cyc)
struct clock_data *clock = &trace_clock_data;
u64 bank = clock->cur ^ 1;
if (!mult || shift >= 64)
return;
clock->data[bank].mult = mult;
clock->data[bank].shift = shift;
clock->data[bank].epoch_ns = epoch_ns;

View File

@ -5,6 +5,7 @@
*/
#include <linux/kvm_host.h>
#include <asm/kvm_emulate.h>
#include <asm/kvm_hyp.h>
#include <asm/kvm_mmu.h>
@ -14,6 +15,7 @@
#include <hyp/fault.h>
#include <nvhe/arm-smccc.h>
#include <nvhe/gfp.h>
#include <nvhe/memory.h>
#include <nvhe/mem_protect.h>
@ -29,6 +31,19 @@ static struct hyp_pool host_s2_pool;
static DEFINE_PER_CPU(struct pkvm_hyp_vm *, __current_vm);
#define current_vm (*this_cpu_ptr(&__current_vm))
static void pkvm_sme_dvmsync_fw_call(void)
{
if (alternative_has_cap_unlikely(ARM64_WORKAROUND_4193714)) {
struct arm_smccc_res res;
/*
* Ignore the return value. Probing for the workaround
* availability took place in init_hyp_mode().
*/
hyp_smccc_1_1_smc(ARM_SMCCC_CPU_WORKAROUND_4193714, &res);
}
}
static void guest_lock_component(struct pkvm_hyp_vm *vm)
{
hyp_spin_lock(&vm->lock);
@ -574,8 +589,14 @@ static int host_stage2_set_owner_metadata_locked(phys_addr_t addr, u64 size,
ret = host_stage2_try(kvm_pgtable_stage2_annotate, &host_mmu.pgt,
addr, size, &host_s2_pool,
KVM_HOST_INVALID_PTE_TYPE_DONATION, annotation);
if (!ret)
if (!ret) {
/*
* After stage2 maintenance has happened, but before the page
* owner has changed.
*/
pkvm_sme_dvmsync_fw_call();
__host_update_page_state(addr, size, PKVM_NOPAGE);
}
return ret;
}
@ -1369,6 +1390,22 @@ int __pkvm_host_reclaim_page_guest(u64 gfn, struct pkvm_hyp_vm *vm)
return ret && ret != -EHWPOISON ? ret : 0;
}
/*
* share/donate install at most one stage-2 leaf (PAGE_SIZE, or one
* KVM_PGTABLE_LAST_LEVEL - 1 block for share). kvm_mmu_cache_min_pages()
* bounds the worst-case allocation: exact for the PAGE_SIZE leaf,
* conservative by one for the block.
*/
static int __guest_check_pgtable_memcache(struct pkvm_hyp_vcpu *vcpu)
{
struct pkvm_hyp_vm *vm = pkvm_hyp_vcpu_to_hyp_vm(vcpu);
if (vcpu->vcpu.arch.pkvm_memcache.nr_pages < kvm_mmu_cache_min_pages(vm->pgt.mmu))
return -ENOMEM;
return 0;
}
int __pkvm_host_donate_guest(u64 pfn, u64 gfn, struct pkvm_hyp_vcpu *vcpu)
{
struct pkvm_hyp_vm *vm = pkvm_hyp_vcpu_to_hyp_vm(vcpu);
@ -1388,6 +1425,10 @@ int __pkvm_host_donate_guest(u64 pfn, u64 gfn, struct pkvm_hyp_vcpu *vcpu)
if (ret)
goto unlock;
ret = __guest_check_pgtable_memcache(vcpu);
if (ret)
goto unlock;
meta = host_stage2_encode_gfn_meta(vm, gfn);
WARN_ON(host_stage2_set_owner_metadata_locked(phys, PAGE_SIZE,
PKVM_ID_GUEST, meta));
@ -1453,6 +1494,10 @@ int __pkvm_host_share_guest(u64 pfn, u64 gfn, u64 nr_pages, struct pkvm_hyp_vcpu
}
}
ret = __guest_check_pgtable_memcache(vcpu);
if (ret)
goto unlock;
for_each_hyp_page(page, phys, size) {
set_host_state(page, PKVM_PAGE_SHARED_OWNED);
page->host_share_guest_count++;

View File

@ -752,16 +752,30 @@ static struct pkvm_hyp_vcpu selftest_vcpu = {
struct pkvm_hyp_vcpu *init_selftest_vm(void *virt)
{
struct hyp_page *p = hyp_virt_to_page(virt);
unsigned long min_pages, seeded = 0;
int i;
selftest_vm.kvm.arch.mmu.vtcr = host_mmu.arch.mmu.vtcr;
WARN_ON(kvm_guest_prepare_stage2(&selftest_vm, virt));
/*
* Mirror pkvm_refill_memcache() for the share/donate pre-checks;
* the selftest invokes those functions directly and would
* otherwise see an empty memcache.
*/
min_pages = kvm_mmu_cache_min_pages(&selftest_vm.kvm.arch.mmu);
for (i = 0; i < pkvm_selftest_pages(); i++) {
if (p[i].refcount)
continue;
p[i].refcount = 1;
hyp_put_page(&selftest_vm.pool, hyp_page_to_virt(&p[i]));
if (seeded < min_pages) {
push_hyp_memcache(&selftest_vcpu.vcpu.arch.pkvm_memcache,
hyp_page_to_virt(&p[i]), hyp_virt_to_phys);
seeded++;
} else {
hyp_put_page(&selftest_vm.pool, hyp_page_to_virt(&p[i]));
}
}
selftest_vm.kvm.arch.pkvm.handle = __pkvm_reserve_vm();

View File

@ -663,7 +663,8 @@ static void __noreturn __hyp_call_panic(u64 spsr, u64 elr, u64 par)
host_ctxt = host_data_ptr(host_ctxt);
vcpu = host_ctxt->__hyp_running_vcpu;
__deactivate_traps(vcpu);
if (vcpu)
__deactivate_traps(vcpu);
sysreg_restore_host_state_vhe(host_ctxt);
panic("HYP panic:\nPS:%08llx PC:%016llx ESR:%08llx\nFAR:%016llx HPFAR:%016llx PAR:%016llx\nVCPU:%p\n",

View File

@ -1576,21 +1576,24 @@ struct kvm_s2_fault_desc {
static int gmem_abort(const struct kvm_s2_fault_desc *s2fd)
{
bool write_fault, exec_fault;
bool perm_fault = kvm_vcpu_trap_is_permission_fault(s2fd->vcpu);
enum kvm_pgtable_walk_flags flags = KVM_PGTABLE_WALK_SHARED;
enum kvm_pgtable_prot prot = KVM_PGTABLE_PROT_R;
struct kvm_pgtable *pgt = s2fd->vcpu->arch.hw_mmu->pgt;
unsigned long mmu_seq;
struct page *page;
struct kvm *kvm = s2fd->vcpu->kvm;
void *memcache;
void *memcache = NULL;
kvm_pfn_t pfn;
gfn_t gfn;
int ret;
memcache = get_mmu_memcache(s2fd->vcpu);
ret = topup_mmu_memcache(s2fd->vcpu, memcache);
if (ret)
return ret;
if (!perm_fault) {
memcache = get_mmu_memcache(s2fd->vcpu);
ret = topup_mmu_memcache(s2fd->vcpu, memcache);
if (ret)
return ret;
}
if (s2fd->nested)
gfn = kvm_s2_trans_output(s2fd->nested) >> PAGE_SHIFT;
@ -1631,9 +1634,19 @@ static int gmem_abort(const struct kvm_s2_fault_desc *s2fd)
goto out_unlock;
}
ret = KVM_PGT_FN(kvm_pgtable_stage2_map)(pgt, s2fd->fault_ipa, PAGE_SIZE,
__pfn_to_phys(pfn), prot,
memcache, flags);
if (perm_fault) {
/*
* Drop the SW bits in favour of those stored in the
* PTE, which will be preserved.
*/
prot &= ~KVM_NV_GUEST_MAP_SZ;
ret = KVM_PGT_FN(kvm_pgtable_stage2_relax_perms)(pgt, s2fd->fault_ipa,
prot, flags);
} else {
ret = KVM_PGT_FN(kvm_pgtable_stage2_map)(pgt, s2fd->fault_ipa, PAGE_SIZE,
__pfn_to_phys(pfn), prot,
memcache, flags);
}
out_unlock:
kvm_release_faultin_page(kvm, page, !!ret, prot & KVM_PGTABLE_PROT_W);

View File

@ -83,11 +83,10 @@ config MSI_BITMAP_SELFTEST
depends on DEBUG_KERNEL
config GUEST_STATE_BUFFER_TEST
def_tristate n
def_tristate KUNIT_ALL_TESTS
prompt "Enable Guest State Buffer unit tests"
depends on KUNIT
depends on KVM_BOOK3S_HV_POSSIBLE
default KUNIT_ALL_TESTS
help
The Guest State Buffer is a data format specified in the PAPR.
It is by hcalls to communicate the state of L2 guests between

View File

@ -85,6 +85,8 @@ CONFIG_PMAC_SMU=y
CONFIG_MAC_EMUMOUSEBTN=y
CONFIG_WINDFARM=y
CONFIG_WINDFARM_PM81=y
CONFIG_WINDFARM_PM72=y
CONFIG_WINDFARM_RM31=y
CONFIG_WINDFARM_PM91=y
CONFIG_WINDFARM_PM112=y
CONFIG_WINDFARM_PM121=y

View File

@ -79,10 +79,6 @@ extern int pmac_i2c_match_adapter(struct device_node *dev,
struct i2c_adapter *adapter);
/* (legacy) Locking functions exposed to i2c-keywest */
extern int pmac_low_i2c_lock(struct device_node *np);
extern int pmac_low_i2c_unlock(struct device_node *np);
/* Access functions for platform code */
extern int pmac_i2c_open(struct pmac_i2c_bus *bus, int polled);
extern void pmac_i2c_close(struct pmac_i2c_bus *bus);

View File

@ -458,6 +458,10 @@ DEFINE_PER_CPU(u8, irq_work_pending);
#endif /* 32 vs 64 bit */
/*
* Must be called with preemption disabled since it updates
* per-CPU irq_work state and programs the local CPU decrementer.
*/
void arch_irq_work_raise(void)
{
/*
@ -471,10 +475,8 @@ void arch_irq_work_raise(void)
* which could get tangled up if we're messing with the same state
* here.
*/
preempt_disable();
set_irq_work_pending_flag();
set_dec(1);
preempt_enable();
}
static void set_dec_or_work(u64 val)

View File

@ -210,7 +210,7 @@ static ssize_t processor_bus_topology_show(struct device *dev, struct device_att
0, 0, buf, &n, arg);
if (!ret)
return n;
goto out_success;
if (ret != H_PARAMETER)
goto out;
@ -244,12 +244,14 @@ static ssize_t processor_bus_topology_show(struct device *dev, struct device_att
starting_index, 0, buf, &n, arg);
if (!ret)
return n;
goto out_success;
if (ret != H_PARAMETER)
goto out;
}
out_success:
put_cpu_var(hv_gpci_reqb);
return n;
out:
@ -278,7 +280,7 @@ static ssize_t processor_config_show(struct device *dev, struct device_attribute
0, 0, buf, &n, arg);
if (!ret)
return n;
goto out_success;
if (ret != H_PARAMETER)
goto out;
@ -312,12 +314,14 @@ static ssize_t processor_config_show(struct device *dev, struct device_attribute
starting_index, 0, buf, &n, arg);
if (!ret)
return n;
goto out_success;
if (ret != H_PARAMETER)
goto out;
}
out_success:
put_cpu_var(hv_gpci_reqb);
return n;
out:
@ -346,7 +350,7 @@ static ssize_t affinity_domain_via_virtual_processor_show(struct device *dev,
0, 0, buf, &n, arg);
if (!ret)
return n;
goto out_success;
if (ret != H_PARAMETER)
goto out;
@ -382,12 +386,14 @@ static ssize_t affinity_domain_via_virtual_processor_show(struct device *dev,
starting_index, secondary_index, buf, &n, arg);
if (!ret)
return n;
goto out_success;
if (ret != H_PARAMETER)
goto out;
}
out_success:
put_cpu_var(hv_gpci_reqb);
return n;
out:
@ -416,7 +422,7 @@ static ssize_t affinity_domain_via_domain_show(struct device *dev, struct device
0, 0, buf, &n, arg);
if (!ret)
return n;
goto out_success;
if (ret != H_PARAMETER)
goto out;
@ -448,12 +454,14 @@ static ssize_t affinity_domain_via_domain_show(struct device *dev, struct device
starting_index, 0, buf, &n, arg);
if (!ret)
return n;
goto out_success;
if (ret != H_PARAMETER)
goto out;
}
out_success:
put_cpu_var(hv_gpci_reqb);
return n;
out:

View File

@ -293,6 +293,8 @@ static int pika_dtm_thread(void __iomem *fpga)
schedule_timeout(HZ);
}
put_device(&client->dev);
return 0;
}

View File

@ -27,8 +27,8 @@
static void __init km82xx_pic_init(void)
{
struct device_node *np __free(device_node);
np = of_find_compatible_node(NULL, NULL, "fsl,pq2-pic");
struct device_node *np __free(device_node) = of_find_compatible_node(NULL,
NULL, "fsl,pq2-pic");
if (!np) {
pr_err("PIC init: can not find cpm-pic node\n");

View File

@ -1058,40 +1058,6 @@ int pmac_i2c_match_adapter(struct device_node *dev, struct i2c_adapter *adapter)
}
EXPORT_SYMBOL_GPL(pmac_i2c_match_adapter);
int pmac_low_i2c_lock(struct device_node *np)
{
struct pmac_i2c_bus *bus, *found = NULL;
list_for_each_entry(bus, &pmac_i2c_busses, link) {
if (np == bus->controller) {
found = bus;
break;
}
}
if (!found)
return -ENODEV;
return pmac_i2c_open(bus, 0);
}
EXPORT_SYMBOL_GPL(pmac_low_i2c_lock);
int pmac_low_i2c_unlock(struct device_node *np)
{
struct pmac_i2c_bus *bus, *found = NULL;
list_for_each_entry(bus, &pmac_i2c_busses, link) {
if (np == bus->controller) {
found = bus;
break;
}
}
if (!found)
return -ENODEV;
pmac_i2c_close(bus);
return 0;
}
EXPORT_SYMBOL_GPL(pmac_low_i2c_unlock);
int pmac_i2c_open(struct pmac_i2c_bus *bus, int polled)
{
int rc;

View File

@ -937,6 +937,28 @@ config RISCV_VECTOR_MISALIGNED
help
Enable detecting support for vector misaligned loads and stores.
config RISCV_SBI_FWFT_DELEGATE_MISALIGNED
bool "Request firmware delegation of unaligned access exceptions"
depends on RISCV_SBI
depends on NONPORTABLE
help
Use SBI FWFT to request delegation of load address misaligned and
store address misaligned exceptions, if possible, and prefer Linux
kernel emulation of these accesses to firmware emulation.
Unfortunately, Linux's emulation is still incomplete. Namely, it
currently does not handle vector instructions and KVM guest accesses.
On platforms where these accesses would have been handled by firmware,
enabling this causes unexpected kernel oopses, userspaces crashes and
KVM guest crashes. If you are sure that these are not a problem for
your platform, you can say Y here, which may improve performance.
Saying N here will not worsen emulation support for unaligned accesses
even in the case where the firmware also has incomplete support. It
simply keeps the firmware's emulation enabled.
If you don't know what to do here, say N.
choice
prompt "Unaligned Accesses Support"
default RISCV_PROBE_UNALIGNED_ACCESS

View File

@ -57,7 +57,7 @@ void mips_errata_patch_func(struct alt_entry *begin, struct alt_entry *end,
}
tmp = (1U << alt->patch_id);
if (cpu_req_errata && tmp) {
if (cpu_req_errata & tmp) {
mutex_lock(&text_mutex);
patch_text_nosync(ALT_OLD_PTR(alt), ALT_ALT_PTR(alt),
alt->alt_len);

View File

@ -107,6 +107,8 @@ static long compat_restore_sigcontext(struct pt_regs *regs,
/* sc_regs is structured the same as the start of pt_regs */
err = __copy_from_user(&cregs, &sc->sc_regs, sizeof(sc->sc_regs));
if (unlikely(err))
return err;
cregs_to_regs(&cregs, regs);

View File

@ -1,6 +1,7 @@
/* SPDX-License-Identifier: GPL-2.0 */
/* Copyright (C) 2023 Rivos Inc. */
#include <linux/cfi_types.h>
#include <linux/linkage.h>
#include <asm/asm.h>
@ -9,7 +10,7 @@
/* void __riscv_copy_words_unaligned(void *, const void *, size_t) */
/* Performs a memcpy without aligning buffers, using word loads and stores. */
/* Note: The size is truncated to a multiple of 8 * SZREG */
SYM_FUNC_START(__riscv_copy_words_unaligned)
SYM_TYPED_FUNC_START(__riscv_copy_words_unaligned)
andi a4, a2, ~((8*SZREG)-1)
beqz a4, 2f
add a3, a1, a4
@ -41,7 +42,7 @@ SYM_FUNC_END(__riscv_copy_words_unaligned)
/* void __riscv_copy_bytes_unaligned(void *, const void *, size_t) */
/* Performs a memcpy without aligning buffers, using only byte accesses. */
/* Note: The size is truncated to a multiple of 8 */
SYM_FUNC_START(__riscv_copy_bytes_unaligned)
SYM_TYPED_FUNC_START(__riscv_copy_bytes_unaligned)
andi a4, a2, ~(8-1)
beqz a4, 2f
add a3, a1, a4

View File

@ -896,10 +896,8 @@ static void __init riscv_fill_hwcap_from_isa_string(unsigned long *isa2hwcap)
* CPU cores with the ratified spec will contain non-zero
* marchid.
*/
if (acpi_disabled && boot_vendorid == THEAD_VENDOR_ID && boot_archid == 0x0) {
this_hwcap &= ~isa2hwcap[RISCV_ISA_EXT_v];
if (acpi_disabled && boot_vendorid == THEAD_VENDOR_ID && boot_archid == 0x0)
clear_bit(RISCV_ISA_EXT_v, source_isa);
}
riscv_resolve_isa(source_isa, isainfo->isa, &this_hwcap, isa2hwcap);
@ -1104,16 +1102,16 @@ early_param("riscv_isa_fallback", riscv_isa_fallback_setup);
void __init riscv_fill_hwcap(void)
{
char print_str[NUM_ALPHA_EXTS + 1];
unsigned long isa2hwcap[26] = {0};
unsigned long isa2hwcap[RISCV_ISA_EXT_BASE] = {0};
int i, j;
isa2hwcap['i' - 'a'] = COMPAT_HWCAP_ISA_I;
isa2hwcap['m' - 'a'] = COMPAT_HWCAP_ISA_M;
isa2hwcap['a' - 'a'] = COMPAT_HWCAP_ISA_A;
isa2hwcap['f' - 'a'] = COMPAT_HWCAP_ISA_F;
isa2hwcap['d' - 'a'] = COMPAT_HWCAP_ISA_D;
isa2hwcap['c' - 'a'] = COMPAT_HWCAP_ISA_C;
isa2hwcap['v' - 'a'] = COMPAT_HWCAP_ISA_V;
isa2hwcap[RISCV_ISA_EXT_i] = COMPAT_HWCAP_ISA_I;
isa2hwcap[RISCV_ISA_EXT_m] = COMPAT_HWCAP_ISA_M;
isa2hwcap[RISCV_ISA_EXT_a] = COMPAT_HWCAP_ISA_A;
isa2hwcap[RISCV_ISA_EXT_f] = COMPAT_HWCAP_ISA_F;
isa2hwcap[RISCV_ISA_EXT_d] = COMPAT_HWCAP_ISA_D;
isa2hwcap[RISCV_ISA_EXT_c] = COMPAT_HWCAP_ISA_C;
isa2hwcap[RISCV_ISA_EXT_v] = COMPAT_HWCAP_ISA_V;
if (!acpi_disabled) {
riscv_fill_hwcap_from_isa_string(isa2hwcap);

View File

@ -577,8 +577,8 @@ static int compat_riscv_gpr_set(struct task_struct *target,
struct compat_user_regs_struct cregs;
ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, &cregs, 0, -1);
cregs_to_regs(&cregs, task_pt_regs(target));
if (!ret)
cregs_to_regs(&cregs, task_pt_regs(target));
return ret;
}

View File

@ -584,7 +584,7 @@ static int cpu_online_check_unaligned_access_emulated(unsigned int cpu)
static bool misaligned_traps_delegated;
#ifdef CONFIG_RISCV_SBI
#if defined(CONFIG_RISCV_SBI_FWFT_DELEGATE_MISALIGNED)
static int cpu_online_sbi_unaligned_setup(unsigned int cpu)
{

View File

@ -109,15 +109,16 @@ void set_indir_lp_lock(struct task_struct *task, bool lock)
task->thread_info.user_cfi_state.ufcfi_locked = lock;
}
/*
* If size is 0, then to be compatible with regular stack we want it to be as big as
* regular stack. Else PAGE_ALIGN it and return back
* The shadow stack only stores the return address and not any variables
* this should be more than sufficient for most applications.
* Else PAGE_ALIGN it and return back
*/
static unsigned long calc_shstk_size(unsigned long size)
{
if (size)
return PAGE_ALIGN(size);
return PAGE_ALIGN(min_t(unsigned long long, rlimit(RLIMIT_STACK), SZ_4G));
return PAGE_ALIGN(min(rlimit(RLIMIT_STACK) / 2, SZ_2G));
}
/*

View File

@ -2,6 +2,7 @@
/* Copyright (C) 2024 Rivos Inc. */
#include <linux/args.h>
#include <linux/cfi_types.h>
#include <linux/linkage.h>
#include <asm/asm.h>
@ -16,7 +17,7 @@
/* void __riscv_copy_vec_words_unaligned(void *, const void *, size_t) */
/* Performs a memcpy without aligning buffers, using word loads and stores. */
/* Note: The size is truncated to a multiple of WORD_EEW */
SYM_FUNC_START(__riscv_copy_vec_words_unaligned)
SYM_TYPED_FUNC_START(__riscv_copy_vec_words_unaligned)
andi a4, a2, ~(WORD_EEW-1)
beqz a4, 2f
add a3, a1, a4
@ -38,7 +39,7 @@ SYM_FUNC_END(__riscv_copy_vec_words_unaligned)
/* void __riscv_copy_vec_bytes_unaligned(void *, const void *, size_t) */
/* Performs a memcpy without aligning buffers, using only byte accesses. */
/* Note: The size is truncated to a multiple of 8 */
SYM_FUNC_START(__riscv_copy_vec_bytes_unaligned)
SYM_TYPED_FUNC_START(__riscv_copy_vec_bytes_unaligned)
andi a4, a2, ~(8-1)
beqz a4, 2f
add a3, a1, a4

View File

@ -792,6 +792,27 @@ static void __init set_mmap_rnd_bits_max(void)
mmap_rnd_bits_max = MMAP_VA_BITS - PAGE_SHIFT - 3;
}
static bool __init is_vaddr_valid(unsigned long va)
{
unsigned long up = 0;
switch (satp_mode) {
case SATP_MODE_39:
up = 1UL << 38;
break;
case SATP_MODE_48:
up = 1UL << 47;
break;
case SATP_MODE_57:
up = 1UL << 56;
break;
default:
return false;
}
return (va < up) || (va >= (ULONG_MAX - up + 1));
}
/*
* There is a simple way to determine if 4-level is supported by the
* underlying hardware: establish 1:1 mapping in 4-level page table mode
@ -833,6 +854,9 @@ static __init void set_satp_mode(uintptr_t dtb_pa)
set_satp_mode_pmd + PMD_SIZE,
PMD_SIZE, PAGE_KERNEL_EXEC);
retry:
if (!is_vaddr_valid(set_satp_mode_pmd))
goto out;
create_pgd_mapping(early_pg_dir,
set_satp_mode_pmd,
pgtable_l5_enabled ?
@ -855,6 +879,7 @@ static __init void set_satp_mode(uintptr_t dtb_pa)
disable_pgtable_l4();
}
out:
memset(early_pg_dir, 0, PAGE_SIZE);
memset(early_p4d, 0, PAGE_SIZE);
memset(early_pud, 0, PAGE_SIZE);

View File

@ -3310,8 +3310,7 @@ static void aen_host_forward(unsigned long si)
struct zpci_gaite *gaite;
struct kvm *kvm;
gaite = (struct zpci_gaite *)aift->gait +
(si * sizeof(struct zpci_gaite));
gaite = aift->gait + si;
if (gaite->count == 0)
return;
if (gaite->aisb != 0)

View File

@ -166,7 +166,7 @@ static int kvm_zpci_set_airq(struct zpci_dev *zdev)
fib.fmt0.noi = airq_iv_end(zdev->aibv);
fib.fmt0.aibv = virt_to_phys(zdev->aibv->vector);
fib.fmt0.aibvo = 0;
fib.fmt0.aisb = virt_to_phys(aift->sbv->vector + (zdev->aisb / 64) * 8);
fib.fmt0.aisb = virt_to_phys(aift->sbv->vector) + (zdev->aisb / 64) * 8;
fib.fmt0.aisbo = zdev->aisb & 63;
fib.gd = zdev->gisa;
@ -290,8 +290,7 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib,
phys_to_virt(fib->fmt0.aibv));
spin_lock_irq(&aift->gait_lock);
gaite = (struct zpci_gaite *)aift->gait + (zdev->aisb *
sizeof(struct zpci_gaite));
gaite = aift->gait + zdev->aisb;
/* If assist not requested, host will get all alerts */
if (assist)
@ -309,7 +308,7 @@ static int kvm_s390_pci_aif_enable(struct zpci_dev *zdev, struct zpci_fib *fib,
/* Update guest FIB for re-issue */
fib->fmt0.aisbo = zdev->aisb & 63;
fib->fmt0.aisb = virt_to_phys(aift->sbv->vector + (zdev->aisb / 64) * 8);
fib->fmt0.aisb = virt_to_phys(aift->sbv->vector) + (zdev->aisb / 64) * 8;
fib->fmt0.isc = gisc;
/* Save some guest fib values in the host for later use */
@ -357,8 +356,7 @@ static int kvm_s390_pci_aif_disable(struct zpci_dev *zdev, bool force)
if (zdev->kzdev->fib.fmt0.aibv == 0)
goto out;
spin_lock_irq(&aift->gait_lock);
gaite = (struct zpci_gaite *)aift->gait + (zdev->aisb *
sizeof(struct zpci_gaite));
gaite = aift->gait + zdev->aisb;
isc = gaite->gisc;
gaite->count--;
if (gaite->count == 0) {

View File

@ -803,9 +803,10 @@
#define MSR_AMD64_LBR_SELECT 0xc000010e
/* Zen4 */
#define MSR_ZEN4_BP_CFG 0xc001102e
#define MSR_ZEN4_BP_CFG 0xc001102e
#define MSR_ZEN4_BP_CFG_BP_SPEC_REDUCE_BIT 4
#define MSR_ZEN4_BP_CFG_SHARED_BTB_FIX_BIT 5
#define MSR_ZEN2_BP_CFG_BUG_FIX_BIT 33
/* Fam 19h MSRs */
#define MSR_F19H_UMC_PERF_CTL 0xc0010800

View File

@ -88,19 +88,19 @@ static void amd_set_max_freq_ratio(void)
rc = cppc_get_perf_caps(0, &perf_caps);
if (rc) {
pr_warn("Could not retrieve perf counters (%d)\n", rc);
pr_debug("Could not retrieve perf counters (%d)\n", rc);
return;
}
rc = amd_get_boost_ratio_numerator(0, &numerator);
if (rc) {
pr_warn("Could not retrieve highest performance (%d)\n", rc);
pr_debug("Could not retrieve highest performance (%d)\n", rc);
return;
}
nominal_perf = perf_caps.nominal_perf;
if (!nominal_perf) {
pr_warn("Could not retrieve nominal performance\n");
pr_debug("Could not retrieve nominal performance\n");
return;
}

View File

@ -989,6 +989,9 @@ static void init_amd_zen2(struct cpuinfo_x86 *c)
/* Correct misconfigured CPUID on some clients. */
clear_cpu_cap(c, X86_FEATURE_INVLPGB);
if (!cpu_has(c, X86_FEATURE_HYPERVISOR))
msr_set_bit(MSR_ZEN4_BP_CFG, MSR_ZEN2_BP_CFG_BUG_FIX_BIT);
}
static void init_amd_zen3(struct cpuinfo_x86 *c)

View File

@ -90,7 +90,6 @@ struct mca_config mca_cfg __read_mostly = {
};
static DEFINE_PER_CPU(struct mce_hw_err, hw_errs_seen);
static unsigned long mce_need_notify;
/*
* MCA banks polled by the period polling timer for corrected events.
@ -152,8 +151,10 @@ EXPORT_PER_CPU_SYMBOL_GPL(injectm);
void mce_log(struct mce_hw_err *err)
{
if (mce_gen_pool_add(err))
if (mce_gen_pool_add(err)) {
pr_info(HW_ERR "Machine check events logged\n");
irq_work_queue(&mce_irq_work);
}
}
EXPORT_SYMBOL_GPL(mce_log);
@ -585,28 +586,6 @@ bool mce_is_correctable(struct mce *m)
}
EXPORT_SYMBOL_GPL(mce_is_correctable);
/*
* Notify the user(s) about new machine check events.
* Can be called from interrupt context, but not from machine check/NMI
* context.
*/
static bool mce_notify_irq(void)
{
/* Not more than two messages every minute */
static DEFINE_RATELIMIT_STATE(ratelimit, 60*HZ, 2);
if (test_and_clear_bit(0, &mce_need_notify)) {
mce_work_trigger();
if (__ratelimit(&ratelimit))
pr_info(HW_ERR "Machine check events logged\n");
return true;
}
return false;
}
static int mce_early_notifier(struct notifier_block *nb, unsigned long val,
void *data)
{
@ -618,9 +597,7 @@ static int mce_early_notifier(struct notifier_block *nb, unsigned long val,
/* Emit the trace record: */
trace_mce_record(err);
set_bit(0, &mce_need_notify);
mce_notify_irq();
mce_work_trigger();
return NOTIFY_DONE;
}
@ -1804,7 +1781,7 @@ static void mce_timer_fn(struct timer_list *t)
* Alert userspace if needed. If we logged an MCE, reduce the polling
* interval, otherwise increase the polling interval.
*/
if (mce_notify_irq())
if (!mce_gen_pool_empty())
iv = max(iv / 2, (unsigned long) HZ/100);
else
iv = min(iv * 2, round_jiffies_relative(check_interval * HZ));

View File

@ -136,6 +136,14 @@ SYM_CODE_START_LOCAL_NOALIGN(identity_mapped)
* %r13 original CR4 when relocate_kernel() was invoked
*/
/*
* Set return address to 0 if not preserving context. The purgatory
* shipped in kexec-tools will unconditionally look for the return
* address on the stack and set a kexec_jump_back_entry= command
* line option if it's non-zero. There's no other way that it can
* tell a preserve-context (kjump) kexec from a normal one.
*/
pushq $0
/* store the start address on the stack */
pushq %rdx

View File

@ -4481,7 +4481,7 @@ static const struct opcode opcode_map_0f_38[256] = {
X16(N), X16(N),
/* 0x20 - 0x2f */
X8(N),
X2(N), GP(SrcReg | DstMem | ModRM | Mov | Aligned, &pfx_0f_e7_0f_38_2a), N, N, N, N, N,
X2(N), GP(SrcMem | DstReg | ModRM | Mov | Aligned, &pfx_0f_e7_0f_38_2a), N, N, N, N, N,
/* 0x30 - 0x7f */
X16(N), X16(N), X16(N), X16(N), X16(N),
/* 0x80 - 0xef */

View File

@ -2526,6 +2526,23 @@ static void shadow_walk_next(struct kvm_shadow_walk_iterator *iterator)
__shadow_walk_next(iterator, *iterator->sptep);
}
/*
* Note: while normally KVM uses a "bool flush" return value to let
* the caller batch flushes, __link_shadow_page() flushes immediately
* before populating the parent PTE with the new shadow page. The
* typical callers, direct_map() and FNAME(fetch)(), are not going
* to zap more than one huge SPTE anyway.
*
* The only exception, where @flush can be false, is when a huge SPTE
* is replaced with a shadow page SPTE with a fully populated page table,
* which can happen from shadow_mmu_split_huge_page(). In this case,
* no memory is unmapped across the change to the page tables and no
* immediate flush is needed for correctness.
*
* Even in that case, calls to kvm_mmu_commit_zap_page() are not
* batched. Doing so would require adding an invalid_list argument
* all the way down to __walk_slot_rmaps().
*/
static void __link_shadow_page(struct kvm *kvm,
struct kvm_mmu_memory_cache *cache, u64 *sptep,
struct kvm_mmu_page *sp, bool flush)
@ -2541,8 +2558,10 @@ static void __link_shadow_page(struct kvm *kvm,
parent_sp = sptep_to_sp(sptep);
WARN_ON_ONCE(parent_sp->role.level == PG_LEVEL_4K);
mmu_page_zap_pte(kvm, parent_sp, sptep, &invalid_list);
kvm_mmu_remote_flush_or_zap(kvm, &invalid_list, true);
if (mmu_page_zap_pte(kvm, parent_sp, sptep, &invalid_list))
kvm_mmu_commit_zap_page(kvm, &invalid_list);
else if (flush)
kvm_flush_remote_tlbs_sptep(kvm, sptep);
}
spte = make_nonleaf_spte(sp->spt, sp_ad_disabled(sp));

View File

@ -160,6 +160,16 @@ void nested_vmcb02_recalc_intercepts(struct vcpu_svm *svm)
if (!intercept_smi)
vmcb_clr_intercept(&vmcb02->control, INTERCEPT_SMI);
/*
* Intercept PAUSE if and only if L1 wants to. KVM intercepts PAUSE so
* that a vCPU that may be spinning waiting for a lock can be scheduled
* out in favor of the vCPU that holds said lock. KVM doesn't support
* yielding across L2 vCPUs, as KVM has limited visilibity into which
* L2 vCPUs are in the same L2 VM, i.e. may be contending for locks.
*/
if (!vmcb12_is_intercept(&svm->nested.ctl, INTERCEPT_PAUSE))
vmcb_clr_intercept(&vmcb02->control, INTERCEPT_PAUSE);
if (nested_vmcb_needs_vls_intercept(svm)) {
/*
* If the virtual VMLOAD/VMSAVE is not enabled for the L2,
@ -819,7 +829,6 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm)
struct vmcb *vmcb02 = svm->nested.vmcb02.ptr;
struct vmcb *vmcb01 = svm->vmcb01.ptr;
struct kvm_vcpu *vcpu = &svm->vcpu;
u32 pause_count12, pause_thresh12;
nested_svm_transition_tlb_flush(vcpu);
@ -947,31 +956,13 @@ static void nested_vmcb02_prepare_control(struct vcpu_svm *svm)
vmcb02->control.misc_ctl2 |= SVM_MISC2_ENABLE_V_VMLOAD_VMSAVE;
if (guest_cpu_cap_has(vcpu, X86_FEATURE_PAUSEFILTER))
pause_count12 = vmcb12_ctrl->pause_filter_count;
vmcb02->control.pause_filter_count = vmcb12_ctrl->pause_filter_count;
else
pause_count12 = 0;
vmcb02->control.pause_filter_count = 0;
if (guest_cpu_cap_has(vcpu, X86_FEATURE_PFTHRESHOLD))
pause_thresh12 = vmcb12_ctrl->pause_filter_thresh;
vmcb02->control.pause_filter_thresh = vmcb12_ctrl->pause_filter_thresh;
else
pause_thresh12 = 0;
if (kvm_pause_in_guest(svm->vcpu.kvm)) {
/* use guest values since host doesn't intercept PAUSE */
vmcb02->control.pause_filter_count = pause_count12;
vmcb02->control.pause_filter_thresh = pause_thresh12;
} else {
/* start from host values otherwise */
vmcb02->control.pause_filter_count = vmcb01->control.pause_filter_count;
vmcb02->control.pause_filter_thresh = vmcb01->control.pause_filter_thresh;
/* ... but ensure filtering is disabled if so requested. */
if (vmcb12_is_intercept(vmcb12_ctrl, INTERCEPT_PAUSE)) {
if (!pause_count12)
vmcb02->control.pause_filter_count = 0;
if (!pause_thresh12)
vmcb02->control.pause_filter_thresh = 0;
}
}
vmcb02->control.pause_filter_thresh = 0;
/*
* Take ALLOW_LARGER_RAP from vmcb12 even though it should be safe to
@ -1298,12 +1289,6 @@ void nested_svm_vmexit(struct vcpu_svm *svm)
/* in case we halted in L2 */
kvm_set_mp_state(vcpu, KVM_MP_STATE_RUNNABLE);
if (!kvm_pause_in_guest(vcpu->kvm)) {
vmcb01->control.pause_filter_count = vmcb02->control.pause_filter_count;
vmcb_mark_dirty(vmcb01, VMCB_INTERCEPTS);
}
/*
* Invalidate last_bus_lock_rip unless KVM is still waiting for the
* guest to make forward progress before re-enabling bus lock detection.

View File

@ -913,7 +913,15 @@ static void grow_ple_window(struct kvm_vcpu *vcpu)
struct vmcb_control_area *control = &svm->vmcb->control;
int old = control->pause_filter_count;
if (kvm_pause_in_guest(vcpu->kvm))
/* Adjusting pause_filter_count makes no sense if PLE is disabled. */
WARN_ON_ONCE(kvm_pause_in_guest(vcpu->kvm));
/*
* While running L2, KVM should intercept PAUSE if and only if L1 wants
* to intercept PAUSE, and L1's intercept should take priority, i.e.
* KVM should never handle a PAUSE intercept from L2.
*/
if (WARN_ON_ONCE(is_guest_mode(vcpu)))
return;
control->pause_filter_count = __grow_ple_window(old,
@ -934,7 +942,10 @@ static void shrink_ple_window(struct kvm_vcpu *vcpu)
struct vmcb_control_area *control = &svm->vmcb->control;
int old = control->pause_filter_count;
if (kvm_pause_in_guest(vcpu->kvm))
/* Adjusting pause_filter_count makes no sense if PLE is disabled. */
WARN_ON_ONCE(kvm_pause_in_guest(vcpu->kvm));
if (is_guest_mode(vcpu))
return;
control->pause_filter_count =

View File

@ -154,7 +154,7 @@ TRACE_EVENT(kvm_xen_hypercall,
__entry->a2 = a2;
__entry->a3 = a3;
__entry->a4 = a4;
__entry->a4 = a5;
__entry->a5 = a5;
),
TP_printk("cpl %d nr 0x%lx a0 0x%lx a1 0x%lx a2 0x%lx a3 0x%lx a4 0x%lx a5 %lx",

View File

@ -14,6 +14,7 @@ extern bool __read_mostly flexpriority_enabled;
extern bool __read_mostly enable_ept;
extern bool __read_mostly enable_unrestricted_guest;
extern bool __read_mostly enable_ept_ad_bits;
extern bool __read_mostly enable_cet;
extern bool __read_mostly enable_pml;
extern int __read_mostly pt_mode;

View File

@ -108,6 +108,9 @@ module_param_named(unrestricted_guest,
bool __read_mostly enable_ept_ad_bits = 1;
module_param_named(eptad, enable_ept_ad_bits, bool, 0444);
bool __read_mostly enable_cet = 1;
module_param_named(cet, enable_cet, bool, 0444);
static bool __read_mostly emulate_invalid_guest_state = true;
module_param(emulate_invalid_guest_state, bool, 0444);
@ -4476,7 +4479,7 @@ void vmx_set_constant_host_state(struct vcpu_vmx *vmx)
* SSP is reloaded from IA32_PL3_SSP. Check SDM Vol.2A/B Chapter
* 3 and 4 for details.
*/
if (cpu_has_load_cet_ctrl()) {
if (enable_cet) {
vmcs_writel(HOST_S_CET, kvm_host.s_cet);
vmcs_writel(HOST_SSP, 0);
vmcs_writel(HOST_INTR_SSP_TABLE, 0);
@ -4532,6 +4535,10 @@ static u32 vmx_get_initial_vmentry_ctrl(void)
if (vmx_pt_mode_is_system())
vmentry_ctrl &= ~(VM_ENTRY_PT_CONCEAL_PIP |
VM_ENTRY_LOAD_IA32_RTIT_CTL);
if (!enable_cet)
vmentry_ctrl &= ~VM_ENTRY_LOAD_CET_STATE;
/*
* IA32e mode, and loading of EFER and PERF_GLOBAL_CTRL are toggled dynamically.
*/
@ -4546,6 +4553,9 @@ static u32 vmx_get_initial_vmexit_ctrl(void)
{
u32 vmexit_ctrl = vmcs_config.vmexit_ctrl;
if (!enable_cet)
vmexit_ctrl &= ~VM_EXIT_LOAD_CET_STATE;
/*
* Not used by KVM and never set in vmcs01 or vmcs02, but emulated for
* nested virtualization and thus allowed to be set in vmcs12.
@ -8155,7 +8165,7 @@ static __init void vmx_set_cpu_caps(void)
* VMX_BASIC[bit56] == 0, inject #CP at VMX entry with error code
* fails, so disable CET in this case too.
*/
if (!cpu_has_load_cet_ctrl() || !enable_unrestricted_guest ||
if (!enable_cet || !enable_unrestricted_guest ||
!cpu_has_vmx_basic_no_hw_errcode_cc()) {
kvm_cpu_cap_clear(X86_FEATURE_SHSTK);
kvm_cpu_cap_clear(X86_FEATURE_IBT);
@ -8630,6 +8640,9 @@ __init int vmx_hardware_setup(void)
!cpu_has_vmx_invept_global())
enable_ept = 0;
if (!cpu_has_load_cet_ctrl())
enable_cet = 0;
/* NX support is required for shadow paging. */
if (!enable_ept && !boot_cpu_has(X86_FEATURE_NX)) {
pr_err_ratelimited("NX (Execute Disable) not supported\n");

View File

@ -2145,7 +2145,10 @@ static void xen_set_fixmap(unsigned idx, phys_addr_t phys, pgprot_t prot)
static void xen_enter_lazy_mmu(void)
{
enter_lazy(XEN_LAZY_MMU);
preempt_disable();
if (xen_get_lazy_mode() != XEN_LAZY_MMU)
enter_lazy(XEN_LAZY_MMU);
preempt_enable();
}
static void xen_flush_lazy_mmu(void)
@ -2182,7 +2185,8 @@ static void xen_leave_lazy_mmu(void)
{
preempt_disable();
xen_mc_flush();
leave_lazy(XEN_LAZY_MMU);
if (xen_get_lazy_mode() != XEN_LAZY_NONE)
leave_lazy(XEN_LAZY_MMU);
preempt_enable();
}

View File

@ -655,7 +655,7 @@ static void __init xen_e820_swap_entry_with_ram(struct e820_entry *swap_entry)
/* Fill new entry (keep size and page offset). */
entry->type = swap_entry->type;
entry->addr = entry_end - swap_size +
swap_addr - swap_entry->addr;
swap_entry->addr - swap_addr;
entry->size = swap_entry->size;
/* Convert old entry to RAM, align to pages. */

View File

@ -308,7 +308,6 @@ static int bio_integrity_copy_user(struct bio *bio, struct bio_vec *bvec,
}
bip->bip_flags |= BIP_COPY_USER;
bip->bip_vcnt = nr_vecs;
return 0;
free_bip:
bio_integrity_free(bio);
@ -403,6 +402,24 @@ int bio_integrity_map_user(struct bio *bio, struct iov_iter *iter)
if (unlikely(ret < 0))
goto free_bvec;
/*
* Handle partial pinning. This can happen when pin_user_pages_fast()
* returns fewer pages than requested.
*/
if (user_backed_iter(iter) && unlikely(ret != bytes)) {
if (ret > 0) {
int npinned = DIV_ROUND_UP(offset + ret, PAGE_SIZE);
int i;
for (i = 0; i < npinned; i++)
unpin_user_page(pages[i]);
}
if (pages != stack_pages)
kvfree(pages);
ret = -EFAULT;
goto free_bvec;
}
nr_bvecs = bvec_from_pages(bvec, pages, nr_vecs, bytes, offset,
&is_p2p);
if (pages != stack_pages)

View File

@ -1279,11 +1279,12 @@ int bio_iov_iter_get_pages(struct bio *bio, struct iov_iter *iter,
return bio_iov_iter_align_down(bio, iter, len_align_mask);
}
static struct folio *folio_alloc_greedy(gfp_t gfp, size_t *size)
static struct folio *folio_alloc_greedy(gfp_t gfp, size_t *size,
size_t minsize)
{
struct folio *folio;
while (*size > PAGE_SIZE) {
while (*size > minsize) {
folio = folio_alloc(gfp | __GFP_NORETRY, get_order(*size));
if (folio)
return folio;
@ -1307,7 +1308,7 @@ static void bio_free_folios(struct bio *bio)
}
static int bio_iov_iter_bounce_write(struct bio *bio, struct iov_iter *iter,
size_t maxlen)
size_t maxlen, size_t minsize)
{
size_t total_len = min(maxlen, iov_iter_count(iter));
@ -1322,13 +1323,13 @@ static int bio_iov_iter_bounce_write(struct bio *bio, struct iov_iter *iter,
size_t this_len = min(total_len, SZ_1M);
struct folio *folio;
if (this_len > PAGE_SIZE * 2)
if (this_len > minsize * 2)
this_len = rounddown_pow_of_two(this_len);
if (bio->bi_iter.bi_size > BIO_MAX_SIZE - this_len)
break;
folio = folio_alloc_greedy(GFP_KERNEL, &this_len);
folio = folio_alloc_greedy(GFP_KERNEL, &this_len, minsize);
if (!folio)
break;
bio_add_folio_nofail(bio, folio, this_len, 0);
@ -1344,16 +1345,16 @@ static int bio_iov_iter_bounce_write(struct bio *bio, struct iov_iter *iter,
if (!bio->bi_iter.bi_size)
return -ENOMEM;
return 0;
return bio_iov_iter_align_down(bio, iter, minsize - 1);
}
static int bio_iov_iter_bounce_read(struct bio *bio, struct iov_iter *iter,
size_t maxlen)
size_t maxlen, size_t minsize)
{
size_t len = min3(iov_iter_count(iter), maxlen, SZ_1M);
struct folio *folio;
folio = folio_alloc_greedy(GFP_KERNEL, &len);
folio = folio_alloc_greedy(GFP_KERNEL, &len, minsize);
if (!folio)
return -ENOMEM;
@ -1382,7 +1383,7 @@ static int bio_iov_iter_bounce_read(struct bio *bio, struct iov_iter *iter,
bvec_set_folio(&bio->bi_io_vec[0], folio, bio->bi_iter.bi_size, 0);
if (iov_iter_extract_will_pin(iter))
bio_set_flag(bio, BIO_PAGE_PINNED);
return 0;
return bio_iov_iter_align_down(bio, iter, minsize - 1);
}
/**
@ -1390,6 +1391,7 @@ static int bio_iov_iter_bounce_read(struct bio *bio, struct iov_iter *iter,
* @bio: bio to send
* @iter: iter to read from / write into
* @maxlen: maximum size to bounce
* @minsize: minimum folio allocation size
*
* Helper for direct I/O implementations that need to bounce buffer because
* we need to checksum the data or perform other operations that require
@ -1397,11 +1399,12 @@ static int bio_iov_iter_bounce_read(struct bio *bio, struct iov_iter *iter,
* copies the data into it. Needs to be paired with bio_iov_iter_unbounce()
* called on completion.
*/
int bio_iov_iter_bounce(struct bio *bio, struct iov_iter *iter, size_t maxlen)
int bio_iov_iter_bounce(struct bio *bio, struct iov_iter *iter, size_t maxlen,
size_t minsize)
{
if (op_is_write(bio_op(bio)))
return bio_iov_iter_bounce_write(bio, iter, maxlen);
return bio_iov_iter_bounce_read(bio, iter, maxlen);
return bio_iov_iter_bounce_write(bio, iter, maxlen, minsize);
return bio_iov_iter_bounce_read(bio, iter, maxlen, minsize);
}
static void bvec_unpin(struct bio_vec *bv, bool mark_dirty)

View File

@ -3307,6 +3307,25 @@ blk_status_t blk_insert_cloned_request(struct request *rq)
return BLK_STS_IOERR;
}
/*
* Integrity segment counting depends on the same queue limits
* (virt_boundary_mask, seg_boundary_mask, max_segment_size) that
* vary across stacked queues, so recompute against the bottom
* queue just like nr_phys_segments above.
*/
if (blk_integrity_rq(rq) && rq->bio) {
unsigned short max_int_segs = queue_max_integrity_segments(q);
rq->nr_integrity_segments =
blk_rq_count_integrity_sg(rq->q, rq->bio);
if (rq->nr_integrity_segments > max_int_segs) {
printk(KERN_ERR "%s: over max integrity segments limit. (%u > %u)\n",
__func__, rq->nr_integrity_segments,
max_int_segs);
return BLK_STS_IOERR;
}
}
if (q->disk && should_fail_request(q->disk->part0, blk_rq_bytes(rq)))
return BLK_STS_IOERR;

View File

@ -623,6 +623,28 @@ static void disk_mark_zone_wplug_dead(struct blk_zone_wplug *zwplug)
}
}
static inline bool disk_check_zone_wplug_dead(struct blk_zone_wplug *zwplug)
{
if (!(zwplug->flags & BLK_ZONE_WPLUG_DEAD))
return false;
/*
* If a new write is received right after a zone reset completes and
* while the disk_zone_wplugs_worker() thread has not yet released the
* reference on the zone write plug after processing the last write to
* the zone, then the new write BIO will see the zone write plug marked
* as dead. This case is however a false positive and a perfectly valid
* pattern. In such case, restore the zone write plug to a live one.
*/
if (!zwplug->wp_offset && bio_list_empty(&zwplug->bio_list)) {
zwplug->flags &= ~BLK_ZONE_WPLUG_DEAD;
refcount_inc(&zwplug->ref);
return false;
}
return true;
}
static bool disk_zone_wplug_submit_bio(struct gendisk *disk,
struct blk_zone_wplug *zwplug);
@ -1444,12 +1466,12 @@ static bool blk_zone_wplug_handle_write(struct bio *bio, unsigned int nr_segs)
spin_lock_irqsave(&zwplug->lock, flags);
/*
* If we got a zone write plug marked as dead, then the user is issuing
* writes to a full zone, or without synchronizing with zone reset or
* zone finish operations. In such case, fail the BIO to signal this
* invalid usage.
* Check if we got a zone write plug marked as dead. If yes, then the
* user is likely issuing writes to a full zone, or without
* synchronizing with zone reset or zone finish operations. In such
* case, fail the BIO to signal this invalid usage.
*/
if (zwplug->flags & BLK_ZONE_WPLUG_DEAD) {
if (disk_check_zone_wplug_dead(zwplug)) {
spin_unlock_irqrestore(&zwplug->lock, flags);
disk_put_zone_wplug(zwplug);
bio_io_error(bio);

View File

@ -606,8 +606,11 @@ static const struct vm_operations_struct drm_vm_ops = {
static int qaic_gem_object_mmap(struct drm_gem_object *obj, struct vm_area_struct *vma)
{
struct qaic_bo *bo = to_qaic_bo(obj);
unsigned long remap_start;
unsigned long offset = 0;
unsigned long remap_end;
struct scatterlist *sg;
unsigned long length;
int ret = 0;
if (drm_gem_is_imported(obj))
@ -615,11 +618,27 @@ static int qaic_gem_object_mmap(struct drm_gem_object *obj, struct vm_area_struc
for (sg = bo->sgt->sgl; sg; sg = sg_next(sg)) {
if (sg_page(sg)) {
/* if sg is too large for the VMA, so truncate it to fit */
if (check_add_overflow(vma->vm_start, offset, &remap_start))
return -EINVAL;
if (check_add_overflow(remap_start, sg->length, &remap_end))
return -EINVAL;
if (remap_end > vma->vm_end) {
if (check_sub_overflow(vma->vm_end, remap_start, &length))
return -EINVAL;
} else {
length = sg->length;
}
if (length == 0)
goto out;
ret = remap_pfn_range(vma, vma->vm_start + offset, page_to_pfn(sg_page(sg)),
sg->length, vma->vm_page_prot);
length, vma->vm_page_prot);
if (ret)
goto out;
offset += sg->length;
offset += length;
}
}

View File

@ -145,6 +145,8 @@ int rocket_ioctl_prep_bo(struct drm_device *dev, void *data, struct drm_file *fi
ret = dma_resv_wait_timeout(gem_obj->resv, DMA_RESV_USAGE_WRITE, true, timeout);
if (!ret)
ret = timeout ? -ETIMEDOUT : -EBUSY;
else if (ret > 0)
ret = 0;
shmem_obj = &to_rocket_bo(gem_obj)->base;

View File

@ -192,11 +192,15 @@ static const struct dmi_system_id ac_dmi_table[] __initconst = {
static int acpi_ac_probe(struct platform_device *pdev)
{
struct acpi_device *adev = ACPI_COMPANION(&pdev->dev);
struct power_supply_config psy_cfg = {};
struct acpi_device *adev;
struct acpi_ac *ac;
int result;
adev = ACPI_COMPANION(&pdev->dev);
if (!adev)
return -ENODEV;
ac = kzalloc_obj(struct acpi_ac);
if (!ac)
return -ENOMEM;

View File

@ -423,7 +423,11 @@ static void acpi_pad_notify(acpi_handle handle, u32 event, void *data)
static int acpi_pad_probe(struct platform_device *pdev)
{
struct acpi_device *adev = ACPI_COMPANION(&pdev->dev);
struct acpi_device *adev;
adev = ACPI_COMPANION(&pdev->dev);
if (!adev)
return -ENODEV;
return acpi_dev_install_notify_handler(adev, ACPI_DEVICE_NOTIFY,
acpi_pad_notify, adev);

View File

@ -815,12 +815,16 @@ static void acpi_tad_remove(void *data)
static int acpi_tad_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
acpi_handle handle = ACPI_HANDLE(dev);
struct acpi_tad_driver_data *dd;
acpi_handle handle;
acpi_status status;
unsigned long long caps;
int ret;
handle = ACPI_HANDLE(dev);
if (!handle)
return -ENODEV;
/*
* Initialization failure messages are mostly about firmware issues, so
* print them at the "info" level.

View File

@ -1214,10 +1214,14 @@ static void sysfs_battery_cleanup(struct acpi_battery *battery)
static int acpi_battery_probe(struct platform_device *pdev)
{
struct acpi_device *device = ACPI_COMPANION(&pdev->dev);
struct acpi_battery *battery;
struct acpi_device *device;
int result;
device = ACPI_COMPANION(&pdev->dev);
if (!device)
return -ENODEV;
if (device->dep_unmet)
return -EPROBE_DEFER;

View File

@ -531,15 +531,20 @@ static int acpi_lid_input_open(struct input_dev *input)
static int acpi_button_probe(struct platform_device *pdev)
{
struct acpi_device *device = ACPI_COMPANION(&pdev->dev);
acpi_notify_handler handler;
struct acpi_device *device;
struct acpi_button *button;
struct input_dev *input;
const char *hid = acpi_device_hid(device);
acpi_status status;
char *name, *class;
const char *hid;
int error = 0;
device = ACPI_COMPANION(&pdev->dev);
if (!device)
return -ENODEV;
hid = acpi_device_hid(device);
if (!strcmp(hid, ACPI_BUTTON_HID_LID) &&
lid_init_state == ACPI_BUTTON_LID_INIT_DISABLED)
return -ENODEV;

View File

@ -1676,10 +1676,14 @@ static int acpi_ec_setup(struct acpi_ec *ec, struct acpi_device *device, bool ca
static int acpi_ec_probe(struct platform_device *pdev)
{
struct acpi_device *device = ACPI_COMPANION(&pdev->dev);
struct acpi_device *device;
struct acpi_ec *ec;
int ret;
device = ACPI_COMPANION(&pdev->dev);
if (!device)
return -ENODEV;
if (boot_ec && (boot_ec->handle == device->handle ||
!strcmp(acpi_device_hid(device), ACPI_ECDT_HID))) {
/* Fast path: this device corresponds to the boot EC. */

View File

@ -50,9 +50,13 @@ static void acpi_hed_notify(acpi_handle handle, u32 event, void *data)
static int acpi_hed_probe(struct platform_device *pdev)
{
struct acpi_device *device = ACPI_COMPANION(&pdev->dev);
struct acpi_device *device;
int err;
device = ACPI_COMPANION(&pdev->dev);
if (!device)
return -ENODEV;
/* Only one hardware error device */
if (hed_handle)
return -EINVAL;

View File

@ -3341,12 +3341,16 @@ static int acpi_nfit_probe(struct platform_device *pdev)
struct acpi_buffer buf = { ACPI_ALLOCATE_BUFFER, NULL };
struct acpi_nfit_desc *acpi_desc;
struct device *dev = &pdev->dev;
struct acpi_device *adev = ACPI_COMPANION(dev);
struct acpi_table_header *tbl;
struct acpi_device *adev;
acpi_status status = AE_OK;
acpi_size sz;
int rc = 0;
adev = ACPI_COMPANION(&pdev->dev);
if (!adev)
return -ENODEV;
rc = acpi_dev_install_notify_handler(adev, ACPI_DEVICE_NOTIFY,
acpi_nfit_notify, dev);
if (rc)

View File

@ -360,10 +360,14 @@ static void pfrt_log_put_idx(void *data)
static int acpi_pfrt_log_probe(struct platform_device *pdev)
{
acpi_handle handle = ACPI_HANDLE(&pdev->dev);
struct pfrt_log_device *pfrt_log_dev;
acpi_handle handle;
int ret;
handle = ACPI_HANDLE(&pdev->dev);
if (!handle)
return -ENODEV;
if (!acpi_has_method(handle, "_DSM")) {
dev_dbg(&pdev->dev, "Missing _DSM\n");
return -ENODEV;

View File

@ -538,10 +538,14 @@ static void pfru_put_idx(void *data)
static int acpi_pfru_probe(struct platform_device *pdev)
{
acpi_handle handle = ACPI_HANDLE(&pdev->dev);
struct pfru_device *pfru_dev;
acpi_handle handle;
int ret;
handle = ACPI_HANDLE(&pdev->dev);
if (!handle)
return -ENODEV;
if (!acpi_has_method(handle, "_DSM")) {
dev_dbg(&pdev->dev, "Missing _DSM\n");
return -ENODEV;

View File

@ -629,11 +629,15 @@ static void acpi_sbs_callback(void *context)
static int acpi_sbs_probe(struct platform_device *pdev)
{
struct acpi_device *device = ACPI_COMPANION(&pdev->dev);
struct acpi_device *device;
struct acpi_sbs *sbs;
int result = 0;
int id;
device = ACPI_COMPANION(&pdev->dev);
if (!device)
return -ENODEV;
sbs = kzalloc_obj(struct acpi_sbs);
if (!sbs) {
result = -ENOMEM;

View File

@ -237,11 +237,15 @@ static int smbus_alarm(void *context)
static int acpi_smbus_hc_probe(struct platform_device *pdev)
{
struct acpi_device *device = ACPI_COMPANION(&pdev->dev);
struct acpi_device *device;
int status;
unsigned long long val;
struct acpi_smb_hc *hc;
device = ACPI_COMPANION(&pdev->dev);
if (!device)
return -ENODEV;
status = acpi_evaluate_integer(device->handle, "_EC", NULL, &val);
if (ACPI_FAILURE(status)) {
pr_err("error obtaining _EC.\n");

View File

@ -789,7 +789,7 @@ static int acpi_thermal_probe(struct platform_device *pdev)
int i;
if (!device)
return -EINVAL;
return -ENODEV;
tz = kzalloc_obj(struct acpi_thermal);
if (!tz)

View File

@ -38,9 +38,13 @@ static u32 acpi_tiny_power_button_event(void *not_used)
static int acpi_tiny_power_button_probe(struct platform_device *pdev)
{
struct acpi_device *device = ACPI_COMPANION(&pdev->dev);
struct acpi_device *device;
acpi_status status;
device = ACPI_COMPANION(&pdev->dev);
if (!device)
return -ENODEV;
if (device->device_type == ACPI_BUS_TYPE_POWER_BUTTON) {
status = acpi_install_fixed_event_handler(ACPI_EVENT_POWER_BUTTON,
acpi_tiny_power_button_event,

View File

@ -920,6 +920,9 @@ static int ublk_validate_params(const struct ublk_device *ub)
if (p->max_sectors > (ub->dev_info.max_io_buf_bytes >> 9))
return -EINVAL;
if (p->max_sectors < PAGE_SECTORS)
return -EINVAL;
if (ublk_dev_is_zoned(ub) && !p->chunk_sectors)
return -EINVAL;
} else

View File

@ -552,8 +552,9 @@ static ssize_t amdgpu_debugfs_ring_read(struct file *f, char __user *buf,
size_t size, loff_t *pos)
{
struct amdgpu_ring *ring = file_inode(f)->i_private;
uint32_t value, result, early[3];
u32 value, result, early[3] = { 0 };
uint64_t p;
u32 avail_dw, start_dw, read_dw;
loff_t i;
int r;
@ -565,10 +566,10 @@ static ssize_t amdgpu_debugfs_ring_read(struct file *f, char __user *buf,
result = 0;
if (*pos < 12) {
if (ring->funcs->type == AMDGPU_RING_TYPE_CPER)
mutex_lock(&ring->adev->cper.ring_lock);
if (ring->funcs->type == AMDGPU_RING_TYPE_CPER)
mutex_lock(&ring->adev->cper.ring_lock);
if (*pos < 12) {
early[0] = amdgpu_ring_get_rptr(ring) & ring->buf_mask;
early[1] = amdgpu_ring_get_wptr(ring) & ring->buf_mask;
early[2] = ring->wptr & ring->buf_mask;
@ -600,13 +601,24 @@ static ssize_t amdgpu_debugfs_ring_read(struct file *f, char __user *buf,
*pos += 4;
}
} else {
early[0] = amdgpu_ring_get_rptr(ring) & ring->buf_mask;
early[1] = amdgpu_ring_get_wptr(ring) & ring->buf_mask;
p = early[0];
if (early[0] <= early[1])
size = (early[1] - early[0]);
avail_dw = early[1] - early[0];
else
size = ring->ring_size - (early[0] - early[1]);
avail_dw = ring->buf_mask + 1 - (early[0] - early[1]);
while (size) {
start_dw = (*pos > 12) ? ((*pos - 12) >> 2) : 0;
if (start_dw >= avail_dw)
goto out;
p = (p + start_dw) & ring->ptr_mask;
avail_dw -= start_dw;
read_dw = min_t(u32, avail_dw, size >> 2);
while (read_dw) {
if (p == early[1])
goto out;
@ -619,9 +631,10 @@ static ssize_t amdgpu_debugfs_ring_read(struct file *f, char __user *buf,
buf += 4;
result += 4;
size--;
read_dw--;
p++;
p &= ring->ptr_mask;
*pos += 4;
}
}

View File

@ -106,9 +106,6 @@ amdgpu_userq_detect_and_reset_queues(struct amdgpu_userq_mgr *uq_mgr)
int r = 0;
int i;
/* Warning if current process mutex is not held */
WARN_ON(!mutex_is_locked(&uq_mgr->userq_mutex));
if (unlikely(adev->debug_disable_gpu_ring_reset)) {
dev_err(adev->dev, "userq reset disabled by debug mask\n");
return 0;
@ -127,9 +124,11 @@ amdgpu_userq_detect_and_reset_queues(struct amdgpu_userq_mgr *uq_mgr)
*/
for (i = 0; i < num_queue_types; i++) {
int ring_type = queue_types[i];
const struct amdgpu_userq_funcs *funcs = adev->userq_funcs[ring_type];
const struct amdgpu_userq_funcs *funcs =
adev->userq_funcs[ring_type];
if (!amdgpu_userq_is_reset_type_supported(adev, ring_type, AMDGPU_RESET_TYPE_PER_QUEUE))
if (!amdgpu_userq_is_reset_type_supported(adev, ring_type,
AMDGPU_RESET_TYPE_PER_QUEUE))
continue;
if (atomic_read(&uq_mgr->userq_count[ring_type]) > 0 &&
@ -150,38 +149,22 @@ amdgpu_userq_detect_and_reset_queues(struct amdgpu_userq_mgr *uq_mgr)
static void amdgpu_userq_hang_detect_work(struct work_struct *work)
{
struct amdgpu_usermode_queue *queue = container_of(work,
struct amdgpu_usermode_queue,
hang_detect_work.work);
struct dma_fence *fence;
struct amdgpu_userq_mgr *uq_mgr;
struct amdgpu_usermode_queue *queue =
container_of(work, struct amdgpu_usermode_queue,
hang_detect_work.work);
if (!queue->userq_mgr)
return;
uq_mgr = queue->userq_mgr;
fence = READ_ONCE(queue->hang_detect_fence);
/* Fence already signaled no action needed */
if (!fence || dma_fence_is_signaled(fence))
return;
mutex_lock(&uq_mgr->userq_mutex);
amdgpu_userq_detect_and_reset_queues(uq_mgr);
mutex_unlock(&uq_mgr->userq_mutex);
amdgpu_userq_detect_and_reset_queues(queue->userq_mgr);
}
/*
* Start hang detection for a user queue fence. A delayed work will be scheduled
* to check if the fence is still pending after the timeout period.
*/
* to reset the queues when the fence doesn't signal in time.
*/
void amdgpu_userq_start_hang_detect_work(struct amdgpu_usermode_queue *queue)
{
struct amdgpu_device *adev;
unsigned long timeout_ms;
if (!queue || !queue->userq_mgr || !queue->userq_mgr->adev)
return;
adev = queue->userq_mgr->adev;
/* Determine timeout based on queue type */
switch (queue->queue_type) {
@ -199,8 +182,6 @@ void amdgpu_userq_start_hang_detect_work(struct amdgpu_usermode_queue *queue)
break;
}
/* Store the fence to monitor and schedule hang detection */
WRITE_ONCE(queue->hang_detect_fence, queue->last_fence);
schedule_delayed_work(&queue->hang_detect_work,
msecs_to_jiffies(timeout_ms));
}
@ -210,18 +191,24 @@ void amdgpu_userq_process_fence_irq(struct amdgpu_device *adev, u32 doorbell)
struct xarray *xa = &adev->userq_doorbell_xa;
struct amdgpu_usermode_queue *queue;
unsigned long flags;
int r;
xa_lock_irqsave(xa, flags);
queue = xa_load(xa, doorbell);
if (queue)
amdgpu_userq_fence_driver_process(queue->fence_drv);
xa_unlock_irqrestore(xa, flags);
}
if (queue) {
r = amdgpu_userq_fence_driver_process(queue->fence_drv);
/*
* We are in interrupt context here, this *can't* wait for
* reset work to finish.
*/
if (r >= 0)
cancel_delayed_work(&queue->hang_detect_work);
static void amdgpu_userq_init_hang_detect_work(struct amdgpu_usermode_queue *queue)
{
INIT_DELAYED_WORK(&queue->hang_detect_work, amdgpu_userq_hang_detect_work);
queue->hang_detect_fence = NULL;
/* Restart the timer when there are still fences pending */
if (r == 1)
amdgpu_userq_start_hang_detect_work(queue);
}
xa_unlock_irqrestore(xa, flags);
}
static int amdgpu_userq_buffer_va_list_add(struct amdgpu_usermode_queue *queue,
@ -345,23 +332,18 @@ static int amdgpu_userq_preempt_helper(struct amdgpu_usermode_queue *queue)
struct amdgpu_device *adev = uq_mgr->adev;
const struct amdgpu_userq_funcs *userq_funcs =
adev->userq_funcs[queue->queue_type];
bool found_hung_queue = false;
int r = 0;
int r;
if (queue->state == AMDGPU_USERQ_STATE_MAPPED) {
r = userq_funcs->preempt(queue);
if (r) {
queue->state = AMDGPU_USERQ_STATE_HUNG;
found_hung_queue = true;
return r;
} else {
queue->state = AMDGPU_USERQ_STATE_PREEMPTED;
}
}
if (found_hung_queue)
amdgpu_userq_detect_and_reset_queues(uq_mgr);
return r;
return 0;
}
static int amdgpu_userq_restore_helper(struct amdgpu_usermode_queue *queue)
@ -390,24 +372,21 @@ static int amdgpu_userq_unmap_helper(struct amdgpu_usermode_queue *queue)
struct amdgpu_device *adev = uq_mgr->adev;
const struct amdgpu_userq_funcs *userq_funcs =
adev->userq_funcs[queue->queue_type];
bool found_hung_queue = false;
int r = 0;
int r;
if ((queue->state == AMDGPU_USERQ_STATE_MAPPED) ||
(queue->state == AMDGPU_USERQ_STATE_PREEMPTED)) {
(queue->state == AMDGPU_USERQ_STATE_PREEMPTED)) {
r = userq_funcs->unmap(queue);
if (r) {
queue->state = AMDGPU_USERQ_STATE_HUNG;
found_hung_queue = true;
return r;
} else {
queue->state = AMDGPU_USERQ_STATE_UNMAPPED;
}
}
if (found_hung_queue)
amdgpu_userq_detect_and_reset_queues(uq_mgr);
return r;
return 0;
}
static int amdgpu_userq_map_helper(struct amdgpu_usermode_queue *queue)
@ -416,19 +395,19 @@ static int amdgpu_userq_map_helper(struct amdgpu_usermode_queue *queue)
struct amdgpu_device *adev = uq_mgr->adev;
const struct amdgpu_userq_funcs *userq_funcs =
adev->userq_funcs[queue->queue_type];
int r = 0;
int r;
if (queue->state == AMDGPU_USERQ_STATE_UNMAPPED) {
r = userq_funcs->map(queue);
if (r) {
queue->state = AMDGPU_USERQ_STATE_HUNG;
amdgpu_userq_detect_and_reset_queues(uq_mgr);
return r;
} else {
queue->state = AMDGPU_USERQ_STATE_MAPPED;
}
}
return r;
return 0;
}
static void amdgpu_userq_wait_for_last_fence(struct amdgpu_usermode_queue *queue)
@ -648,13 +627,11 @@ amdgpu_userq_destroy(struct amdgpu_userq_mgr *uq_mgr, struct amdgpu_usermode_que
amdgpu_bo_unreserve(vm->root.bo);
mutex_lock(&uq_mgr->userq_mutex);
queue->hang_detect_fence = NULL;
amdgpu_userq_wait_for_last_fence(queue);
#if defined(CONFIG_DEBUG_FS)
debugfs_remove_recursive(queue->debugfs_queue);
#endif
amdgpu_userq_detect_and_reset_queues(uq_mgr);
r = amdgpu_userq_unmap_helper(queue);
atomic_dec(&uq_mgr->userq_count[queue->queue_type]);
amdgpu_userq_cleanup(queue);
@ -800,6 +777,7 @@ amdgpu_userq_create(struct drm_file *filp, union drm_amdgpu_userq *args)
}
queue->doorbell_index = index;
mutex_init(&queue->fence_drv_lock);
xa_init_flags(&queue->fence_drv_xa, XA_FLAGS_ALLOC);
r = amdgpu_userq_fence_driver_alloc(adev, &queue->fence_drv);
if (r) {
@ -855,7 +833,8 @@ amdgpu_userq_create(struct drm_file *filp, union drm_amdgpu_userq *args)
up_read(&adev->reset_domain->sem);
amdgpu_debugfs_userq_init(filp, queue, qid);
amdgpu_userq_init_hang_detect_work(queue);
INIT_DELAYED_WORK(&queue->hang_detect_work,
amdgpu_userq_hang_detect_work);
args->out.queue_id = qid;
atomic_inc(&uq_mgr->userq_count[queue->queue_type]);
@ -873,6 +852,7 @@ amdgpu_userq_create(struct drm_file *filp, union drm_amdgpu_userq *args)
amdgpu_bo_reserve(fpriv->vm.root.bo, true);
amdgpu_userq_buffer_vas_list_cleanup(adev, queue);
amdgpu_bo_unreserve(fpriv->vm.root.bo);
mutex_destroy(&queue->fence_drv_lock);
free_queue:
kfree(queue);
err_pm_runtime:
@ -1262,7 +1242,6 @@ amdgpu_userq_evict_all(struct amdgpu_userq_mgr *uq_mgr)
unsigned long queue_id;
int ret = 0, r;
amdgpu_userq_detect_and_reset_queues(uq_mgr);
/* Try to unmap all the queues in this process ctx */
xa_for_each(&uq_mgr->userq_xa, queue_id, queue) {
r = amdgpu_userq_preempt_helper(queue);
@ -1270,9 +1249,11 @@ amdgpu_userq_evict_all(struct amdgpu_userq_mgr *uq_mgr)
ret = r;
}
if (ret)
if (ret) {
drm_file_err(uq_mgr->file,
"Couldn't unmap all the queues, eviction failed ret=%d\n", ret);
amdgpu_userq_detect_and_reset_queues(uq_mgr);
}
return ret;
}
@ -1372,7 +1353,6 @@ int amdgpu_userq_suspend(struct amdgpu_device *adev)
uqm = queue->userq_mgr;
cancel_delayed_work_sync(&uqm->resume_work);
guard(mutex)(&uqm->userq_mutex);
amdgpu_userq_detect_and_reset_queues(uqm);
if (adev->in_s0ix)
r = amdgpu_userq_preempt_helper(queue);
else
@ -1431,7 +1411,6 @@ int amdgpu_userq_stop_sched_for_enforce_isolation(struct amdgpu_device *adev,
if (((queue->queue_type == AMDGPU_HW_IP_GFX) ||
(queue->queue_type == AMDGPU_HW_IP_COMPUTE)) &&
(queue->xcp_id == idx)) {
amdgpu_userq_detect_and_reset_queues(uqm);
r = amdgpu_userq_preempt_helper(queue);
if (r)
ret = r;
@ -1504,23 +1483,21 @@ void amdgpu_userq_pre_reset(struct amdgpu_device *adev)
{
const struct amdgpu_userq_funcs *userq_funcs;
struct amdgpu_usermode_queue *queue;
struct amdgpu_userq_mgr *uqm;
unsigned long queue_id;
/* TODO: We probably need a new lock for the queue state */
xa_for_each(&adev->userq_doorbell_xa, queue_id, queue) {
uqm = queue->userq_mgr;
cancel_delayed_work_sync(&uqm->resume_work);
if (queue->state == AMDGPU_USERQ_STATE_MAPPED) {
amdgpu_userq_wait_for_last_fence(queue);
userq_funcs = adev->userq_funcs[queue->queue_type];
userq_funcs->unmap(queue);
/* just mark all queues as hung at this point.
* if unmap succeeds, we could map again
* in amdgpu_userq_post_reset() if vram is not lost
*/
queue->state = AMDGPU_USERQ_STATE_HUNG;
amdgpu_userq_fence_driver_force_completion(queue);
}
if (queue->state != AMDGPU_USERQ_STATE_MAPPED)
continue;
userq_funcs = adev->userq_funcs[queue->queue_type];
userq_funcs->unmap(queue);
/* just mark all queues as hung at this point.
* if unmap succeeds, we could map again
* in amdgpu_userq_post_reset() if vram is not lost
*/
queue->state = AMDGPU_USERQ_STATE_HUNG;
amdgpu_userq_fence_driver_force_completion(queue);
}
}

View File

@ -66,6 +66,18 @@ struct amdgpu_usermode_queue {
struct amdgpu_userq_obj db_obj;
struct amdgpu_userq_obj fw_obj;
struct amdgpu_userq_obj wptr_obj;
/**
* @fence_drv_lock: Protecting @fence_drv_xa.
*/
struct mutex fence_drv_lock;
/**
* @fence_drv_xa:
*
* References to the external fence drivers returned by wait_ioctl.
* Dropped on the next signaled dma_fence or queue destruction.
*/
struct xarray fence_drv_xa;
struct amdgpu_userq_fence_driver *fence_drv;
struct dma_fence *last_fence;
@ -73,7 +85,6 @@ struct amdgpu_usermode_queue {
int priority;
struct dentry *debugfs_queue;
struct delayed_work hang_detect_work;
struct dma_fence *hang_detect_fence;
struct kref refcount;
struct list_head userq_va_list;

View File

@ -121,6 +121,7 @@ amdgpu_userq_fence_driver_free(struct amdgpu_usermode_queue *userq)
userq->last_fence = NULL;
amdgpu_userq_walk_and_drop_fence_drv(&userq->fence_drv_xa);
xa_destroy(&userq->fence_drv_xa);
mutex_destroy(&userq->fence_drv_lock);
/* Drop the queue's ownership reference to fence_drv explicitly */
amdgpu_userq_fence_driver_put(userq->fence_drv);
}
@ -134,7 +135,14 @@ amdgpu_userq_fence_put_fence_drv_array(struct amdgpu_userq_fence *userq_fence)
userq_fence->fence_drv_array_count = 0;
}
void amdgpu_userq_fence_driver_process(struct amdgpu_userq_fence_driver *fence_drv)
/*
* Returns:
* -ENOENT when no fences were processes
* 1 when more fences are pending
* 0 when no fences are pending any more
*/
int
amdgpu_userq_fence_driver_process(struct amdgpu_userq_fence_driver *fence_drv)
{
struct amdgpu_userq_fence *userq_fence, *tmp;
LIST_HEAD(to_be_signaled);
@ -142,9 +150,6 @@ void amdgpu_userq_fence_driver_process(struct amdgpu_userq_fence_driver *fence_d
unsigned long flags;
u64 rptr;
if (!fence_drv)
return;
spin_lock_irqsave(&fence_drv->fence_list_lock, flags);
rptr = amdgpu_userq_fence_read(fence_drv);
@ -157,6 +162,9 @@ void amdgpu_userq_fence_driver_process(struct amdgpu_userq_fence_driver *fence_d
&userq_fence->link);
spin_unlock_irqrestore(&fence_drv->fence_list_lock, flags);
if (list_empty(&to_be_signaled))
return -ENOENT;
list_for_each_entry_safe(userq_fence, tmp, &to_be_signaled, link) {
fence = &userq_fence->base;
list_del_init(&userq_fence->link);
@ -168,6 +176,8 @@ void amdgpu_userq_fence_driver_process(struct amdgpu_userq_fence_driver *fence_d
dma_fence_put(fence);
}
/* That doesn't need to be accurate so no locking */
return list_empty(&fence_drv->fences) ? 0 : 1;
}
void amdgpu_userq_fence_driver_destroy(struct kref *ref)
@ -209,80 +219,84 @@ void amdgpu_userq_fence_driver_put(struct amdgpu_userq_fence_driver *fence_drv)
kref_put(&fence_drv->refcount, amdgpu_userq_fence_driver_destroy);
}
static int amdgpu_userq_fence_alloc(struct amdgpu_userq_fence **userq_fence)
static int amdgpu_userq_fence_alloc(struct amdgpu_usermode_queue *userq,
struct amdgpu_userq_fence **pfence)
{
*userq_fence = kmalloc(sizeof(**userq_fence), GFP_KERNEL);
return *userq_fence ? 0 : -ENOMEM;
struct amdgpu_userq_fence_driver *fence_drv = userq->fence_drv;
struct amdgpu_userq_fence *userq_fence;
void *entry;
userq_fence = kmalloc(sizeof(*userq_fence), GFP_KERNEL);
if (!userq_fence)
return -ENOMEM;
/*
* Get the next unused entry, since we fill from the start this can be
* used as size to allocate the array.
*/
mutex_lock(&userq->fence_drv_lock);
XA_STATE(xas, &userq->fence_drv_xa, 0);
rcu_read_lock();
do {
entry = xas_find_marked(&xas, ULONG_MAX, XA_FREE_MARK);
} while (xas_retry(&xas, entry));
rcu_read_unlock();
userq_fence->fence_drv_array = kvmalloc_array(xas.xa_index,
sizeof(fence_drv),
GFP_KERNEL);
if (!userq_fence->fence_drv_array) {
mutex_unlock(&userq->fence_drv_lock);
kfree(userq_fence);
return -ENOMEM;
}
userq_fence->fence_drv_array_count = xas.xa_index;
xa_extract(&userq->fence_drv_xa, (void **)userq_fence->fence_drv_array,
0, ULONG_MAX, xas.xa_index, XA_PRESENT);
xa_destroy(&userq->fence_drv_xa);
mutex_unlock(&userq->fence_drv_lock);
amdgpu_userq_fence_driver_get(fence_drv);
userq_fence->fence_drv = fence_drv;
*pfence = userq_fence;
return 0;
}
static int amdgpu_userq_fence_create(struct amdgpu_usermode_queue *userq,
struct amdgpu_userq_fence *userq_fence,
u64 seq, struct dma_fence **f)
static void amdgpu_userq_fence_init(struct amdgpu_usermode_queue *userq,
struct amdgpu_userq_fence *fence,
u64 seq)
{
struct amdgpu_userq_fence_driver *fence_drv;
struct dma_fence *fence;
struct amdgpu_userq_fence_driver *fence_drv = userq->fence_drv;
unsigned long flags;
bool signaled = false;
fence_drv = userq->fence_drv;
if (!fence_drv)
return -EINVAL;
spin_lock_init(&userq_fence->lock);
INIT_LIST_HEAD(&userq_fence->link);
fence = &userq_fence->base;
userq_fence->fence_drv = fence_drv;
dma_fence_init64(fence, &amdgpu_userq_fence_ops, &userq_fence->lock,
spin_lock_init(&fence->lock);
dma_fence_init64(&fence->base, &amdgpu_userq_fence_ops, &fence->lock,
fence_drv->context, seq);
amdgpu_userq_fence_driver_get(fence_drv);
dma_fence_get(fence);
/* Make sure the fence is visible to the hang detect worker */
dma_fence_put(userq->last_fence);
userq->last_fence = dma_fence_get(&fence->base);
if (!xa_empty(&userq->fence_drv_xa)) {
struct amdgpu_userq_fence_driver *stored_fence_drv;
unsigned long index, count = 0;
int i = 0;
xa_lock(&userq->fence_drv_xa);
xa_for_each(&userq->fence_drv_xa, index, stored_fence_drv)
count++;
userq_fence->fence_drv_array =
kvmalloc_objs(struct amdgpu_userq_fence_driver *, count,
GFP_ATOMIC);
if (userq_fence->fence_drv_array) {
xa_for_each(&userq->fence_drv_xa, index, stored_fence_drv) {
userq_fence->fence_drv_array[i] = stored_fence_drv;
__xa_erase(&userq->fence_drv_xa, index);
i++;
}
}
userq_fence->fence_drv_array_count = i;
xa_unlock(&userq->fence_drv_xa);
} else {
userq_fence->fence_drv_array = NULL;
userq_fence->fence_drv_array_count = 0;
}
/* Check if hardware has already processed the job */
/* Check if hardware has already processed the fence */
spin_lock_irqsave(&fence_drv->fence_list_lock, flags);
if (!dma_fence_is_signaled(fence)) {
list_add_tail(&userq_fence->link, &fence_drv->fences);
if (!dma_fence_is_signaled(&fence->base)) {
dma_fence_get(&fence->base);
list_add_tail(&fence->link, &fence_drv->fences);
} else {
INIT_LIST_HEAD(&fence->link);
signaled = true;
dma_fence_put(fence);
}
spin_unlock_irqrestore(&fence_drv->fence_list_lock, flags);
if (signaled)
amdgpu_userq_fence_put_fence_drv_array(userq_fence);
*f = fence;
return 0;
amdgpu_userq_fence_put_fence_drv_array(fence);
else
amdgpu_userq_start_hang_detect_work(userq);
}
static const char *amdgpu_userq_fence_get_driver_name(struct dma_fence *f)
@ -403,11 +417,6 @@ static int amdgpu_userq_fence_read_wptr(struct amdgpu_device *adev,
return r;
}
static void amdgpu_userq_fence_cleanup(struct dma_fence *fence)
{
dma_fence_put(fence);
}
static void
amdgpu_userq_fence_driver_set_error(struct amdgpu_userq_fence *fence,
int error)
@ -451,13 +460,14 @@ int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data,
const unsigned int num_read_bo_handles = args->num_bo_read_handles;
struct amdgpu_fpriv *fpriv = filp->driver_priv;
struct amdgpu_userq_mgr *userq_mgr = &fpriv->userq_mgr;
struct drm_gem_object **gobj_write, **gobj_read;
u32 *syncobj_handles, num_syncobj_handles;
struct amdgpu_userq_fence *userq_fence;
struct amdgpu_usermode_queue *queue = NULL;
struct drm_syncobj **syncobj = NULL;
struct dma_fence *fence;
struct amdgpu_usermode_queue *queue;
struct amdgpu_userq_fence *fence;
struct drm_syncobj **syncobj;
struct drm_exec exec;
void __user *ptr;
int r, i, entry;
u64 wptr;
@ -469,13 +479,14 @@ int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data,
return -EINVAL;
num_syncobj_handles = args->num_syncobj_handles;
syncobj_handles = memdup_array_user(u64_to_user_ptr(args->syncobj_handles),
num_syncobj_handles, sizeof(u32));
ptr = u64_to_user_ptr(args->syncobj_handles);
syncobj_handles = memdup_array_user(ptr, num_syncobj_handles,
sizeof(u32));
if (IS_ERR(syncobj_handles))
return PTR_ERR(syncobj_handles);
/* Array of pointers to the looked up syncobjs */
syncobj = kmalloc_array(num_syncobj_handles, sizeof(*syncobj), GFP_KERNEL);
syncobj = kmalloc_array(num_syncobj_handles, sizeof(*syncobj),
GFP_KERNEL);
if (!syncobj) {
r = -ENOMEM;
goto free_syncobj_handles;
@ -489,21 +500,17 @@ int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data,
}
}
r = drm_gem_objects_lookup(filp,
u64_to_user_ptr(args->bo_read_handles),
num_read_bo_handles,
&gobj_read);
ptr = u64_to_user_ptr(args->bo_read_handles);
r = drm_gem_objects_lookup(filp, ptr, num_read_bo_handles, &gobj_read);
if (r)
goto free_syncobj;
r = drm_gem_objects_lookup(filp,
u64_to_user_ptr(args->bo_write_handles),
num_write_bo_handles,
ptr = u64_to_user_ptr(args->bo_write_handles);
r = drm_gem_objects_lookup(filp, ptr, num_write_bo_handles,
&gobj_write);
if (r)
goto put_gobj_read;
/* Retrieve the user queue */
queue = amdgpu_userq_get(userq_mgr, args->queue_id);
if (!queue) {
r = -ENOENT;
@ -512,73 +519,61 @@ int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data,
r = amdgpu_userq_fence_read_wptr(adev, queue, &wptr);
if (r)
goto put_gobj_write;
goto put_queue;
r = amdgpu_userq_fence_alloc(&userq_fence);
r = amdgpu_userq_fence_alloc(queue, &fence);
if (r)
goto put_gobj_write;
goto put_queue;
/* We are here means UQ is active, make sure the eviction fence is valid */
amdgpu_userq_ensure_ev_fence(&fpriv->userq_mgr, &fpriv->evf_mgr);
/* Create a new fence */
r = amdgpu_userq_fence_create(queue, userq_fence, wptr, &fence);
if (r) {
mutex_unlock(&userq_mgr->userq_mutex);
kfree(userq_fence);
goto put_gobj_write;
}
/* Create the new fence */
amdgpu_userq_fence_init(queue, fence, wptr);
dma_fence_put(queue->last_fence);
queue->last_fence = dma_fence_get(fence);
amdgpu_userq_start_hang_detect_work(queue);
mutex_unlock(&userq_mgr->userq_mutex);
/*
* This needs to come after the fence is created since
* amdgpu_userq_ensure_ev_fence() can't be called while holding the resv
* locks.
*/
drm_exec_init(&exec, DRM_EXEC_INTERRUPTIBLE_WAIT,
(num_read_bo_handles + num_write_bo_handles));
/* Lock all BOs with retry handling */
drm_exec_until_all_locked(&exec) {
r = drm_exec_prepare_array(&exec, gobj_read, num_read_bo_handles, 1);
r = drm_exec_prepare_array(&exec, gobj_read,
num_read_bo_handles, 1);
drm_exec_retry_on_contention(&exec);
if (r) {
amdgpu_userq_fence_cleanup(fence);
if (r)
goto exec_fini;
}
r = drm_exec_prepare_array(&exec, gobj_write, num_write_bo_handles, 1);
r = drm_exec_prepare_array(&exec, gobj_write,
num_write_bo_handles, 1);
drm_exec_retry_on_contention(&exec);
if (r) {
amdgpu_userq_fence_cleanup(fence);
if (r)
goto exec_fini;
}
}
for (i = 0; i < num_read_bo_handles; i++) {
if (!gobj_read || !gobj_read[i]->resv)
continue;
dma_resv_add_fence(gobj_read[i]->resv, fence,
/* And publish the new fence in the BOs and syncobj */
for (i = 0; i < num_read_bo_handles; i++)
dma_resv_add_fence(gobj_read[i]->resv, &fence->base,
DMA_RESV_USAGE_READ);
}
for (i = 0; i < num_write_bo_handles; i++) {
if (!gobj_write || !gobj_write[i]->resv)
continue;
dma_resv_add_fence(gobj_write[i]->resv, fence,
for (i = 0; i < num_write_bo_handles; i++)
dma_resv_add_fence(gobj_write[i]->resv, &fence->base,
DMA_RESV_USAGE_WRITE);
}
/* Add the created fence to syncobj/BO's */
for (i = 0; i < num_syncobj_handles; i++)
drm_syncobj_replace_fence(syncobj[i], fence);
/* drop the reference acquired in fence creation function */
dma_fence_put(fence);
drm_syncobj_replace_fence(syncobj[i], &fence->base);
exec_fini:
/* drop the reference acquired in fence creation function */
dma_fence_put(&fence->base);
drm_exec_fini(&exec);
put_queue:
amdgpu_userq_put(queue);
put_gobj_write:
for (i = 0; i < num_write_bo_handles; i++)
drm_gem_object_put(gobj_write[i]);
@ -589,15 +584,11 @@ int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data,
kvfree(gobj_read);
free_syncobj:
while (entry-- > 0)
if (syncobj[entry])
drm_syncobj_put(syncobj[entry]);
drm_syncobj_put(syncobj[entry]);
kfree(syncobj);
free_syncobj_handles:
kfree(syncobj_handles);
if (queue)
amdgpu_userq_put(queue);
return r;
}
@ -872,8 +863,10 @@ amdgpu_userq_wait_return_fence_info(struct drm_file *filp,
* Otherwise, we would gather those references until we don't
* have any more space left and crash.
*/
mutex_lock(&waitq->fence_drv_lock);
r = xa_alloc(&waitq->fence_drv_xa, &index, fence_drv,
xa_limit_32b, GFP_KERNEL);
mutex_unlock(&waitq->fence_drv_lock);
if (r)
goto put_waitq;

View File

@ -63,7 +63,7 @@ void amdgpu_userq_fence_driver_put(struct amdgpu_userq_fence_driver *fence_drv);
int amdgpu_userq_fence_driver_alloc(struct amdgpu_device *adev,
struct amdgpu_userq_fence_driver **fence_drv_req);
void amdgpu_userq_fence_driver_free(struct amdgpu_usermode_queue *userq);
void amdgpu_userq_fence_driver_process(struct amdgpu_userq_fence_driver *fence_drv);
int amdgpu_userq_fence_driver_process(struct amdgpu_userq_fence_driver *fence_drv);
void amdgpu_userq_fence_driver_force_completion(struct amdgpu_usermode_queue *userq);
void amdgpu_userq_fence_driver_destroy(struct kref *ref);
int amdgpu_userq_signal_ioctl(struct drm_device *dev, void *data,

View File

@ -602,6 +602,13 @@ static int gfx_v12_0_init_microcode(struct amdgpu_device *adev)
"amdgpu/%s_pfp.bin", ucode_prefix);
if (err)
goto out;
adev->gfx.rs64_enable = amdgpu_ucode_hdr_version(
(union amdgpu_firmware_header *)
adev->gfx.pfp_fw->data, 2, 0);
if (adev->gfx.rs64_enable)
dev_dbg(adev->dev, "CP RS64 enable\n");
amdgpu_gfx_cp_init_microcode(adev, AMDGPU_UCODE_ID_CP_RS64_PFP);
amdgpu_gfx_cp_init_microcode(adev, AMDGPU_UCODE_ID_CP_RS64_PFP_P0_STACK);

View File

@ -92,9 +92,14 @@
#include "dml/dcn32/dcn32_fpu.h"
#include "dc_state_priv.h"
#include "dc_fpu.h"
#include "dml2_0/dml2_wrapper.h"
#if !defined(DC_RUN_WITH_PREEMPTION_ENABLED)
#define DC_RUN_WITH_PREEMPTION_ENABLED(code) code
#endif
#define DC_LOGGER_INIT(logger)
enum dcn32_clk_src_array_id {
@ -1684,7 +1689,8 @@ static void dcn32_enable_phantom_plane(struct dc *dc,
if (curr_pipe->top_pipe && curr_pipe->top_pipe->plane_state == curr_pipe->plane_state)
phantom_plane = prev_phantom_plane;
else
phantom_plane = dc_state_create_phantom_plane(dc, context, curr_pipe->plane_state);
DC_RUN_WITH_PREEMPTION_ENABLED(phantom_plane =
dc_state_create_phantom_plane(dc, context, curr_pipe->plane_state));
if (!phantom_plane)
continue;

View File

@ -222,52 +222,58 @@ static const struct drm_bridge_funcs imx8qxp_pxl2dpi_bridge_funcs = {
imx8qxp_pxl2dpi_bridge_atomic_get_output_bus_fmts,
};
static struct device_node *
static int
imx8qxp_pxl2dpi_get_available_ep_from_port(struct imx8qxp_pxl2dpi *p2d,
u32 port_id)
u32 port_id,
struct device_node **ep)
{
struct device_node *port, *ep;
struct device_node *port;
int ret = 0;
int ep_cnt;
*ep = NULL;
port = of_graph_get_port_by_id(p2d->dev->of_node, port_id);
if (!port) {
DRM_DEV_ERROR(p2d->dev, "failed to get port@%u\n", port_id);
return ERR_PTR(-ENODEV);
return -ENODEV;
}
ep_cnt = of_get_available_child_count(port);
if (ep_cnt == 0) {
DRM_DEV_ERROR(p2d->dev, "no available endpoints of port@%u\n",
port_id);
ep = ERR_PTR(-ENODEV);
ret = -ENODEV;
goto out;
} else if (ep_cnt > 1) {
DRM_DEV_ERROR(p2d->dev,
"invalid available endpoints of port@%u\n",
port_id);
ep = ERR_PTR(-EINVAL);
ret = -EINVAL;
goto out;
}
ep = of_get_next_available_child(port, NULL);
if (!ep) {
*ep = of_get_next_available_child(port, NULL);
if (!*ep) {
DRM_DEV_ERROR(p2d->dev,
"failed to get available endpoint of port@%u\n",
port_id);
ep = ERR_PTR(-ENODEV);
ret = -ENODEV;
goto out;
}
out:
of_node_put(port);
return ep;
return ret;
}
static int imx8qxp_pxl2dpi_find_next_bridge(struct imx8qxp_pxl2dpi *p2d)
{
struct device_node *ep __free(device_node) =
imx8qxp_pxl2dpi_get_available_ep_from_port(p2d, 1);
if (IS_ERR(ep))
return PTR_ERR(ep);
struct device_node *ep __free(device_node) = NULL;
int ret;
ret = imx8qxp_pxl2dpi_get_available_ep_from_port(p2d, 1, &ep);
if (ret)
return ret;
struct device_node *remote __free(device_node) = of_graph_get_remote_port_parent(ep);
if (!remote || !of_device_is_available(remote)) {
@ -291,9 +297,9 @@ static int imx8qxp_pxl2dpi_set_pixel_link_sel(struct imx8qxp_pxl2dpi *p2d)
struct of_endpoint endpoint;
int ret;
ep = imx8qxp_pxl2dpi_get_available_ep_from_port(p2d, 0);
if (IS_ERR(ep))
return PTR_ERR(ep);
ret = imx8qxp_pxl2dpi_get_available_ep_from_port(p2d, 0, &ep);
if (ret)
return ret;
ret = of_graph_parse_endpoint(ep, &endpoint);
if (ret) {

View File

@ -1067,17 +1067,12 @@ int drm_gem_change_handle_ioctl(struct drm_device *dev, void *data,
spin_unlock(&file_priv->table_lock);
if (ret < 0)
goto out_unlock;
if (obj->dma_buf) {
ret = drm_prime_add_buf_handle(&file_priv->prime, obj->dma_buf,
handle);
if (ret < 0) {
spin_lock(&file_priv->table_lock);
idr_remove(&file_priv->object_idr, handle);
idrobj = idr_replace(&file_priv->object_idr, obj, handle);
WARN_ON(idrobj != NULL);
spin_unlock(&file_priv->table_lock);
goto out_unlock;
}
@ -1089,7 +1084,9 @@ int drm_gem_change_handle_ioctl(struct drm_device *dev, void *data,
spin_lock(&file_priv->table_lock);
idr_remove(&file_priv->object_idr, args->handle);
idrobj = idr_replace(&file_priv->object_idr, obj, handle);
spin_unlock(&file_priv->table_lock);
WARN_ON(idrobj != NULL);
out_unlock:
mutex_unlock(&file_priv->prime.lock);

View File

@ -580,6 +580,7 @@ static int oaktrail_hdmi_get_modes(struct drm_connector *connector)
} else {
edid = (struct edid *)raw_edid;
/* FIXME ? edid = drm_get_edid(connector, i2c_adap); */
i2c_put_adapter(i2c_adap);
}
if (edid) {

View File

@ -293,7 +293,7 @@ void oaktrail_lvds_init(struct drm_device *dev,
{
struct gma_encoder *gma_encoder;
struct gma_connector *gma_connector;
struct gma_i2c_chan *ddc_bus;
struct gma_i2c_chan *ddc_bus = NULL;
struct drm_connector *connector;
struct drm_encoder *encoder;
struct drm_psb_private *dev_priv = to_drm_psb_private(dev);
@ -367,6 +367,8 @@ void oaktrail_lvds_init(struct drm_device *dev,
if (edid == NULL && dev_priv->lpc_gpio_base) {
ddc_bus = oaktrail_lvds_i2c_init(dev);
if (!IS_ERR(ddc_bus)) {
if (i2c_adap)
i2c_put_adapter(i2c_adap);
i2c_adap = &ddc_bus->base;
edid = drm_get_edid(connector, i2c_adap);
}
@ -421,7 +423,10 @@ void oaktrail_lvds_init(struct drm_device *dev,
err_unlock:
mutex_unlock(&dev->mode_config.mutex);
gma_i2c_destroy(to_gma_i2c_chan(connector->ddc));
if (!IS_ERR_OR_NULL(ddc_bus))
gma_i2c_destroy(ddc_bus);
else if (i2c_adap)
i2c_put_adapter(i2c_adap);
drm_encoder_cleanup(encoder);
err_connector_cleanup:
drm_connector_cleanup(connector);

View File

@ -3119,8 +3119,13 @@ static void intel_dp_compute_vsc_colorimetry(const struct intel_crtc_state *crtc
drm_WARN_ON(display->drm,
vsc->bpc == 6 && vsc->pixelformat != DP_PIXELFORMAT_RGB);
/* all YCbCr are always limited range */
vsc->dynamic_range = DP_DYNAMIC_RANGE_CTA;
/* All YCbCr formats are always limited range. */
if (vsc->pixelformat == DP_PIXELFORMAT_RGB)
vsc->dynamic_range = crtc_state->limited_color_range ?
DP_DYNAMIC_RANGE_CTA : DP_DYNAMIC_RANGE_VESA;
else
vsc->dynamic_range = DP_DYNAMIC_RANGE_CTA;
vsc->content_type = DP_CONTENT_TYPE_NOT_DEFINED;
}

View File

@ -132,7 +132,8 @@ void __i915_request_reset(struct i915_request *rq, bool guilty)
rcu_read_lock(); /* protect the GEM context */
if (guilty) {
i915_request_set_error_once(rq, -EIO);
__i915_request_skip(rq);
if (!i915_request_signaled(rq))
__i915_request_skip(rq);
banned = mark_guilty(rq);
} else {
i915_request_set_error_once(rq, -EAGAIN);

View File

@ -293,7 +293,7 @@ static int lsdc_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
vga_client_register(pdev, lsdc_vga_set_decode);
drm_kms_helper_poll_init(ddev);
drmm_kms_helper_poll_init(ddev);
if (loongson_vblank) {
ret = drm_vblank_init(ddev, descp->num_of_crtc);

View File

@ -390,6 +390,8 @@ panfrost_ioctl_wait_bo(struct drm_device *dev, void *data,
true, timeout);
if (!ret)
ret = timeout ? -ETIMEDOUT : -EBUSY;
else if (ret > 0)
ret = 0;
drm_gem_object_put(gem_obj);

View File

@ -739,7 +739,7 @@ static int ttm_bo_alloc_resource(struct ttm_buffer_object *bo,
may_evict = (force_space && place->mem_type != TTM_PL_SYSTEM);
ret = ttm_resource_alloc(bo, place, res, force_space ? &limit_pool : NULL);
if (ret) {
if (ret != -ENOSPC && ret != -EAGAIN) {
if (ret != -ENOSPC) {
dmem_cgroup_pool_state_put(limit_pool);
return ret;
}
@ -1177,17 +1177,13 @@ ttm_bo_swapout_cb(struct ttm_lru_walk *walk, struct ttm_buffer_object *bo)
bdev->funcs->swap_notify(bo);
if (ttm_tt_is_populated(tt)) {
spin_lock(&bdev->lru_lock);
ttm_resource_del_bulk_move(bo->resource, bo);
spin_unlock(&bdev->lru_lock);
ret = ttm_tt_swapout(bdev, tt, swapout_walk->gfp_flags);
spin_lock(&bdev->lru_lock);
if (ret)
ttm_resource_add_bulk_move(bo->resource, bo);
ttm_resource_move_to_lru_tail(bo->resource);
spin_unlock(&bdev->lru_lock);
if (!ret) {
spin_lock(&bdev->lru_lock);
ttm_resource_del_bulk_move_unevictable(bo->resource, bo);
ttm_resource_move_to_lru_tail(bo->resource);
spin_unlock(&bdev->lru_lock);
}
}
out:

View File

@ -1112,19 +1112,14 @@ long ttm_bo_shrink(struct ttm_operation_ctx *ctx, struct ttm_buffer_object *bo,
if (lret < 0)
return lret;
if (bo->bulk_move) {
spin_lock(&bdev->lru_lock);
ttm_resource_del_bulk_move(bo->resource, bo);
spin_unlock(&bdev->lru_lock);
}
lret = ttm_tt_backup(bdev, bo->ttm, (struct ttm_backup_flags)
{.purge = flags.purge,
.writeback = flags.writeback});
if (lret <= 0 && bo->bulk_move) {
if (lret > 0) {
spin_lock(&bdev->lru_lock);
ttm_resource_add_bulk_move(bo->resource, bo);
ttm_resource_del_bulk_move_unevictable(bo->resource, bo);
ttm_resource_move_to_lru_tail(bo->resource);
spin_unlock(&bdev->lru_lock);
}

View File

@ -292,6 +292,19 @@ void ttm_resource_del_bulk_move(struct ttm_resource *res,
ttm_lru_bulk_move_del(bo->bulk_move, res);
}
/*
* Remove a resource from its bulk_move, bypassing the unevictable check.
* Use only when the resource is known to still be tracked in the range despite
* the BO having just become unevictable; asserts that this is the case.
*/
void ttm_resource_del_bulk_move_unevictable(struct ttm_resource *res,
struct ttm_buffer_object *bo)
{
WARN_ON_ONCE(!ttm_resource_unevictable(res, bo));
if (bo->bulk_move)
ttm_lru_bulk_move_del(bo->bulk_move, res);
}
/* Move a resource to the LRU or bulk tail */
void ttm_resource_move_to_lru_tail(struct ttm_resource *res)
{
@ -385,8 +398,11 @@ int ttm_resource_alloc(struct ttm_buffer_object *bo,
if (man->cg) {
ret = dmem_cgroup_try_charge(man->cg, bo->base.size, &pool, ret_limit_pool);
if (ret)
if (ret) {
if (ret == -EAGAIN)
ret = -ENOSPC;
return ret;
}
}
ret = man->func->alloc(man, bo, place, res_ptr);

View File

@ -897,10 +897,10 @@ void xe_bo_set_purgeable_state(struct xe_bo *bo,
new_state == XE_MADV_PURGEABLE_PURGED);
/* Once purged, always purged - cannot transition out */
xe_assert(xe, !(bo->madv_purgeable == XE_MADV_PURGEABLE_PURGED &&
xe_assert(xe, !(bo->purgeable.state == XE_MADV_PURGEABLE_PURGED &&
new_state != XE_MADV_PURGEABLE_PURGED));
bo->madv_purgeable = new_state;
bo->purgeable.state = new_state;
xe_bo_set_purgeable_shrinker(bo, new_state);
}
@ -2368,7 +2368,7 @@ struct xe_bo *xe_bo_init_locked(struct xe_device *xe, struct xe_bo *bo,
INIT_LIST_HEAD(&bo->vram_userfault_link);
/* Initialize purge advisory state */
bo->madv_purgeable = XE_MADV_PURGEABLE_WILLNEED;
bo->purgeable.state = XE_MADV_PURGEABLE_WILLNEED;
drm_gem_private_object_init(&xe->drm, &bo->ttm.base, size);

Some files were not shown because too many files have changed in this diff Show More