github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-11-06 21:58:01 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph

VFIO updates for v7.4-rc1

Browse Source 
- Fix missing selftest <libgen.h> include for basename() to support
    non-glibc builds. (Hisam Mehboob)
 
  - Add vfio selftest documentation covering the libvfio helper library
    and guidance for developers to aid in adding and understanding the
    framework. (David Matlack)
 
  - Fix vfio cdev open path to prevent a second open from clobbering
    the vf_token or clearing the KVM reference.  Add a selftest to
    avoid future recurrences. (Alex Williamson)
 
  - Extend vfio selftest mix_and_match timeouts, in particular to
    account for systems where generated faults can induce heavy SMM
    load, delaying internal timeouts.  Also extend timeout for
    execution via runner to account for various tests exceeding default
    45s timeout.  (Alex Williamson)
 
  - Add tracking for non-reserved mappings into a vfio_dma object in
    order to optimize the tear-down paths where per-page tests can be
    entirely skipped.  This resolves a soft lockup for large memory
    GPUs. (Samuel Crossley)
 
  - Add vfio selftest driver for mlx5, providing another target for
    exercising DMA and interrupts against a physical device.
    (Jason Gunthorpe)
 
  - Hardening against read-only region promotion, preserving vm_pgoff
    for unmap_mapping_range(), and reject non-shared MMIO mmaps as
    necessary through vfio-platform, vfio-fsl-mc, and vfio-cdx.  These
    prevent future issues, though none are currently reachable.
    (Abdifatah Suruur)
 
  - Fix 8-byte ioeventfd support, removing a static error test that was
    erroneously left in place when generalizing 8-byte io accesses.
    (Li RongQing)
 
  - Fix missed RCU drain on module exit. (Jiale Yao)
 
  - Fix off-by-one boundary check in migration support within the mlx5
    vfio-pci variant driver. (Seongjun Hong)
 
  - Remove drvdata usage convention across vfio-pci-core, allowing
    variant drivers to freely create their own conventions and enabling
    a future Rust-based variant driver.  A DRM topic branch is also
    merged in support of this effort, modifying the vga arbiter API
    to more cleanly remove the drvdata dependency. (Simon Song)
 
  - Fix hisi_acc vfio-pci variant driver to avoid NULL pointer accesses
    when bound to unsupported devices and also disabling migration
    support when running on 64K host page size due to BAR mapping
    constraints. (Longfang Liu)
 
  - Add device IDs for Crescent Island and Nova Lake devices to the Xe
    vfio-pci variant driver. (Satyanarayana K V P)
 
  - Fix fallback mode in nvgrace-gpu variant driver to use symmetric
    open and close handlers to avoid reported use-after-free issue.
    Remove the resulting unnecessary test in the now single-purpose
    open function. (Andrea Parri)
 -----BEGIN PGP SIGNATURE-----
 
 iQJFBAABCgAvFiEEQvbATlQL0amee4qQI5ubbjuwiyIFAmrZgKwRHGFsZXhAc2hh
 emJvdC5vcmcACgkQI5ubbjuwiyJzzQ//YVE+dgmZyop6V4INVheuYOt9jlTwEsbN
 CV7BcHG2AQt/0yx9LtKQ6CFKJSJYcPUTQjBq5FMYxXVUYiKCjn2czk7gKJeCXX+6
 8AkWqNi2Ds46j7YwRaNZEP4yrDPmZeQw5Z/cqE95zBB2QtujGl3FJ4Jt6hAnT3AM
 EIzMT4ZkOWGOaurK9blUY4sH+N8Ft9haJeAeEsBtT1wSD52MvAdXdCKhhfxR0vm9
 9R9/DYNVff5ynFOWDIWUqspxT3ITIX2iPJMmsPzlXZXytIjwTcA3NSmH48KvNSCw
 iI9+UYPHU9wBoRlrh020ZPl2kxFoOEZLCfonCgEeQuEtZiyaEjmjZ8uU4J/JF6UO
 Uu4xUG9DBjNtqbDeEtPlGM/6Scuqn+EDlEGHMVOhjJwllGVilGeVeA93wJo064J2
 lXGsvQ6EBP6VDzfvbW/7KApm6JwIjV5h4/+bA0Eq5wFkNxDPoahtjh2IvYflLpIV
 2jBIC+OG759QBnJ+KlHI+07m1tQKy5MdYBJg8Lpo4pL/wH14UXTDNnEYPdusrKql
 GAuMERJe+3/PrL3xzgptwDBuVwIkWzX8ivdWPWQsIgq0OY/RpENZBgeRL7nXrIpn
 lT14Os+FOflGeVyIXJS14cqG+gzTV5/nEP+EDXyOIwKL2Kff5zBFpXfyRiYXtwvJ
 oRsxu6XNfT8=
 =zgQL
 -----END PGP SIGNATURE-----

Merge tag 'vfio-v7.4-rc1' of https://github.com/awilliam/linux-vfio

Pull VFIO updates from Alex Williamson:

 - Fix missing selftest <libgen.h> include for basename() to support
   non-glibc builds (Hisam Mehboob)

 - Add vfio selftest documentation covering the libvfio helper library
   and guidance for developers to aid in adding and understanding the
   framework (David Matlack)

 - Fix vfio cdev open path to prevent a second open from clobbering the
   vf_token or clearing the KVM reference. Add a selftest to avoid
   future recurrences (Alex Williamson)

 - Extend vfio selftest mix_and_match timeouts, in particular to account
   for systems where generated faults can induce heavy SMM load,
   delaying internal timeouts. Also extend timeout for execution via
   runner to account for various tests exceeding default 45s timeout
   (Alex Williamson)

 - Add tracking for non-reserved mappings into a vfio_dma object in
   order to optimize the tear-down paths where per-page tests can be
   entirely skipped. This resolves a soft lockup for large memory GPUs
   (Samuel Crossley)

 - Add vfio selftest driver for mlx5, providing another target for
   exercising DMA and interrupts against a physical device (Jason
   Gunthorpe)

 - Hardening against read-only region promotion, preserving vm_pgoff for
   unmap_mapping_range(), and reject non-shared MMIO mmaps as necessary
   through vfio-platform, vfio-fsl-mc, and vfio-cdx. These prevent
   future issues, though none are currently reachable (Abdifatah Suruur)

 - Fix 8-byte ioeventfd support, removing a static error test that was
   erroneously left in place when generalizing 8-byte io accesses (Li
   RongQing)

 - Fix missed RCU drain on module exit (Jiale Yao)

 - Fix off-by-one boundary check in migration support within the mlx5
   vfio-pci variant driver (Seongjun Hong)

 - Remove drvdata usage convention across vfio-pci-core, allowing
   variant drivers to freely create their own conventions and enabling a
   future Rust-based variant driver. A DRM topic branch is also merged
   in support of this effort, modifying the vga arbiter API to more
   cleanly remove the drvdata dependency (Simon Song)

 - Fix hisi_acc vfio-pci variant driver to avoid NULL pointer accesses
   when bound to unsupported devices and also disabling migration
   support when running on 64K host page size due to BAR mapping
   constraints (Longfang Liu)

 - Add device IDs for Crescent Island and Nova Lake devices to the Xe
   vfio-pci variant driver (Satyanarayana K V P)

 - Fix fallback mode in nvgrace-gpu variant driver to use symmetric open
   and close handlers to avoid reported use-after-free issue. Remove the
   resulting unnecessary test in the now single-purpose open function
   (Andrea Parri)

* tag 'vfio-v7.4-rc1' of https://github.com/awilliam/linux-vfio: (35 commits)
  vfio/nvgrace-gpu: Drop the usemem length check in open_device
  vfio/nvgrace-gpu: Use a core-only open_device in fallback mode
  vfio/xe: Add CRI and NVL device IDs
  hisi_acc_vfio_pci: reject live migration on 64KB page with QM_HW_V3 hardware
  hisi_acc_vfio_pci: fix NULL dereference in reset_prepare on PF passthrough
  vfio/pci: Remove the core dependency on driver data
  vfio/mlx5: Fix boundary check in set_report_output()
  vfio/pci: Drain eventfd RCU callbacks on module exit
  PCI/VGA: pass a data pointer to be used in the decode callback (v2)
  vfio/pci: Restore 8-byte ioeventfd support
  vfio/cdx: reject non-shared MMIO mmaps
  vfio/cdx: keep logical vm_pgoff in MMIO region mmap
  vfio/fsl-mc: keep logical vm_pgoff in MMIO region mmap
  vfio/platform: keep logical vm_pgoff in MMIO region mmap
  vfio/cdx: prevent read-only region mappings from becoming writable
  vfio/fsl-mc: prevent read-only region mappings from becoming writable
  vfio/platform: prevent read-only region mappings from becoming writable
  vfio: selftests: Verify a failed second open preserves the vf_token
  vfio: selftests: mlx5 driver - add send_msi support
  vfio: selftests: Add mlx5 driver - data path and memcpy ops
  ...
This commit is contained in:
Linus Torvalds 2026-10-23 14:22:52 -10:00
commit 26344c0dd3
61 changed files with 3080 additions and 420 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

1
Documentation/driver-api/index.rst
View File

@ -47,6 +47,7 @@ of interest to most developers working on device drivers.
vfio-mediated-device
vfio
vfio-pci-device-specific-driver-acceptance
vfio-selftests
Bus-level documentation
=======================

252
Documentation/driver-api/vfio-selftests.rst Normal file
View File

@ -0,0 +1,252 @@
.. SPDX-License-Identifier: GPL-2.0
==============
VFIO Selftests
==============
VFIO selftests are built on top of the kernel's selftests framework and are
located in ``tools/testing/selftests/vfio``.
VFIO selftests enable kernel developers to write and run tests that take the
form of userspace programs that interact with VFIO and IOMMUFD uAPIs. VFIO
selftests can be used to write functional tests for new features, regression
tests for bugs, and performance tests for optimizations.
These tests are designed to interact with real PCI devices, i.e. they do not
rely on mocking out or faking any behavior in the kernel. This allows the tests
to exercise not only VFIO but also IOMMUFD, the IOMMU driver, interrupt
remapping, IRQ handling, etc.
Running Tests
=============
Running VFIO selftests requires a device bound to the ``vfio-pci`` driver.
There are scripts in ``tools/testing/selftests/vfio/scripts/`` to help with
setting one up.
Picking a Device
----------------
Some VFIO selftests require a device with a supported driver implementation in
the selftests library (see `Driver Framework`_), while other tests (such as
basic PCI configuration space or reset tests) can use any arbitrary PCI device.
This section will be expanded in the future to guide developers on how to find
a device with a supported driver. In the meantime, the selftests library
includes driver support for the following devices:
- Intel IGB Gigabit Ethernet controllers
- Intel IOAT (I/O Acceleration Technology) DMA engines
- Intel DSA (Data Streaming Accelerator)
- NVIDIA Falcon engines (found in many NVIDIA GPUs, e.g., GTX 1080 or RTX 2080)
Note that using Intel DSA devices requires setting the module parameter
``vfio_pci.disable_denylist=Y`` to allow ``vfio-pci`` to bind to DSA devices.
Setting up Devices
------------------
.. code-block:: sh
tools/testing/selftests/vfio/scripts/setup.sh 0000:01:00.0
The script handles unbinding the device from its current driver and binding it
to ``vfio-pci``. Metadata about this device is stored in
``/tmp/vfio-selftests-devices`` so that the device can be cleaned up later
(bound back to its original driver).
Execution
---------
VFIO selftests expect the PCI device BDF string as their final command-line
argument. e.g.
.. code-block:: sh
tools/testing/selftests/vfio/vfio_pci_device_test 0000:01:00.0
Alternatively, the PCI device BDF can be passed via the ``VFIO_SELFTESTS_BDF``
environment variable, which can be useful when running all tests:
.. code-block:: sh
export VFIO_SELFTESTS_BDF=0000:01:00.0
make -C tools/testing/selftests TARGETS=vfio install
tools/testing/selftests/kselftest_install/run_kselftest.sh -c vfio
Cleanup
-------
.. code-block:: sh
tools/testing/selftests/vfio/scripts/cleanup.sh
This script handles unbinding all previously setup devices from ``vfio-pci``
and binding them back to their original driver. Specific devices can be cleaned
up by passing their BDF string to the cleanup script.
Writing Tests
=============
VFIO selftests leverage the ``kselftest_harness.h`` macros to structure setup
and teardown scenarios. A basic test involves:
1. Declaring a test FIXTURE containing the device and IOMMU state.
2. Generating test variants that span the different IOMMU modes utilizing
``FIXTURE_VARIANT_ADD_ALL_IOMMU_MODES()``.
3. Initializing the device and binding the default driver ops.
For a complete example of writing a VFIO selftest, refer to
``tools/testing/selftests/vfio/vfio_pci_device_test.c``.
VFIO Selftests Library (libvfio)
================================
The selftests directory contains a helper library compiled from the ``lib/``
subdirectory. It provides structures and helper functions to handle standard
operations like opening VFIO devices, creating VFIO containers, and interacting
with VFIO and IOMMUFD.
All VFIO selftests have libvfio linked in by default. Tests can access the
library by including ``<libvfio.h>``.
Core Objects
------------
The VFIO selftests framework revolves around two distinct core objects:
``struct iommu`` and ``struct vfio_pci_device``. It is important to understand
the distinction and relationship between them:
- ``struct iommu`` represents an IOMMU domain and the API used to program it
(e.g. legacy VFIO Type1 vs IOMMUFD). It is responsible for managing the I/O
address space, handling DMA mappings, and translating between host virtual
addresses (HVA) and IO virtual addresses (IOVA).
- ``struct vfio_pci_device`` represents a single PCI device. It encapsulates
the VFIO device file descriptor, handles access to the PCI configuration
space, manages memory-mapped BARs, and drives device interrupts.
A typical test initializes an IOMMU instance first, then passes it when
initializing one or more PCI devices, linking the devices to the shared IOMMU
domain.
``struct iommu``
^^^^^^^^^^^^^^^^
The ``struct iommu`` abstracts away the complexity of managing VFIO containers,
IOMMU groups, and IOMMUFD contexts. A test allocates its IOMMU handle using
``iommu_init()`` passing one of the supported IOMMU modes:
.. code-block:: c
struct iommu *iommu = iommu_init(MODE_IOMMUFD);
...
iommu_cleanup(iommu);
The selftest framework is designed to work uniformly across all standard kernel
IOMMU APIs. The defined modes (in ``iommu.h``) include:
- ``MODE_VFIO_TYPE1_IOMMU``
- ``MODE_VFIO_TYPE1V2_IOMMU``
- ``MODE_IOMMUFD_COMPAT_TYPE1``
- ``MODE_IOMMUFD_COMPAT_TYPE1V2``
- ``MODE_IOMMUFD``
VFIO selftests can replicate their test cases across all possible IOMMU modes
using FIXTURE_VARIANT_ADD_ALL_IOMMU_MODES() to ensure kernel IOMMU APIs remain
compatible.
``struct vfio_pci_device``
^^^^^^^^^^^^^^^^^^^^^^^^^^
A PCI device is represented as a ``struct vfio_pci_device``. The device
representation is initialized using ``vfio_pci_device_init()``, which takes a
target BDF string, opens the device in VFIO, and binds it to an existing
``struct iommu`` instance:
.. code-block:: c
struct vfio_pci_device *device = vfio_pci_device_init(device_bdf, iommu);
...
vfio_pci_device_cleanup(device);
``struct iova_allocator``
^^^^^^^^^^^^^^^^^^^^^^^^^
To facilitate DMA mappings without hardcoding IOVAs (IO virtual addresses) that
might conflict with reserved platform address ranges, the library provides a
``struct iova_allocator``.
The IOVA allocator queries the underlying IOMMU (via ``struct iommu``) for its
valid target IOVA ranges. Tests can safely carve out unique contiguous chunks
of IOVA space for device DMA by repeatedly calling:
.. code-block:: c
struct iova_allocator *allocator = iova_allocator_init(iommu);
iova_t addr = iova_allocator_alloc(allocator, size);
Driver Framework
----------------
The primary goal of VFIO selftests is to verify the correctness of the kernel
code handling VFIO devices, including VFIO core, IOMMUFD, IOMMU drivers, page
pinning, DMA mapping/unmapping, interrupt remapping, and IRQ delivery.
Because different physical devices have vastly different programming interfaces
for triggering work, the selftests library implements a standard driver
framework. This framework abstracts diverse hardware endpoints (such as Intel
IGB, IOAT, or DSA) behind a uniform interface, allowing tests to provoke a
broad range of devices to generate DMA operations and interrupts in a
standardized manner.
By exercising DMA and interrupts with real endpoints, tests can thoroughly
stress and validate the underlying kernel paths, such as IOMMU page table
management, dirty page tracking, Device TLB and IOTLB invalidations, and
interrupt delivery under authentic hardware conditions.
All endpoint driver implementations populate a common set of operations defined
in ``struct vfio_pci_driver_ops``:
- ``probe()``: Validate whether the driver supports the current PCI device
(e.g. matching vendor/device IDs).
- ``init()``: Set up specific control registers or ring-buffer data structures.
- ``remove()``: Clean up structures before destruction.
- ``memcpy_start()``: Initiate DMA copies via the device.
- ``memcpy_wait()``: Poll and wait until previously initiated copies complete
safely.
- ``send_msi()``: Provoke the device to signal an MSI interrupt.
Tests are required to set up a dedicated DMA region for the driver to use
within ``struct vfio_pci_driver`` to serve as the driver's main memory
allocation region (e.g. used for in-memory descriptor rings and hardware
buffers). Tests must provision and map this DMA region before completing driver
initialization.
For a complete example of interacting with the driver framework from a test
perspective, refer to ``tools/testing/selftests/vfio/vfio_pci_driver_test.c``.
When contributing a new driver implementation to the selftests library, the
primary acceptance criteria is that ``vfio_pci_driver_test`` passes using the
new driver and target device.
Integration with KVM Selftests
------------------------------
libvfio is not strictly limited to VFIO selftests, it is also leveraged within
KVM selftests (``tools/testing/selftests/kvm/``). KVM selftests build their own
copy of libvfio and link against it, enabling libvfio to be used in KVM
selftests.
This integration enables testing the interaction between KVM and VFIO and
IOMMUFD. For example, testing the end-to-end delivery of device interrupts into
running vCPUs.
Contribution Guidelines
=======================
- Prefer the shortlog prefix ``vfio: selftests: ...`` for commits that modify
files in ``tools/testing/selftests/vfio/``.
- Follow the Coding Style, Comments, Changelogs, and Function References
guidelines from the
:doc:`KVM x86 maintainers handbook </process/maintainer-kvm-x86>`.

1
MAINTAINERS
View File

@ -28974,6 +28974,7 @@ VFIO SELFTESTS
M: David Matlack <dmatlack@google.com>
L: kvm@vger.kernel.org
S: Maintained
F: Documentation/driver-api/vfio-selftests.rst
F: tools/testing/selftests/vfio/
VFIO VIRTIO PCI DRIVER

9
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
View File

@ -1429,16 +1429,15 @@ bool amdgpu_device_should_use_aspm(struct amdgpu_device *adev)
/**
* amdgpu_device_vga_set_decode - enable/disable vga decode
*
* @pdev: PCI device pointer
* @data: driver private data (adev)
* @state: enable/disable vga decode
*
* Enable/disable vga decode (all asics).
* Returns VGA resource flags.
*/
static unsigned int amdgpu_device_vga_set_decode(struct pci_dev *pdev,
bool state)
static unsigned int amdgpu_device_vga_set_decode(void *data, bool state)
{
struct amdgpu_device *adev = drm_to_adev(pci_get_drvdata(pdev));
struct amdgpu_device *adev = data;
amdgpu_asic_set_vga_state(adev, state);
if (state)
@ -4199,7 +4198,7 @@ int amdgpu_device_init(struct amdgpu_device *adev,
* ignore it
*/
if ((adev->pdev->class >> 8) == PCI_CLASS_DISPLAY_VGA)
vga_client_register(adev->pdev, amdgpu_device_vga_set_decode);
vga_client_register(adev->pdev, amdgpu_device_vga_set_decode, adev);
px = amdgpu_device_supports_px(adev);

7
drivers/gpu/drm/i915/display/intel_vga.c
View File

@ -283,9 +283,10 @@ void intel_vga_disable(struct intel_display *display)
intel_de_posting_read(display, vga_reg);
}
static unsigned int intel_vga_set_decode(struct pci_dev *pdev, bool enable_decode)
static unsigned int intel_vga_set_decode(void *data, bool enable_decode)
{
struct intel_display *display = to_intel_display(pdev);
struct intel_display *display = data;
struct pci_dev *pdev = to_pci_dev(display->drm->dev);
unsigned int decodes = VGA_RSRC_NORMAL_IO | VGA_RSRC_NORMAL_MEM;
drm_dbg_kms(display->drm, "%s VGA decode due to VGA arbitration\n",
@ -331,7 +332,7 @@ void intel_vga_register(struct intel_display *display)
* then we do not take part in VGA arbitration and the
* vga_client_register() fails with -ENODEV.
*/
ret = vga_client_register(pdev, intel_vga_set_decode);
ret = vga_client_register(pdev, intel_vga_set_decode, display);
drm_WARN_ON(display->drm, ret && ret != -ENODEV);
}

4
drivers/gpu/drm/loongson/lsdc_drv.c
View File

@ -254,7 +254,7 @@ lsdc_create_device(struct pci_dev *pdev,
/* For multiple GPU driver instance co-exixt in the system */
static unsigned int lsdc_vga_set_decode(struct pci_dev *pdev, bool state)
static unsigned int lsdc_vga_set_decode(void *data, bool state)
{
return VGA_RSRC_NORMAL_IO | VGA_RSRC_NORMAL_MEM;
}
@ -291,7 +291,7 @@ static int lsdc_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
pci_set_drvdata(pdev, ddev);
vga_client_register(pdev, lsdc_vga_set_decode);
vga_client_register(pdev, lsdc_vga_set_decode, NULL);
drmm_kms_helper_poll_init(ddev);

6
drivers/gpu/drm/nouveau/nouveau_vga.c
View File

@ -9,9 +9,9 @@
#include "nouveau_vga.h"
static unsigned int
nouveau_vga_set_decode(struct pci_dev *pdev, bool state)
nouveau_vga_set_decode(void *data, bool state)
{
struct nouveau_drm *drm = pci_get_drvdata(pdev);
struct nouveau_drm *drm = data;
struct nvif_object *device = &drm->client.device.object;
if (drm->client.device.info.family == NV_DEVICE_INFO_V0_CURIE &&
@ -95,7 +95,7 @@ nouveau_vga_init(struct nouveau_drm *drm)
return;
pdev = to_pci_dev(dev->dev);
vga_client_register(pdev, nouveau_vga_set_decode);
vga_client_register(pdev, nouveau_vga_set_decode, drm);
/* don't register Thunderbolt eGPU with vga_switcheroo */
if (pci_is_thunderbolt_attached(pdev))

9
drivers/gpu/drm/radeon/radeon_device.c
View File

@ -1070,16 +1070,15 @@ void radeon_combios_fini(struct radeon_device *rdev)
/**
* radeon_vga_set_decode - enable/disable vga decode
*
* @pdev: PCI device
* @data: driver private data (rdev)
* @state: enable/disable vga decode
*
* Enable/disable vga decode (all asics).
* Returns VGA resource flags.
*/
static unsigned int radeon_vga_set_decode(struct pci_dev *pdev, bool state)
static unsigned int radeon_vga_set_decode(void *data, bool state)
{
struct drm_device *dev = pci_get_drvdata(pdev);
struct radeon_device *rdev = dev->dev_private;
struct radeon_device *rdev = data;
radeon_vga_set_state(rdev, state);
if (state)
return VGA_RSRC_LEGACY_IO | VGA_RSRC_LEGACY_MEM |
@ -1424,7 +1423,7 @@ int radeon_device_init(struct radeon_device *rdev,
/* if we have > 1 VGA cards, then disable the radeon VGA resources */
/* this will fail for cards that aren't VGA class devices, just
* ignore it */
vga_client_register(rdev->pdev, radeon_vga_set_decode);
vga_client_register(rdev->pdev, radeon_vga_set_decode, rdev);
if (rdev->flags & RADEON_IS_PX)
runtime = true;

13
drivers/pci/vgaarb.c
View File

@ -51,7 +51,8 @@ struct vga_device {
unsigned int mem_norm_cnt; /* normal MEM count */
bool bridge_has_one_vga;
bool is_firmware_default; /* device selected by firmware */
unsigned int (*set_decode)(struct pci_dev *pdev, bool decode);
unsigned int (*set_decode)(void *data, bool decode);
void *set_decode_data;
};
static LIST_HEAD(vga_list);
@ -935,6 +936,7 @@ EXPORT_SYMBOL(vga_set_legacy_decoding);
* vga_client_register - register or unregister a VGA arbitration client
* @pdev: PCI device of the VGA client
* @set_decode: VGA decode change callback
* @data: private data to pass to callback
*
* Clients have two callback mechanisms they can use.
*
@ -956,15 +958,18 @@ EXPORT_SYMBOL(vga_set_legacy_decoding);
* Returns: 0 on success, -ENODEV on failure
*/
int vga_client_register(struct pci_dev *pdev,
unsigned int (*set_decode)(struct pci_dev *pdev, bool decode))
unsigned int (*set_decode)(void *data, bool decode),
void *data)
{
unsigned long flags;
struct vga_device *vgadev;
spin_lock_irqsave(&vga_lock, flags);
vgadev = vgadev_find(pdev);
if (vgadev)
if (vgadev) {
vgadev->set_decode = set_decode;
vgadev->set_decode_data = data;
}
spin_unlock_irqrestore(&vga_lock, flags);
if (!vgadev)
return -ENODEV;
@ -1464,7 +1469,7 @@ static void vga_arbiter_notify_clients(void)
spin_lock_irqsave(&vga_lock, flags);
list_for_each_entry(vgadev, &vga_list, list) {
if (vgadev->set_decode) {
new_decodes = vgadev->set_decode(vgadev->pdev,
new_decodes = vgadev->set_decode(vgadev->set_decode_data,
new_state);
vga_update_device_decodes(vgadev, new_decodes);
}

17
drivers/vfio/cdx/main.c
View File

@ -254,10 +254,10 @@ static int vfio_cdx_mmap_mmio(struct vfio_cdx_region region,
if (base + size > region.size)
return -EINVAL;
vma->vm_pgoff = (region.addr >> PAGE_SHIFT) + pgoff;
vma->vm_page_prot = pgprot_device(vma->vm_page_prot);
return io_remap_pfn_range(vma, vma->vm_start, vma->vm_pgoff,
return io_remap_pfn_range(vma, vma->vm_start,
(region.addr >> PAGE_SHIFT) + pgoff,
size, vma->vm_page_prot);
}
@ -271,6 +271,9 @@ static int vfio_cdx_mmap(struct vfio_device *core_vdev,
index = vma->vm_pgoff >> (VFIO_CDX_OFFSET_SHIFT - PAGE_SHIFT);
if (!(vma->vm_flags & VM_SHARED))
return -EINVAL;
if (index >= cdx_dev->res_count)
return -EINVAL;
@ -281,9 +284,13 @@ static int vfio_cdx_mmap(struct vfio_device *core_vdev,
(vma->vm_flags & VM_READ))
return -EPERM;
if (!(vdev->regions[index].flags & VFIO_REGION_INFO_FLAG_WRITE) &&
(vma->vm_flags & VM_WRITE))
return -EPERM;
/* Prevent read-only region mappings from being upgraded with mprotect() */
if (!(vdev->regions[index].flags & VFIO_REGION_INFO_FLAG_WRITE)) {
if (vma->vm_flags & VM_WRITE)
return -EPERM;
vm_flags_clear(vma, VM_MAYWRITE);
}
return vfio_cdx_mmap_mmio(vdev->regions[index], vma);
}

12
drivers/vfio/device_cdev.c
View File

@ -130,6 +130,18 @@ long vfio_df_ioctl_bind_iommufd(struct vfio_device_file *df,
goto out_unlock;
}
/*
* The cdev path allows only a single open. Reject a second open here,
* before the VF token and device->kvm updates below would clobber the
* current opener's state on a bind that cannot complete. Return -EBUSY
* rather than -EINVAL since a delayed release of the prior opener can
* make this transient.
*/
if (device->open_count) {
ret = -EBUSY;
goto out_unlock;
}
ret = vfio_df_check_token(device, &bind);
if (ret)
goto out_unlock;

15
drivers/vfio/fsl-mc/vfio_fsl_mc.c
View File

@ -369,9 +369,8 @@ static int vfio_fsl_mc_mmap_mmio(struct vfio_fsl_mc_region region,
if (!region_cacheable)
vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
vma->vm_pgoff = (region.addr >> PAGE_SHIFT) + pgoff;
return remap_pfn_range(vma, vma->vm_start, vma->vm_pgoff,
return remap_pfn_range(vma, vma->vm_start,
(region.addr >> PAGE_SHIFT) + pgoff,
size, vma->vm_page_prot);
}
@ -403,9 +402,13 @@ static int vfio_fsl_mc_mmap(struct vfio_device *core_vdev,
&& (vma->vm_flags & VM_READ))
return -EINVAL;
if (!(vdev->regions[index].flags & VFIO_REGION_INFO_FLAG_WRITE)
&& (vma->vm_flags & VM_WRITE))
return -EINVAL;
/* Prevent read-only region mappings from being upgraded with mprotect() */
if (!(vdev->regions[index].flags & VFIO_REGION_INFO_FLAG_WRITE)) {
if (vma->vm_flags & VM_WRITE)
return -EINVAL;
vm_flags_clear(vma, VM_MAYWRITE);
}
vma->vm_private_data = mc_dev;

2
drivers/vfio/mdev/mdev_core.c
View File

@ -50,7 +50,7 @@ static int mdev_device_remove_cb(struct device *dev, void *data)
* @types: Array of supported mdev types
* @nr_types: Number of entries in @types
*
* Registers the @parent stucture as a parent for mdev types and thus mdev
* Registers the @parent structure as a parent for mdev types and thus mdev
* devices. The caller needs to hold a reference on @dev that must not be
* released until after the call to mdev_unregister_parent().
*

73
drivers/vfio/pci/hisilicon/hisi_acc_vfio_pci.c
View File

@ -16,6 +16,9 @@
#include "hisi_acc_vfio_pci.h"
VFIO_PCI_CORE_DEFINE_CALLBACKS(hisi_acc_vf, struct hisi_acc_vf_core_device,
core_device)
/* Return 0 on VM acc device ready, -ETIMEDOUT hardware timeout */
static int qm_wait_dev_not_ready(struct hisi_qm *qm)
{
@ -588,10 +591,7 @@ static int vf_qm_state_save(struct hisi_acc_vf_core_device *hisi_acc_vdev,
static struct hisi_acc_vf_core_device *hisi_acc_drvdata(struct pci_dev *pdev)
{
struct vfio_pci_core_device *core_device = dev_get_drvdata(&pdev->dev);
return container_of(core_device, struct hisi_acc_vf_core_device,
core_device);
return pci_get_drvdata(pdev);
}
/* Check the PF's RAS state and Function INT state */
@ -1154,14 +1154,16 @@ static void hisi_acc_vf_pci_reset_prepare(struct pci_dev *pdev)
{
struct hisi_acc_vf_core_device *hisi_acc_vdev = hisi_acc_drvdata(pdev);
struct hisi_qm *qm = hisi_acc_vdev->pf_qm;
struct device *dev = &qm->pdev->dev;
u32 delay = 0;
if (!hisi_acc_vdev->core_device.vdev.mig_ops)
return;
/* All reset requests need to be queued for processing */
while (test_and_set_bit(QM_RESETTING, &qm->misc_ctl)) {
msleep(1);
if (++delay > QM_RESET_WAIT_TIMEOUT) {
dev_err(dev, "reset prepare failed\n");
dev_err(&qm->pdev->dev, "reset prepare failed\n");
return;
}
}
@ -1174,12 +1176,14 @@ static void hisi_acc_vf_pci_aer_reset_done(struct pci_dev *pdev)
struct hisi_acc_vf_core_device *hisi_acc_vdev = hisi_acc_drvdata(pdev);
struct hisi_qm *qm = hisi_acc_vdev->pf_qm;
if (hisi_acc_vdev->set_reset_flag)
clear_bit(QM_RESETTING, &qm->misc_ctl);
if (!hisi_acc_vdev->core_device.vdev.mig_ops)
return;
if (hisi_acc_vdev->set_reset_flag) {
clear_bit(QM_RESETTING, &qm->misc_ctl);
hisi_acc_vdev->set_reset_flag = false;
}
mutex_lock(&hisi_acc_vdev->state_mutex);
hisi_acc_vf_reset(hisi_acc_vdev);
mutex_unlock(&hisi_acc_vdev->state_mutex);
@ -1191,13 +1195,6 @@ static int hisi_acc_vf_qm_init(struct hisi_acc_vf_core_device *hisi_acc_vdev)
struct hisi_qm *vf_qm = &hisi_acc_vdev->vf_qm;
struct hisi_qm *pf_qm = hisi_acc_vdev->pf_qm;
struct pci_dev *vf_dev = vdev->pdev;
u32 val;
val = readl(pf_qm->io_base + QM_MIG_REGION_SEL);
if (pf_qm->ver > QM_HW_V3 && (val & QM_MIG_REGION_EN))
hisi_acc_vdev->drv_mode = HW_ACC_MIG_PF_CTRL;
else
hisi_acc_vdev->drv_mode = HW_ACC_MIG_VF_CTRL;
if (hisi_acc_vdev->drv_mode == HW_ACC_MIG_PF_CTRL) {
/*
@ -1416,9 +1413,8 @@ static int hisi_acc_vf_debug_check(struct seq_file *seq, struct vfio_device *vde
static int hisi_acc_vf_debug_cmd(struct seq_file *seq, void *data)
{
struct device *vf_dev = seq->private;
struct vfio_pci_core_device *core_device = dev_get_drvdata(vf_dev);
struct vfio_device *vdev = &core_device->vdev;
struct hisi_acc_vf_core_device *hisi_acc_vdev = hisi_acc_get_vf_dev(vdev);
struct hisi_acc_vf_core_device *hisi_acc_vdev = dev_get_drvdata(vf_dev);
struct vfio_device *vdev = &hisi_acc_vdev->core_device.vdev;
struct hisi_qm *vf_qm = &hisi_acc_vdev->vf_qm;
u64 value;
int ret;
@ -1445,9 +1441,8 @@ static int hisi_acc_vf_debug_cmd(struct seq_file *seq, void *data)
static int hisi_acc_vf_dev_read(struct seq_file *seq, void *data)
{
struct device *vf_dev = seq->private;
struct vfio_pci_core_device *core_device = dev_get_drvdata(vf_dev);
struct vfio_device *vdev = &core_device->vdev;
struct hisi_acc_vf_core_device *hisi_acc_vdev = hisi_acc_get_vf_dev(vdev);
struct hisi_acc_vf_core_device *hisi_acc_vdev = dev_get_drvdata(vf_dev);
struct vfio_device *vdev = &hisi_acc_vdev->core_device.vdev;
size_t vf_data_sz = offsetofend(struct acc_vf_data, padding);
struct acc_vf_data *vf_data;
int ret;
@ -1492,9 +1487,7 @@ static int hisi_acc_vf_dev_read(struct seq_file *seq, void *data)
static int hisi_acc_vf_migf_read(struct seq_file *seq, void *data)
{
struct device *vf_dev = seq->private;
struct vfio_pci_core_device *core_device = dev_get_drvdata(vf_dev);
struct vfio_device *vdev = &core_device->vdev;
struct hisi_acc_vf_core_device *hisi_acc_vdev = hisi_acc_get_vf_dev(vdev);
struct hisi_acc_vf_core_device *hisi_acc_vdev = dev_get_drvdata(vf_dev);
size_t vf_data_sz = offsetofend(struct acc_vf_data, padding);
struct hisi_acc_vf_migration_file *debug_migf = hisi_acc_vdev->debug_migf;
@ -1665,21 +1658,34 @@ static void hisi_acc_vf_debugfs_exit(struct hisi_acc_vf_core_device *hisi_acc_vd
hisi_acc_vdev->debug_migf = NULL;
}
static enum hw_drv_mode hisi_acc_vf_get_drv_mode(struct hisi_qm *pf_qm)
{
u32 val;
val = readl(pf_qm->io_base + QM_MIG_REGION_SEL);
if (pf_qm->ver > QM_HW_V3 && (val & QM_MIG_REGION_EN))
return HW_ACC_MIG_PF_CTRL;
return HW_ACC_MIG_VF_CTRL;
}
static int hisi_acc_vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)
{
struct hisi_acc_vf_core_device *hisi_acc_vdev;
const struct vfio_device_ops *ops = &hisi_acc_vfio_pci_ops;
enum hw_drv_mode drv_mode = HW_ACC_MIG_VF_CTRL;
resource_size_t func_len;
struct hisi_qm *pf_qm;
int vf_id;
int ret;
pf_qm = hisi_acc_get_pf_qm(pdev);
if (pf_qm && pf_qm->ver >= QM_HW_V3) {
vf_id = pci_iov_vf_id(pdev);
if (vf_id >= 0)
ops = &hisi_acc_vfio_pci_migrn_ops;
func_len = pci_resource_len(pdev, VFIO_PCI_BAR2_REGION_INDEX) >> 1;
drv_mode = hisi_acc_vf_get_drv_mode(pf_qm);
if (drv_mode == HW_ACC_MIG_VF_CTRL && func_len < PAGE_SIZE)
pci_warn(pdev, "migration not supported on 64KB pages with QM_HW_V3\n");
else
pci_warn(pdev, "migration support failed, continue with generic interface\n");
ops = &hisi_acc_vfio_pci_migrn_ops;
}
hisi_acc_vdev = vfio_alloc_device(hisi_acc_vf_core_device,
@ -1687,7 +1693,9 @@ static int hisi_acc_vfio_pci_probe(struct pci_dev *pdev, const struct pci_device
if (IS_ERR(hisi_acc_vdev))
return PTR_ERR(hisi_acc_vdev);
dev_set_drvdata(&pdev->dev, &hisi_acc_vdev->core_device);
hisi_acc_vdev->drv_mode = drv_mode;
dev_set_drvdata(&pdev->dev, hisi_acc_vdev);
ret = vfio_pci_core_register_device(&hisi_acc_vdev->core_device);
if (ret)
goto out_put_vdev;
@ -1721,7 +1729,7 @@ MODULE_DEVICE_TABLE(pci, hisi_acc_vfio_pci_table);
static const struct pci_error_handlers hisi_acc_vf_err_handlers = {
.reset_prepare = hisi_acc_vf_pci_reset_prepare,
.reset_done = hisi_acc_vf_pci_aer_reset_done,
.error_detected = vfio_pci_core_aer_err_detected,
.error_detected = hisi_acc_vf_aer_err_detected,
};
static struct pci_driver hisi_acc_vfio_pci_driver = {
@ -1729,6 +1737,7 @@ static struct pci_driver hisi_acc_vfio_pci_driver = {
.id_table = hisi_acc_vfio_pci_table,
.probe = hisi_acc_vfio_pci_probe,
.remove = hisi_acc_vfio_pci_remove,
.driver = { .pm = pm_ptr(&hisi_acc_vf_pm_ops) },
.err_handler = &hisi_acc_vf_err_handlers,
.driver_managed_dma = true,
};

19
drivers/vfio/pci/ism/main.c
View File

@ -47,6 +47,9 @@ struct ism_vfio_pci_core_device {
struct kmem_cache *store_block_cache;
};
VFIO_PCI_CORE_DEFINE_CALLBACKS(ism_vfio, struct ism_vfio_pci_core_device,
core_device)
static int ism_vfio_pci_open_device(struct vfio_device *core_vdev)
{
struct ism_vfio_pci_core_device *ivpcd;
@ -363,7 +366,7 @@ static int ism_vfio_pci_probe(struct pci_dev *pdev,
if (IS_ERR(ivpcd))
return PTR_ERR(ivpcd);
dev_set_drvdata(&pdev->dev, &ivpcd->core_device);
dev_set_drvdata(&pdev->dev, ivpcd);
ret = vfio_pci_core_register_device(&ivpcd->core_device);
if (ret)
@ -374,12 +377,7 @@ static int ism_vfio_pci_probe(struct pci_dev *pdev,
static void ism_vfio_pci_remove(struct pci_dev *pdev)
{
struct vfio_pci_core_device *core_device;
struct ism_vfio_pci_core_device *ivpcd;
core_device = dev_get_drvdata(&pdev->dev);
ivpcd = container_of(core_device, struct ism_vfio_pci_core_device,
core_device);
struct ism_vfio_pci_core_device *ivpcd = pci_get_drvdata(pdev);
vfio_pci_core_unregister_device(&ivpcd->core_device);
vfio_put_device(&ivpcd->core_device.vdev);
@ -392,12 +390,17 @@ static const struct pci_device_id ism_device_table[] = {
};
MODULE_DEVICE_TABLE(pci, ism_device_table);
static const struct pci_error_handlers ism_vfio_err_handlers = {
.error_detected = ism_vfio_aer_err_detected,
};
static struct pci_driver ism_vfio_pci_driver = {
.name = KBUILD_MODNAME,
.id_table = ism_device_table,
.probe = ism_vfio_pci_probe,
.remove = ism_vfio_pci_remove,
.err_handler = &vfio_pci_core_err_handlers,
.driver = { .pm = pm_ptr(&ism_vfio_pm_ops) },
.err_handler = &ism_vfio_err_handlers,
.driver_managed_dma = true,
};

2
drivers/vfio/pci/mlx5/cmd.c
View File

@ -1644,7 +1644,7 @@ set_report_output(u32 size, int index, struct mlx5_vhca_qp *qp,
int i;
buf_offset = index * qp->max_msg_size;
if (WARN_ON(buf_offset + size >= qp->recv_buf.npages * PAGE_SIZE ||
if (WARN_ON(buf_offset + size > qp->recv_buf.npages * PAGE_SIZE ||
(nent > qp->max_msg_size / entry_size)))
return;

13
drivers/vfio/pci/mlx5/main.c
View File

@ -21,6 +21,9 @@
#include "cmd.h"
VFIO_PCI_CORE_DEFINE_CALLBACKS(mlx5vf, struct mlx5vf_pci_core_device,
core_device)
/* Device specification max LOAD size */
#define MAX_LOAD_SIZE (BIT_ULL(__mlx5_bit_sz(load_vhca_state_in, size)) - 1)
@ -28,10 +31,7 @@
static struct mlx5vf_pci_core_device *mlx5vf_drvdata(struct pci_dev *pdev)
{
struct vfio_pci_core_device *core_device = dev_get_drvdata(&pdev->dev);
return container_of(core_device, struct mlx5vf_pci_core_device,
core_device);
return pci_get_drvdata(pdev);
}
static void mlx5vf_disable_fd(struct mlx5_vf_migration_file *migf)
@ -1415,7 +1415,7 @@ static int mlx5vf_pci_probe(struct pci_dev *pdev,
if (IS_ERR(mvdev))
return PTR_ERR(mvdev);
dev_set_drvdata(&pdev->dev, &mvdev->core_device);
dev_set_drvdata(&pdev->dev, mvdev);
ret = vfio_pci_core_register_device(&mvdev->core_device);
if (ret)
goto out_put_vdev;
@ -1443,7 +1443,7 @@ MODULE_DEVICE_TABLE(pci, mlx5vf_pci_table);
static const struct pci_error_handlers mlx5vf_err_handlers = {
.reset_done = mlx5vf_pci_aer_reset_done,
.error_detected = vfio_pci_core_aer_err_detected,
.error_detected = mlx5vf_aer_err_detected,
};
static struct pci_driver mlx5vf_pci_driver = {
@ -1451,6 +1451,7 @@ static struct pci_driver mlx5vf_pci_driver = {
.id_table = mlx5vf_pci_table,
.probe = mlx5vf_pci_probe,
.remove = mlx5vf_pci_remove,
.driver = { .pm = pm_ptr(&mlx5vf_pm_ops) },
.err_handler = &mlx5vf_err_handlers,
.driver_managed_dma = true,
};

42
drivers/vfio/pci/nvgrace-gpu/main.c
View File

@ -73,6 +73,9 @@ struct nvgrace_gpu_pci_core_device {
int cxl_dvsec;
};
VFIO_PCI_CORE_DEFINE_CALLBACKS(nvgrace_gpu, struct nvgrace_gpu_pci_core_device,
core_device)
static void nvgrace_gpu_init_fake_bar_emu_regs(struct vfio_device *core_vdev)
{
struct nvgrace_gpu_pci_core_device *nvdev =
@ -185,10 +188,8 @@ static int nvgrace_gpu_open_device(struct vfio_device *core_vdev)
if (ret)
return ret;
if (nvdev->usemem.memlength) {
nvgrace_gpu_init_fake_bar_emu_regs(core_vdev);
mutex_init(&nvdev->remap_lock);
}
nvgrace_gpu_init_fake_bar_emu_regs(core_vdev);
mutex_init(&nvdev->remap_lock);
/*
* GPU readiness is checked by reading the BAR0 registers.
@ -1083,6 +1084,21 @@ static const struct vfio_device_ops nvgrace_gpu_pci_ops = {
.detach_ioas = vfio_iommufd_physical_detach_ioas,
};
static int nvgrace_gpu_core_open_device(struct vfio_device *core_vdev)
{
struct vfio_pci_core_device *vdev =
container_of(core_vdev, struct vfio_pci_core_device, vdev);
int ret;
ret = vfio_pci_core_enable(vdev);
if (ret)
return ret;
vfio_pci_core_finish_enable(vdev);
return 0;
}
static const struct vfio_pci_device_ops nvgrace_gpu_pci_dev_core_ops = {
.get_dmabuf_phys = vfio_pci_core_get_dmabuf_phys,
};
@ -1091,7 +1107,7 @@ static const struct vfio_device_ops nvgrace_gpu_pci_core_ops = {
.name = "nvgrace-gpu-vfio-pci-core",
.init = vfio_pci_core_init_dev,
.release = vfio_pci_core_release_dev,
.open_device = nvgrace_gpu_open_device,
.open_device = nvgrace_gpu_core_open_device,
.close_device = vfio_pci_core_close_device,
.ioctl = vfio_pci_core_ioctl,
.get_region_info_caps = vfio_pci_ioctl_get_region_info,
@ -1341,7 +1357,7 @@ static int nvgrace_gpu_probe(struct pci_dev *pdev,
if (ret)
goto out_put_vdev;
dev_set_drvdata(&pdev->dev, &nvdev->core_device);
dev_set_drvdata(&pdev->dev, nvdev);
if (ops == &nvgrace_gpu_pci_ops) {
nvdev->has_mig_hw_bug = nvgrace_gpu_has_mig_hw_bug(pdev);
@ -1372,10 +1388,10 @@ static int nvgrace_gpu_probe(struct pci_dev *pdev,
static void nvgrace_gpu_remove(struct pci_dev *pdev)
{
struct vfio_pci_core_device *core_device = dev_get_drvdata(&pdev->dev);
struct nvgrace_gpu_pci_core_device *nvdev = pci_get_drvdata(pdev);
vfio_pci_core_unregister_device(core_device);
vfio_put_device(&core_device->vdev);
vfio_pci_core_unregister_device(&nvdev->core_device);
vfio_put_device(&nvdev->core_device.vdev);
}
static const struct pci_device_id nvgrace_gpu_vfio_pci_table[] = {
@ -1407,17 +1423,14 @@ MODULE_DEVICE_TABLE(pci, nvgrace_gpu_vfio_pci_table);
*/
static void nvgrace_gpu_vfio_pci_reset_done(struct pci_dev *pdev)
{
struct vfio_pci_core_device *core_device = dev_get_drvdata(&pdev->dev);
struct nvgrace_gpu_pci_core_device *nvdev =
container_of(core_device, struct nvgrace_gpu_pci_core_device,
core_device);
struct nvgrace_gpu_pci_core_device *nvdev = pci_get_drvdata(pdev);
nvdev->reset_done = true;
}
static const struct pci_error_handlers nvgrace_gpu_vfio_pci_err_handlers = {
.reset_done = nvgrace_gpu_vfio_pci_reset_done,
.error_detected = vfio_pci_core_aer_err_detected,
.error_detected = nvgrace_gpu_aer_err_detected,
};
static struct pci_driver nvgrace_gpu_vfio_pci_driver = {
@ -1425,6 +1438,7 @@ static struct pci_driver nvgrace_gpu_vfio_pci_driver = {
.id_table = nvgrace_gpu_vfio_pci_table,
.probe = nvgrace_gpu_probe,
.remove = nvgrace_gpu_remove,
.driver = { .pm = pm_ptr(&nvgrace_gpu_pm_ops) },
.err_handler = &nvgrace_gpu_vfio_pci_err_handlers,
.driver_managed_dma = true,
};

8
drivers/vfio/pci/pds/pci_drv.c
View File

@ -16,6 +16,9 @@
#include "pci_drv.h"
#include "cmds.h"
VFIO_PCI_CORE_DEFINE_CALLBACKS(pds_vfio, struct pds_vfio_pci_device,
vfio_coredev)
#define PDS_VFIO_DRV_DESCRIPTION "AMD/Pensando VFIO Device Driver"
#define PCI_VENDOR_ID_PENSANDO 0x1dd8
@ -118,7 +121,7 @@ static int pds_vfio_pci_probe(struct pci_dev *pdev,
if (IS_ERR(pds_vfio))
return PTR_ERR(pds_vfio);
dev_set_drvdata(&pdev->dev, &pds_vfio->vfio_coredev);
dev_set_drvdata(&pdev->dev, pds_vfio);
err = vfio_pci_core_register_device(&pds_vfio->vfio_coredev);
if (err)
@ -173,7 +176,7 @@ static void pds_vfio_pci_aer_reset_done(struct pci_dev *pdev)
static const struct pci_error_handlers pds_vfio_pci_err_handlers = {
.reset_done = pds_vfio_pci_aer_reset_done,
.error_detected = vfio_pci_core_aer_err_detected,
.error_detected = pds_vfio_aer_err_detected,
};
static struct pci_driver pds_vfio_pci_driver = {
@ -181,6 +184,7 @@ static struct pci_driver pds_vfio_pci_driver = {
.id_table = pds_vfio_pci_table,
.probe = pds_vfio_pci_probe,
.remove = pds_vfio_pci_remove,
.driver = { .pm = pm_ptr(&pds_vfio_pm_ops) },
.err_handler = &pds_vfio_pci_err_handlers,
.driver_managed_dma = true,
};

5
drivers/vfio/pci/pds/vfio_dev.c
View File

@ -20,10 +20,7 @@ struct device *pds_vfio_to_dev(struct pds_vfio_pci_device *pds_vfio)
struct pds_vfio_pci_device *pds_vfio_pci_drvdata(struct pci_dev *pdev)
{
struct vfio_pci_core_device *core_device = dev_get_drvdata(&pdev->dev);
return container_of(core_device, struct pds_vfio_pci_device,
vfio_coredev);
return pci_get_drvdata(pdev);
}
void pds_vfio_reset(struct pds_vfio_pci_device *pds_vfio,

12
drivers/vfio/pci/qat/main.c
View File

@ -55,6 +55,9 @@ struct qat_vf_core_device {
struct qat_vf_migration_file *saving_migf;
};
VFIO_PCI_CORE_DEFINE_CALLBACKS(qat_vf, struct qat_vf_core_device,
core_device)
static int qat_vf_pci_open_device(struct vfio_device *core_vdev)
{
struct qat_vf_core_device *qat_vdev =
@ -623,9 +626,7 @@ static const struct vfio_device_ops qat_vf_pci_ops = {
static struct qat_vf_core_device *qat_vf_drvdata(struct pci_dev *pdev)
{
struct vfio_pci_core_device *core_device = pci_get_drvdata(pdev);
return container_of(core_device, struct qat_vf_core_device, core_device);
return pci_get_drvdata(pdev);
}
static void qat_vf_pci_aer_reset_done(struct pci_dev *pdev)
@ -651,7 +652,7 @@ qat_vf_vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *id)
if (IS_ERR(qat_vdev))
return PTR_ERR(qat_vdev);
pci_set_drvdata(pdev, &qat_vdev->core_device);
pci_set_drvdata(pdev, qat_vdev);
ret = vfio_pci_core_register_device(&qat_vdev->core_device);
if (ret)
goto out_put_device;
@ -686,7 +687,7 @@ MODULE_DEVICE_TABLE(pci, qat_vf_vfio_pci_table);
static const struct pci_error_handlers qat_vf_err_handlers = {
.reset_done = qat_vf_pci_aer_reset_done,
.error_detected = vfio_pci_core_aer_err_detected,
.error_detected = qat_vf_aer_err_detected,
};
static struct pci_driver qat_vf_vfio_pci_driver = {
@ -694,6 +695,7 @@ static struct pci_driver qat_vf_vfio_pci_driver = {
.id_table = qat_vf_vfio_pci_table,
.probe = qat_vf_vfio_pci_probe,
.remove = qat_vf_vfio_pci_remove,
.driver = { .pm = pm_ptr(&qat_vf_pm_ops) },
.err_handler = &qat_vf_err_handlers,
.driver_managed_dma = true,
};

33
drivers/vfio/pci/vfio_pci.c
View File

@ -223,13 +223,44 @@ static const struct pci_device_id vfio_pci_table[] = {
MODULE_DEVICE_TABLE(pci, vfio_pci_table);
static int __maybe_unused vfio_pci_pm_suspend(struct device *dev)
{
struct vfio_pci_core_device *vdev = dev_get_drvdata(dev);
return vfio_pci_core_runtime_suspend(vdev);
}
static int __maybe_unused vfio_pci_pm_resume(struct device *dev)
{
struct vfio_pci_core_device *vdev = dev_get_drvdata(dev);
return vfio_pci_core_runtime_resume(vdev);
}
static const struct dev_pm_ops vfio_pci_pm_ops = {
SET_RUNTIME_PM_OPS(vfio_pci_pm_suspend, vfio_pci_pm_resume, NULL)
};
static pci_ers_result_t vfio_pci_aer_err_detected(struct pci_dev *pdev,
pci_channel_state_t state)
{
struct vfio_pci_core_device *vdev = pci_get_drvdata(pdev);
return vfio_pci_core_aer_err_detected(vdev);
}
static const struct pci_error_handlers vfio_pci_err_handlers = {
.error_detected = vfio_pci_aer_err_detected,
};
static struct pci_driver vfio_pci_driver = {
.name = "vfio-pci",
.id_table = vfio_pci_table,
.probe = vfio_pci_probe,
.remove = vfio_pci_remove,
.sriov_configure = vfio_pci_sriov_configure,
.err_handler = &vfio_pci_core_err_handlers,
.err_handler = &vfio_pci_err_handlers,
.driver = { .pm = pm_ptr(&vfio_pci_pm_ops) },
.driver_managed_dma = true,
};

46
drivers/vfio/pci/vfio_pci_core.c
View File

@ -162,9 +162,10 @@ static inline void vfio_pci_core_debugfs_init(struct vfio_pci_core_device *vdev)
* has no way to get to it and routing can be disabled externally at the
* bridge.
*/
static unsigned int vfio_pci_set_decode(struct pci_dev *pdev, bool single_vga)
static unsigned int vfio_pci_set_decode(void *data, bool single_vga)
{
struct vfio_pci_core_device *vdev = dev_get_drvdata(&pdev->dev);
struct vfio_pci_core_device *vdev = data;
struct pci_dev *pdev = vdev->pdev;
struct pci_dev *tmp = NULL;
unsigned char max_busnr;
unsigned int decodes;
@ -488,10 +489,8 @@ static int vfio_pci_core_pm_exit(struct vfio_pci_core_device *vdev, u32 flags,
}
#ifdef CONFIG_PM
static int vfio_pci_core_runtime_suspend(struct device *dev)
int vfio_pci_core_runtime_suspend(struct vfio_pci_core_device *vdev)
{
struct vfio_pci_core_device *vdev = dev_get_drvdata(dev);
down_write(&vdev->memory_lock);
/*
* The user can move the device into D3hot state before invoking
@ -516,11 +515,10 @@ static int vfio_pci_core_runtime_suspend(struct device *dev)
return 0;
}
EXPORT_SYMBOL_GPL(vfio_pci_core_runtime_suspend);
static int vfio_pci_core_runtime_resume(struct device *dev)
int vfio_pci_core_runtime_resume(struct vfio_pci_core_device *vdev)
{
struct vfio_pci_core_device *vdev = dev_get_drvdata(dev);
/*
* Resume with a pm_wake_eventfd_ctx signals the eventfd and exit
* low power mode.
@ -537,6 +535,7 @@ static int vfio_pci_core_runtime_resume(struct device *dev)
return 0;
}
EXPORT_SYMBOL_GPL(vfio_pci_core_runtime_resume);
#endif /* CONFIG_PM */
/*
@ -576,18 +575,6 @@ static void vfio_pci_core_map_bars(struct vfio_pci_core_device *vdev)
}
}
/*
* The pci-driver core runtime PM routines always save the device state
* before going into suspended state. If the device is going into low power
* state with only with runtime PM ops, then no explicit handling is needed
* for the devices which have NoSoftRst-.
*/
static const struct dev_pm_ops vfio_pci_core_pm_ops = {
SET_RUNTIME_PM_OPS(vfio_pci_core_runtime_suspend,
vfio_pci_core_runtime_resume,
NULL)
};
int vfio_pci_core_enable(struct vfio_pci_core_device *vdev)
{
struct pci_dev *pdev = vdev->pdev;
@ -2159,10 +2146,10 @@ static int vfio_pci_vga_init(struct vfio_pci_core_device *vdev)
if (ret)
return ret;
ret = vga_client_register(pdev, vfio_pci_set_decode);
ret = vga_client_register(pdev, vfio_pci_set_decode, vdev);
if (ret)
return ret;
vga_set_legacy_decoding(pdev, vfio_pci_set_decode(pdev, false));
vga_set_legacy_decoding(pdev, vfio_pci_set_decode(vdev, false));
return 0;
}
@ -2221,10 +2208,6 @@ int vfio_pci_core_register_device(struct vfio_pci_core_device *vdev)
struct device *dev = &pdev->dev;
int ret;
/* Drivers must set the vfio_pci_core_device to their drvdata */
if (WARN_ON(vdev != dev_get_drvdata(dev)))
return -EINVAL;
/* Drivers must set a name. Required for sequestering SR-IOV VFs */
if (WARN_ON(!vdev->vdev.ops->name))
return -EINVAL;
@ -2292,7 +2275,6 @@ int vfio_pci_core_register_device(struct vfio_pci_core_device *vdev)
*/
vfio_pci_set_power_state(vdev, PCI_D0);
dev->driver->pm = &vfio_pci_core_pm_ops;
pm_runtime_allow(dev);
if (!vdev->disable_idle_d3)
pm_runtime_put(dev);
@ -2333,10 +2315,8 @@ void vfio_pci_core_unregister_device(struct vfio_pci_core_device *vdev)
}
EXPORT_SYMBOL_GPL(vfio_pci_core_unregister_device);
pci_ers_result_t vfio_pci_core_aer_err_detected(struct pci_dev *pdev,
pci_channel_state_t state)
pci_ers_result_t vfio_pci_core_aer_err_detected(struct vfio_pci_core_device *vdev)
{
struct vfio_pci_core_device *vdev = dev_get_drvdata(&pdev->dev);
struct vfio_pci_eventfd *eventfd;
rcu_read_lock();
@ -2419,11 +2399,6 @@ int vfio_pci_core_sriov_configure(struct vfio_pci_core_device *vdev,
}
EXPORT_SYMBOL_GPL(vfio_pci_core_sriov_configure);
const struct pci_error_handlers vfio_pci_core_err_handlers = {
.error_detected = vfio_pci_core_aer_err_detected,
};
EXPORT_SYMBOL_GPL(vfio_pci_core_err_handlers);
static bool vfio_dev_in_groups(struct vfio_device *vdev,
struct vfio_pci_group_info *groups)
{
@ -2675,6 +2650,7 @@ static void vfio_pci_dev_set_try_reset(struct vfio_device_set *dev_set)
static void vfio_pci_core_cleanup(void)
{
vfio_pci_uninit_perm_bits();
rcu_barrier();
}
static int __init vfio_pci_core_init(void)

3
drivers/vfio/pci/vfio_pci_rdwr.c
View File

@ -423,9 +423,6 @@ int vfio_pci_ioeventfd(struct vfio_pci_core_device *vdev, loff_t offset,
pos >= vdev->msix_offset + vdev->msix_size))
return -EINVAL;
if (count == 8)
return -EINVAL;
io = vfio_pci_core_get_iomap(vdev, bar);
if (IS_ERR(io))
return PTR_ERR(io);

8
drivers/vfio/pci/virtio/main.c
View File

@ -18,6 +18,9 @@
#include "common.h"
VFIO_PCI_CORE_DEFINE_CALLBACKS(virtiovf, struct virtiovf_pci_core_device,
core_device)
static int virtiovf_pci_open_device(struct vfio_device *core_vdev)
{
struct virtiovf_pci_core_device *virtvdev = container_of(core_vdev,
@ -174,7 +177,7 @@ static int virtiovf_pci_probe(struct pci_dev *pdev,
if (sup_lm)
virtiovf_set_migratable(virtvdev);
dev_set_drvdata(&pdev->dev, &virtvdev->core_device);
dev_set_drvdata(&pdev->dev, virtvdev);
ret = vfio_pci_core_register_device(&virtvdev->core_device);
if (ret)
goto out;
@ -211,7 +214,7 @@ static void virtiovf_pci_aer_reset_done(struct pci_dev *pdev)
static const struct pci_error_handlers virtiovf_err_handlers = {
.reset_done = virtiovf_pci_aer_reset_done,
.error_detected = vfio_pci_core_aer_err_detected,
.error_detected = virtiovf_aer_err_detected,
};
static struct pci_driver virtiovf_pci_driver = {
@ -219,6 +222,7 @@ static struct pci_driver virtiovf_pci_driver = {
.id_table = virtiovf_pci_table,
.probe = virtiovf_pci_probe,
.remove = virtiovf_pci_remove,
.driver = { .pm = pm_ptr(&virtiovf_pm_ops) },
.err_handler = &virtiovf_err_handlers,
.driver_managed_dma = true,
};

13
drivers/vfio/pci/xe/main.c
View File

@ -38,6 +38,9 @@ struct xe_vfio_pci_core_device {
struct xe_vfio_pci_migration_file *migf;
};
VFIO_PCI_CORE_DEFINE_CALLBACKS(xe_vfio, struct xe_vfio_pci_core_device,
core_device)
#define xe_vdev_to_dev(xe_vdev) (&(xe_vdev)->core_device.pdev->dev)
static void xe_vfio_pci_disable_file(struct xe_vfio_pci_migration_file *migf)
@ -140,7 +143,7 @@ static void xe_vfio_pci_reset_done(struct pci_dev *pdev)
static const struct pci_error_handlers xe_vfio_pci_err_handlers = {
.reset_prepare = xe_vfio_pci_reset_prepare,
.reset_done = xe_vfio_pci_reset_done,
.error_detected = vfio_pci_core_aer_err_detected,
.error_detected = xe_vfio_aer_err_detected,
};
static int xe_vfio_pci_open_device(struct vfio_device *core_vdev)
@ -550,7 +553,7 @@ static int xe_vfio_pci_probe(struct pci_dev *pdev, const struct pci_device_id *i
if (IS_ERR(xe_vdev))
return PTR_ERR(xe_vdev);
dev_set_drvdata(&pdev->dev, &xe_vdev->core_device);
dev_set_drvdata(&pdev->dev, xe_vdev);
ret = vfio_pci_core_register_device(&xe_vdev->core_device);
if (ret) {
@ -574,9 +577,12 @@ static void xe_vfio_pci_remove(struct pci_dev *pdev)
}
static const struct pci_device_id xe_vfio_pci_table[] = {
INTEL_BMG_IDS(INTEL_PCI_VFIO_DEVICE),
INTEL_CRI_IDS(INTEL_PCI_VFIO_DEVICE),
INTEL_NVLS_IDS(INTEL_PCI_VFIO_DEVICE),
INTEL_NVLP_IDS(INTEL_PCI_VFIO_DEVICE),
INTEL_PTL_IDS(INTEL_PCI_VFIO_DEVICE),
INTEL_WCL_IDS(INTEL_PCI_VFIO_DEVICE),
INTEL_BMG_IDS(INTEL_PCI_VFIO_DEVICE),
{}
};
MODULE_DEVICE_TABLE(pci, xe_vfio_pci_table);
@ -586,6 +592,7 @@ static struct pci_driver xe_vfio_pci_driver = {
.id_table = xe_vfio_pci_table,
.probe = xe_vfio_pci_probe,
.remove = xe_vfio_pci_remove,
.driver = { .pm = pm_ptr(&xe_vfio_pm_ops) },
.err_handler = &xe_vfio_pci_err_handlers,
.driver_managed_dma = true,
};

15
drivers/vfio/platform/vfio_platform_common.c
View File

@ -559,9 +559,8 @@ static int vfio_platform_mmap_mmio(struct vfio_platform_region region,
return -EINVAL;
vma->vm_page_prot = pgprot_noncached(vma->vm_page_prot);
vma->vm_pgoff = (region.addr >> PAGE_SHIFT) + pgoff;
return remap_pfn_range(vma, vma->vm_start, vma->vm_pgoff,
return remap_pfn_range(vma, vma->vm_start,
(region.addr >> PAGE_SHIFT) + pgoff,
req_len, vma->vm_page_prot);
}
@ -591,9 +590,13 @@ int vfio_platform_mmap(struct vfio_device *core_vdev, struct vm_area_struct *vma
&& (vma->vm_flags & VM_READ))
return -EINVAL;
if (!(vdev->regions[index].flags & VFIO_REGION_INFO_FLAG_WRITE)
&& (vma->vm_flags & VM_WRITE))
return -EINVAL;
/* Prevent read-only region mappings from being upgraded with mprotect() */
if (!(vdev->regions[index].flags & VFIO_REGION_INFO_FLAG_WRITE)) {
if (vma->vm_flags & VM_WRITE)
return -EINVAL;
vm_flags_clear(vma, VM_MAYWRITE);
}
vma->vm_private_data = vdev;

12
drivers/vfio/vfio_iommu_type1.c
View File

@ -94,6 +94,7 @@ struct vfio_dma {
bool lock_cap; /* capable(CAP_IPC_LOCK) */
bool vaddr_invalid;
bool has_rsvd; /* has 1 or more rsvd pfns */
bool has_non_rsvd; /* has 1 or more !rsvd pfns */
struct task_struct *task;
struct rb_root pfn_list; /* Ex-user pinned pfn list */
unsigned long *bitmap;
@ -791,6 +792,7 @@ static long vfio_pin_pages_remote(struct vfio_dma *dma, unsigned long vaddr,
out:
dma->has_rsvd |= rsvd;
dma->has_non_rsvd |= !rsvd;
ret = vfio_lock_acct(dma, lock_acct, false);
unpin_out:
@ -821,11 +823,13 @@ static long vfio_unpin_pages_remote(struct vfio_dma *dma, dma_addr_t iova,
long unlocked = 0, locked = vpfn_pages(dma, iova, npage);
if (dma->has_rsvd) {
unsigned long i;
if (dma->has_non_rsvd) {
unsigned long i;
for (i = 0; i < npage; i++)
if (put_pfn(pfn++, dma->prot))
unlocked++;
for (i = 0; i < npage; i++)
if (put_pfn(pfn++, dma->prot))
unlocked++;
}
} else {
put_valid_unreserved_pfns(pfn, npage, dma->prot);
unlocked = npage;

7
drivers/vfio/vfio_main.c
View File

@ -574,13 +574,6 @@ int vfio_df_open(struct vfio_device_file *df)
lockdep_assert_held(&device->dev_set->lock);
/*
* Only the group path allows the device to be opened multiple
* times. The device cdev path doesn't have a secure way for it.
*/
if (device->open_count != 0 && !df->group)
return -EINVAL;
device->open_count++;
if (device->open_count == 1) {
ret = vfio_df_device_first_open(df);

10
include/linux/mlx5/cq.h
View File

@ -81,16 +81,6 @@ enum {
enum {
MLX5_CQE_OWNER_MASK = 1,
MLX5_CQE_REQ = 0,
MLX5_CQE_RESP_WR_IMM = 1,
MLX5_CQE_RESP_SEND = 2,
MLX5_CQE_RESP_SEND_IMM = 3,
MLX5_CQE_RESP_SEND_INV = 4,
MLX5_CQE_RESIZE_CQ = 5,
MLX5_CQE_SIG_ERR = 12,
MLX5_CQE_REQ_ERR = 13,
MLX5_CQE_RESP_ERR = 14,
MLX5_CQE_INVALID = 15,
};
enum {

231
include/linux/mlx5/device.h
View File

@ -36,6 +36,7 @@
#include <linux/types.h>
#include <rdma/ib_verbs.h>
#include <linux/mlx5/mlx5_ifc.h>
#include <linux/mlx5/mlx5_ifc_macros.h>
#include <linux/bitfield.h>
#if defined(__LITTLE_ENDIAN)
@ -46,122 +47,6 @@
#error Host endianness not defined
#endif
/* helper macros */
#define __mlx5_nullp(typ) ((struct mlx5_ifc_##typ##_bits *)0)
#define __mlx5_bit_sz(typ, fld) sizeof(__mlx5_nullp(typ)->fld)
#define __mlx5_bit_off(typ, fld) (offsetof(struct mlx5_ifc_##typ##_bits, fld))
#define __mlx5_16_off(typ, fld) (__mlx5_bit_off(typ, fld) / 16)
#define __mlx5_dw_off(typ, fld) (__mlx5_bit_off(typ, fld) / 32)
#define __mlx5_64_off(typ, fld) (__mlx5_bit_off(typ, fld) / 64)
#define __mlx5_16_bit_off(typ, fld) (16 - __mlx5_bit_sz(typ, fld) - (__mlx5_bit_off(typ, fld) & 0xf))
#define __mlx5_dw_bit_off(typ, fld) (32 - __mlx5_bit_sz(typ, fld) - (__mlx5_bit_off(typ, fld) & 0x1f))
#define __mlx5_mask(typ, fld) ((u32)((1ull << __mlx5_bit_sz(typ, fld)) - 1))
#define __mlx5_dw_mask(typ, fld) (__mlx5_mask(typ, fld) << __mlx5_dw_bit_off(typ, fld))
#define __mlx5_mask16(typ, fld) ((u16)((1ull << __mlx5_bit_sz(typ, fld)) - 1))
#define __mlx5_16_mask(typ, fld) (__mlx5_mask16(typ, fld) << __mlx5_16_bit_off(typ, fld))
#define __mlx5_st_sz_bits(typ) sizeof(struct mlx5_ifc_##typ##_bits)
#define MLX5_FLD_SZ_BYTES(typ, fld) (__mlx5_bit_sz(typ, fld) / 8)
#define MLX5_ST_SZ_BYTES(typ) (sizeof(struct mlx5_ifc_##typ##_bits) / 8)
#define MLX5_ST_SZ_DW(typ) (sizeof(struct mlx5_ifc_##typ##_bits) / 32)
#define MLX5_ST_SZ_QW(typ) (sizeof(struct mlx5_ifc_##typ##_bits) / 64)
#define MLX5_UN_SZ_BYTES(typ) (sizeof(union mlx5_ifc_##typ##_bits) / 8)
#define MLX5_UN_SZ_DW(typ) (sizeof(union mlx5_ifc_##typ##_bits) / 32)
#define MLX5_BYTE_OFF(typ, fld) (__mlx5_bit_off(typ, fld) / 8)
#define MLX5_ADDR_OF(typ, p, fld) ((void *)((u8 *)(p) + MLX5_BYTE_OFF(typ, fld)))
/* insert a value to a struct */
#define MLX5_SET(typ, p, fld, v) do { \
u32 _v = v; \
BUILD_BUG_ON(__mlx5_st_sz_bits(typ) % 32); \
*((__be32 *)(p) + __mlx5_dw_off(typ, fld)) = \
cpu_to_be32((be32_to_cpu(*((__be32 *)(p) + __mlx5_dw_off(typ, fld))) & \
(~__mlx5_dw_mask(typ, fld))) | (((_v) & __mlx5_mask(typ, fld)) \
<< __mlx5_dw_bit_off(typ, fld))); \
} while (0)
#define MLX5_ARRAY_SET(typ, p, fld, idx, v) do { \
BUILD_BUG_ON(__mlx5_bit_off(typ, fld) % 32); \
MLX5_SET(typ, p, fld[idx], v); \
} while (0)
#define MLX5_SET_TO_ONES(typ, p, fld) do { \
BUILD_BUG_ON(__mlx5_st_sz_bits(typ) % 32); \
*((__be32 *)(p) + __mlx5_dw_off(typ, fld)) = \
cpu_to_be32((be32_to_cpu(*((__be32 *)(p) + __mlx5_dw_off(typ, fld))) & \
(~__mlx5_dw_mask(typ, fld))) | ((__mlx5_mask(typ, fld)) \
<< __mlx5_dw_bit_off(typ, fld))); \
} while (0)
#define MLX5_GET(typ, p, fld) ((be32_to_cpu(*((__be32 *)(p) +\
__mlx5_dw_off(typ, fld))) >> __mlx5_dw_bit_off(typ, fld)) & \
__mlx5_mask(typ, fld))
#define MLX5_GET_PR(typ, p, fld) ({ \
u32 ___t = MLX5_GET(typ, p, fld); \
pr_debug(#fld " = 0x%x\n", ___t); \
___t; \
})
#define __MLX5_SET64(typ, p, fld, v) do { \
BUILD_BUG_ON(__mlx5_bit_sz(typ, fld) != 64); \
*((__be64 *)(p) + __mlx5_64_off(typ, fld)) = cpu_to_be64(v); \
} while (0)
#define MLX5_SET64(typ, p, fld, v) do { \
BUILD_BUG_ON(__mlx5_bit_off(typ, fld) % 64); \
__MLX5_SET64(typ, p, fld, v); \
} while (0)
#define MLX5_ARRAY_SET64(typ, p, fld, idx, v) do { \
BUILD_BUG_ON(__mlx5_bit_off(typ, fld) % 64); \
__MLX5_SET64(typ, p, fld[idx], v); \
} while (0)
#define MLX5_GET64(typ, p, fld) be64_to_cpu(*((__be64 *)(p) + __mlx5_64_off(typ, fld)))
#define MLX5_GET64_PR(typ, p, fld) ({ \
u64 ___t = MLX5_GET64(typ, p, fld); \
pr_debug(#fld " = 0x%llx\n", ___t); \
___t; \
})
#define MLX5_GET16(typ, p, fld) ((be16_to_cpu(*((__be16 *)(p) +\
__mlx5_16_off(typ, fld))) >> __mlx5_16_bit_off(typ, fld)) & \
__mlx5_mask16(typ, fld))
#define MLX5_SET16(typ, p, fld, v) do { \
u16 _v = v; \
BUILD_BUG_ON(__mlx5_st_sz_bits(typ) % 16); \
*((__be16 *)(p) + __mlx5_16_off(typ, fld)) = \
cpu_to_be16((be16_to_cpu(*((__be16 *)(p) + __mlx5_16_off(typ, fld))) & \
(~__mlx5_16_mask(typ, fld))) | (((_v) & __mlx5_mask16(typ, fld)) \
<< __mlx5_16_bit_off(typ, fld))); \
} while (0)
/* Big endian getters */
#define MLX5_GET64_BE(typ, p, fld) (*((__be64 *)(p) +\
__mlx5_64_off(typ, fld)))
#define MLX5_GET_BE(type_t, typ, p, fld) ({ \
type_t tmp; \
switch (sizeof(tmp)) { \
case sizeof(u8): \
tmp = (__force type_t)MLX5_GET(typ, p, fld); \
break; \
case sizeof(u16): \
tmp = (__force type_t)cpu_to_be16(MLX5_GET(typ, p, fld)); \
break; \
case sizeof(u32): \
tmp = (__force type_t)cpu_to_be32(MLX5_GET(typ, p, fld)); \
break; \
case sizeof(u64): \
tmp = (__force type_t)MLX5_GET64_BE(typ, p, fld); \
break; \
} \
tmp; \
})
enum mlx5_inline_modes {
MLX5_INLINE_MODE_NONE,
MLX5_INLINE_MODE_L2,
@ -172,7 +57,7 @@ enum mlx5_inline_modes {
enum {
MLX5_MAX_COMMANDS = 32,
MLX5_CMD_DATA_BLOCK_SIZE = 512,
MLX5_PCI_CMD_XPORT = 7,
MLX5_PCI_CMD_XPORT = MLX5_CMD_QUEUE_ENTRY_TYPE_PCIE_CMD_IF_TRANSPORT,
MLX5_MKEY_BSF_OCTO_SIZE = 4,
MLX5_MAX_PSVS = 4,
};
@ -308,63 +193,6 @@ enum {
MLX5_EVENT_QUEUE_TYPE_DCT = 6,
};
/* mlx5 components can subscribe to any one of these events via
* mlx5_eq_notifier_register API.
*/
enum mlx5_event {
/* Special value to subscribe to any event */
MLX5_EVENT_TYPE_NOTIFY_ANY = 0x0,
/* HW events enum start: comp events are not subscribable */
MLX5_EVENT_TYPE_COMP = 0x0,
/* HW Async events enum start: subscribable events */
MLX5_EVENT_TYPE_PATH_MIG = 0x01,
MLX5_EVENT_TYPE_COMM_EST = 0x02,
MLX5_EVENT_TYPE_SQ_DRAINED = 0x03,
MLX5_EVENT_TYPE_SRQ_LAST_WQE = 0x13,
MLX5_EVENT_TYPE_SRQ_RQ_LIMIT = 0x14,
MLX5_EVENT_TYPE_CQ_ERROR = 0x04,
MLX5_EVENT_TYPE_WQ_CATAS_ERROR = 0x05,
MLX5_EVENT_TYPE_PATH_MIG_FAILED = 0x07,
MLX5_EVENT_TYPE_WQ_INVAL_REQ_ERROR = 0x10,
MLX5_EVENT_TYPE_WQ_ACCESS_ERROR = 0x11,
MLX5_EVENT_TYPE_SRQ_CATAS_ERROR = 0x12,
MLX5_EVENT_TYPE_OBJECT_CHANGE = 0x27,
MLX5_EVENT_TYPE_INTERNAL_ERROR = 0x08,
MLX5_EVENT_TYPE_PORT_CHANGE = 0x09,
MLX5_EVENT_TYPE_GPIO_EVENT = 0x15,
MLX5_EVENT_TYPE_PORT_MODULE_EVENT = 0x16,
MLX5_EVENT_TYPE_TEMP_WARN_EVENT = 0x17,
MLX5_EVENT_TYPE_XRQ_ERROR = 0x18,
MLX5_EVENT_TYPE_REMOTE_CONFIG = 0x19,
MLX5_EVENT_TYPE_GENERAL_EVENT = 0x22,
MLX5_EVENT_TYPE_MONITOR_COUNTER = 0x24,
MLX5_EVENT_TYPE_PPS_EVENT = 0x25,
MLX5_EVENT_TYPE_DB_BF_CONGESTION = 0x1a,
MLX5_EVENT_TYPE_STALL_EVENT = 0x1b,
MLX5_EVENT_TYPE_CMD = 0x0a,
MLX5_EVENT_TYPE_PAGE_REQUEST = 0xb,
MLX5_EVENT_TYPE_PAGE_FAULT = 0xc,
MLX5_EVENT_TYPE_NIC_VPORT_CHANGE = 0xd,
MLX5_EVENT_TYPE_ESW_FUNCTIONS_CHANGED = 0xe,
MLX5_EVENT_TYPE_VHCA_STATE_CHANGE = 0xf,
MLX5_EVENT_TYPE_DCT_DRAINED = 0x1c,
MLX5_EVENT_TYPE_DCT_KEY_VIOLATION = 0x1d,
MLX5_EVENT_TYPE_FPGA_ERROR = 0x20,
MLX5_EVENT_TYPE_FPGA_QP_ERROR = 0x21,
MLX5_EVENT_TYPE_DEVICE_TRACER = 0x26,
MLX5_EVENT_TYPE_MAX = 0x100,
};
enum mlx5_driver_event {
MLX5_DRIVER_EVENT_TYPE_TRAP = 0,
MLX5_DRIVER_EVENT_UPLINK_NETDEV,
@ -420,22 +248,6 @@ enum {
};
enum {
MLX5_OPCODE_NOP = 0x00,
MLX5_OPCODE_SEND_INVAL = 0x01,
MLX5_OPCODE_RDMA_WRITE = 0x08,
MLX5_OPCODE_RDMA_WRITE_IMM = 0x09,
MLX5_OPCODE_SEND = 0x0a,
MLX5_OPCODE_SEND_IMM = 0x0b,
MLX5_OPCODE_LSO = 0x0e,
MLX5_OPCODE_RDMA_READ = 0x10,
MLX5_OPCODE_ATOMIC_CS = 0x11,
MLX5_OPCODE_ATOMIC_FA = 0x12,
MLX5_OPCODE_ATOMIC_MASKED_CS = 0x14,
MLX5_OPCODE_ATOMIC_MASKED_FA = 0x15,
MLX5_OPCODE_BIND_MW = 0x18,
MLX5_OPCODE_CONFIG_CMD = 0x1f,
MLX5_OPCODE_ENHANCED_MPSW = 0x29,
MLX5_RECV_OPCODE_RDMA_WRITE_IMM = 0x00,
MLX5_RECV_OPCODE_SEND = 0x01,
MLX5_RECV_OPCODE_SEND_IMM = 0x02,
@ -443,19 +255,6 @@ enum {
MLX5_CQE_OPCODE_ERROR = 0x1e,
MLX5_CQE_OPCODE_RESIZE = 0x16,
MLX5_OPCODE_SET_PSV = 0x20,
MLX5_OPCODE_GET_PSV = 0x21,
MLX5_OPCODE_CHECK_PSV = 0x22,
MLX5_OPCODE_DUMP = 0x23,
MLX5_OPCODE_RGET_PSV = 0x26,
MLX5_OPCODE_RCHECK_PSV = 0x27,
MLX5_OPCODE_UMR = 0x25,
MLX5_OPCODE_FLOW_TBL_ACCESS = 0x2c,
MLX5_OPCODE_ACCESS_ASO = 0x2d,
};
enum {
@ -1224,12 +1023,6 @@ enum mlx5_flex_parser_protos {
/* MLX5 DEV CAPs */
/* TODO: EAT.ME */
enum mlx5_cap_mode {
HCA_CAP_OPMOD_GET_MAX = 0,
HCA_CAP_OPMOD_GET_CUR = 1,
};
/* Any new cap addition must update mlx5_hca_caps_alloc() to allocate
* capability memory.
*/
@ -1507,26 +1300,6 @@ enum mlx5_qcam_feature_groups {
#define MLX5_CAP_PSP(mdev, cap)\
MLX5_GET(psp_cap, (mdev)->caps.hca[MLX5_CAP_PSP]->cur, cap)
enum {
MLX5_CMD_STAT_OK = 0x0,
MLX5_CMD_STAT_INT_ERR = 0x1,
MLX5_CMD_STAT_BAD_OP_ERR = 0x2,
MLX5_CMD_STAT_BAD_PARAM_ERR = 0x3,
MLX5_CMD_STAT_BAD_SYS_STATE_ERR = 0x4,
MLX5_CMD_STAT_BAD_RES_ERR = 0x5,
MLX5_CMD_STAT_RES_BUSY = 0x6,
MLX5_CMD_STAT_NOT_READY = 0x7,
MLX5_CMD_STAT_LIM_ERR = 0x8,
MLX5_CMD_STAT_BAD_RES_STATE_ERR = 0x9,
MLX5_CMD_STAT_IX_ERR = 0xa,
MLX5_CMD_STAT_NO_RES_ERR = 0xf,
MLX5_CMD_STAT_BAD_INP_LEN_ERR = 0x50,
MLX5_CMD_STAT_BAD_OUTP_LEN_ERR = 0x51,
MLX5_CMD_STAT_BAD_QP_STATE_ERR = 0x10,
MLX5_CMD_STAT_BAD_PKT_ERR = 0x30,
MLX5_CMD_STAT_BAD_SIZE_OUTS_CQES_ERR = 0x40,
};
enum {
MLX5_IEEE_802_3_COUNTERS_GROUP = 0x0,
MLX5_RFC_2863_COUNTERS_GROUP = 0x1,

178
include/linux/mlx5/mlx5_ifc.h
View File

@ -5986,6 +5986,108 @@ struct mlx5_ifc_cqe_error_syndrome_bits {
u8 syndrome[0x8];
};
struct mlx5_ifc_wqe_ctrl_seg_bits {
u8 opmod[0x8];
u8 wqe_index[0x10];
u8 opcode[0x8];
u8 qp_or_sq[0x18];
u8 reserved_at_38[0x2];
u8 ds[0x6];
u8 signature[0x8];
u8 reserved_at_48[0x10];
u8 fm[0x3];
u8 reserved_at_5b[0x1];
u8 ce[0x2];
u8 se[0x1];
u8 reserved_at_5f[0x1];
u8 imm[0x20];
};
/* Values for wqe_ctrl_seg_bits.opcode */
enum {
MLX5_OPCODE_NOP = 0x00,
MLX5_OPCODE_SEND_INVAL = 0x01,
MLX5_OPCODE_RDMA_WRITE = 0x08,
MLX5_OPCODE_RDMA_WRITE_IMM = 0x09,
MLX5_OPCODE_SEND = 0x0a,
MLX5_OPCODE_SEND_IMM = 0x0b,
MLX5_OPCODE_LSO = 0x0e,
MLX5_OPCODE_RDMA_READ = 0x10,
MLX5_OPCODE_ATOMIC_CS = 0x11,
MLX5_OPCODE_ATOMIC_FA = 0x12,
MLX5_OPCODE_ATOMIC_MASKED_CS = 0x14,
MLX5_OPCODE_ATOMIC_MASKED_FA = 0x15,
MLX5_OPCODE_BIND_MW = 0x18,
MLX5_OPCODE_CONFIG_CMD = 0x1f,
MLX5_OPCODE_SET_PSV = 0x20,
MLX5_OPCODE_GET_PSV = 0x21,
MLX5_OPCODE_CHECK_PSV = 0x22,
MLX5_OPCODE_DUMP = 0x23,
MLX5_OPCODE_UMR = 0x25,
MLX5_OPCODE_RGET_PSV = 0x26,
MLX5_OPCODE_RCHECK_PSV = 0x27,
MLX5_OPCODE_ENHANCED_MPSW = 0x29,
MLX5_OPCODE_FLOW_TBL_ACCESS = 0x2c,
MLX5_OPCODE_ACCESS_ASO = 0x2d,
};
/* Values for wqe_ctrl_seg_bits.ce */
enum {
MLX5_WQE_CE_CQE_ALWAYS = 2,
};
struct mlx5_ifc_wqe_raddr_seg_bits {
u8 raddr[0x40];
u8 rkey[0x20];
u8 reserved_at_60[0x20];
};
struct mlx5_ifc_wqe_data_seg_bits {
u8 reserved_at_0[0x1];
u8 byte_count[0x1f];
u8 lkey[0x20];
u8 addr[0x40];
};
struct mlx5_ifc_cqe64_bits {
u8 reserved_at_0[0x1a0];
union {
u8 reserved_at_1a0[0x20];
struct mlx5_ifc_cqe_error_syndrome_bits error_syndrome;
};
u8 send_wqe_opcode[0x8];
u8 qpn_or_dctn_or_flow_tag[0x18];
u8 wqe_counter[0x10];
u8 signature[0x8];
u8 opcode[0x4];
u8 cqe_format[0x2];
u8 se[0x1];
u8 owner[0x1];
};
/* Values for cqe64_bits.opcode */
enum {
MLX5_CQE_REQ = 0,
MLX5_CQE_RESP_WR_IMM = 1,
MLX5_CQE_RESP_SEND = 2,
MLX5_CQE_RESP_SEND_IMM = 3,
MLX5_CQE_RESP_SEND_INV = 4,
MLX5_CQE_RESIZE_CQ = 5,
MLX5_CQE_SIG_ERR = 12,
MLX5_CQE_REQ_ERR = 13,
MLX5_CQE_RESP_ERR = 14,
MLX5_CQE_INVALID = 15,
};
struct mlx5_ifc_qp_context_extension_bits {
u8 reserved_at_0[0x60];
@ -6482,6 +6584,12 @@ struct mlx5_ifc_query_hca_cap_in_bits {
u8 reserved_at_60[0x20];
};
/* Values for query_hca_cap_in_bits.op_mod */
enum mlx5_cap_mode {
HCA_CAP_OPMOD_GET_MAX = 0,
HCA_CAP_OPMOD_GET_CUR = 1,
};
struct mlx5_ifc_other_hca_cap_bits {
u8 roce[0x1];
u8 reserved_at_1[0x27f];
@ -11280,6 +11388,55 @@ struct mlx5_ifc_eqe_bits {
u8 owner[0x1];
};
/* Values for eqe_bits.event_type */
enum mlx5_event {
/* Special value to subscribe to any event */
MLX5_EVENT_TYPE_NOTIFY_ANY = 0x0,
/* HW events enum start: comp events are not subscribable */
MLX5_EVENT_TYPE_COMP = 0x0,
/* HW Async events enum start: subscribable events */
MLX5_EVENT_TYPE_PATH_MIG = 0x01,
MLX5_EVENT_TYPE_COMM_EST = 0x02,
MLX5_EVENT_TYPE_SQ_DRAINED = 0x03,
MLX5_EVENT_TYPE_SRQ_LAST_WQE = 0x13,
MLX5_EVENT_TYPE_SRQ_RQ_LIMIT = 0x14,
MLX5_EVENT_TYPE_CQ_ERROR = 0x04,
MLX5_EVENT_TYPE_WQ_CATAS_ERROR = 0x05,
MLX5_EVENT_TYPE_PATH_MIG_FAILED = 0x07,
MLX5_EVENT_TYPE_WQ_INVAL_REQ_ERROR = 0x10,
MLX5_EVENT_TYPE_WQ_ACCESS_ERROR = 0x11,
MLX5_EVENT_TYPE_SRQ_CATAS_ERROR = 0x12,
MLX5_EVENT_TYPE_OBJECT_CHANGE = 0x27,
MLX5_EVENT_TYPE_INTERNAL_ERROR = 0x08,
MLX5_EVENT_TYPE_PORT_CHANGE = 0x09,
MLX5_EVENT_TYPE_CMD = 0x0a,
MLX5_EVENT_TYPE_PAGE_REQUEST = 0x0b,
MLX5_EVENT_TYPE_PAGE_FAULT = 0x0c,
MLX5_EVENT_TYPE_NIC_VPORT_CHANGE = 0x0d,
MLX5_EVENT_TYPE_ESW_FUNCTIONS_CHANGED = 0x0e,
MLX5_EVENT_TYPE_VHCA_STATE_CHANGE = 0x0f,
MLX5_EVENT_TYPE_GPIO_EVENT = 0x15,
MLX5_EVENT_TYPE_PORT_MODULE_EVENT = 0x16,
MLX5_EVENT_TYPE_TEMP_WARN_EVENT = 0x17,
MLX5_EVENT_TYPE_XRQ_ERROR = 0x18,
MLX5_EVENT_TYPE_REMOTE_CONFIG = 0x19,
MLX5_EVENT_TYPE_DB_BF_CONGESTION = 0x1a,
MLX5_EVENT_TYPE_STALL_EVENT = 0x1b,
MLX5_EVENT_TYPE_DCT_DRAINED = 0x1c,
MLX5_EVENT_TYPE_DCT_KEY_VIOLATION = 0x1d,
MLX5_EVENT_TYPE_FPGA_ERROR = 0x20,
MLX5_EVENT_TYPE_FPGA_QP_ERROR = 0x21,
MLX5_EVENT_TYPE_GENERAL_EVENT = 0x22,
MLX5_EVENT_TYPE_MONITOR_COUNTER = 0x24,
MLX5_EVENT_TYPE_PPS_EVENT = 0x25,
MLX5_EVENT_TYPE_DEVICE_TRACER = 0x26,
MLX5_EVENT_TYPE_MAX = 0x100,
};
/* Values for cmd_queue_entry_bits.type */
enum {
MLX5_CMD_QUEUE_ENTRY_TYPE_PCIE_CMD_IF_TRANSPORT = 0x7,
};
@ -11322,6 +11479,27 @@ struct mlx5_ifc_cmd_out_bits {
u8 command_output[0x20];
};
/* Values for cmd_out_bits.status */
enum {
MLX5_CMD_STAT_OK = 0x0,
MLX5_CMD_STAT_INT_ERR = 0x1,
MLX5_CMD_STAT_BAD_OP_ERR = 0x2,
MLX5_CMD_STAT_BAD_PARAM_ERR = 0x3,
MLX5_CMD_STAT_BAD_SYS_STATE_ERR = 0x4,
MLX5_CMD_STAT_BAD_RES_ERR = 0x5,
MLX5_CMD_STAT_RES_BUSY = 0x6,
MLX5_CMD_STAT_NOT_READY = 0x7,
MLX5_CMD_STAT_LIM_ERR = 0x8,
MLX5_CMD_STAT_BAD_RES_STATE_ERR = 0x9,
MLX5_CMD_STAT_IX_ERR = 0xa,
MLX5_CMD_STAT_NO_RES_ERR = 0xf,
MLX5_CMD_STAT_BAD_QP_STATE_ERR = 0x10,
MLX5_CMD_STAT_BAD_PKT_ERR = 0x30,
MLX5_CMD_STAT_BAD_SIZE_OUTS_CQES_ERR = 0x40,
MLX5_CMD_STAT_BAD_INP_LEN_ERR = 0x50,
MLX5_CMD_STAT_BAD_OUTP_LEN_ERR = 0x51,
};
struct mlx5_ifc_cmd_in_bits {
u8 opcode[0x10];
u8 reserved_at_10[0x10];

185
include/linux/mlx5/mlx5_ifc_macros.h Normal file
View File

@ -0,0 +1,185 @@
/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
/*
* Copyright (c) 2013-2026, Mellanox Technologies. All rights reserved.
*
* Accessor macros for mlx5 IFC structures.
*
* Extracted from device.h so that code which cannot include device.h
* (e.g. selftests) can still use the MLX5_SET/GET family directly.
*/
#ifndef MLX5_IFC_MACROS_H
#define MLX5_IFC_MACROS_H
/* Internal helpers -- 32-bit */
#define __mlx5_nullp(typ) ((struct mlx5_ifc_##typ##_bits *)0)
#define __mlx5_bit_sz(typ, fld) sizeof(__mlx5_nullp(typ)->fld)
#define __mlx5_bit_off(typ, fld) (offsetof(struct mlx5_ifc_##typ##_bits, fld))
#define __mlx5_16_off(typ, fld) (__mlx5_bit_off(typ, fld) / 16)
#define __mlx5_dw_off(typ, fld) (__mlx5_bit_off(typ, fld) / 32)
#define __mlx5_64_off(typ, fld) (__mlx5_bit_off(typ, fld) / 64)
#define __mlx5_16_bit_off(typ, fld) (16 - __mlx5_bit_sz(typ, fld) - (__mlx5_bit_off(typ, fld) & 0xf))
#define __mlx5_dw_bit_off(typ, fld) (32 - __mlx5_bit_sz(typ, fld) - (__mlx5_bit_off(typ, fld) & 0x1f))
#define __mlx5_mask(typ, fld) ((u32)((1ull << __mlx5_bit_sz(typ, fld)) - 1))
#define __mlx5_dw_mask(typ, fld) (__mlx5_mask(typ, fld) << __mlx5_dw_bit_off(typ, fld))
#define __mlx5_mask16(typ, fld) ((u16)((1ull << __mlx5_bit_sz(typ, fld)) - 1))
#define __mlx5_16_mask(typ, fld) (__mlx5_mask16(typ, fld) << __mlx5_16_bit_off(typ, fld))
#define __mlx5_st_sz_bits(typ) sizeof(struct mlx5_ifc_##typ##_bits)
/* Size and address macros */
#define MLX5_FLD_SZ_BYTES(typ, fld) (__mlx5_bit_sz(typ, fld) / 8)
#define MLX5_ST_SZ_BYTES(typ) (sizeof(struct mlx5_ifc_##typ##_bits) / 8)
#define MLX5_ST_SZ_DW(typ) (sizeof(struct mlx5_ifc_##typ##_bits) / 32)
#define MLX5_ST_SZ_QW(typ) (sizeof(struct mlx5_ifc_##typ##_bits) / 64)
#define MLX5_UN_SZ_BYTES(typ) (sizeof(union mlx5_ifc_##typ##_bits) / 8)
#define MLX5_UN_SZ_DW(typ) (sizeof(union mlx5_ifc_##typ##_bits) / 32)
#define MLX5_BYTE_OFF(typ, fld) (__mlx5_bit_off(typ, fld) / 8)
#define MLX5_ADDR_OF(typ, p, fld) ((void *)((u8 *)(p) + MLX5_BYTE_OFF(typ, fld)))
/* insert a value to a struct */
#define MLX5_SET(typ, p, fld, v) do { \
u32 _v = v; \
BUILD_BUG_ON(__mlx5_st_sz_bits(typ) % 32); \
*((__be32 *)(p) + __mlx5_dw_off(typ, fld)) = \
cpu_to_be32((be32_to_cpu(*((__be32 *)(p) + __mlx5_dw_off(typ, fld))) & \
(~__mlx5_dw_mask(typ, fld))) | (((_v) & __mlx5_mask(typ, fld)) \
<< __mlx5_dw_bit_off(typ, fld))); \
} while (0)
#define MLX5_ARRAY_SET(typ, p, fld, idx, v) do { \
BUILD_BUG_ON(__mlx5_bit_off(typ, fld) % 32); \
MLX5_SET(typ, p, fld[idx], v); \
} while (0)
#define MLX5_SET_TO_ONES(typ, p, fld) do { \
BUILD_BUG_ON(__mlx5_st_sz_bits(typ) % 32); \
*((__be32 *)(p) + __mlx5_dw_off(typ, fld)) = \
cpu_to_be32((be32_to_cpu(*((__be32 *)(p) + __mlx5_dw_off(typ, fld))) & \
(~__mlx5_dw_mask(typ, fld))) | ((__mlx5_mask(typ, fld)) \
<< __mlx5_dw_bit_off(typ, fld))); \
} while (0)
#define MLX5_GET(typ, p, fld) ((be32_to_cpu(*((__be32 *)(p) +\
__mlx5_dw_off(typ, fld))) >> __mlx5_dw_bit_off(typ, fld)) & \
__mlx5_mask(typ, fld))
#define MLX5_GET_PR(typ, p, fld) ({ \
u32 ___t = MLX5_GET(typ, p, fld); \
pr_debug(#fld " = 0x%x\n", ___t); \
___t; \
})
/* 64-bit field accessors */
#define __MLX5_SET64(typ, p, fld, v) do { \
BUILD_BUG_ON(__mlx5_bit_sz(typ, fld) != 64); \
*((__be64 *)(p) + __mlx5_64_off(typ, fld)) = cpu_to_be64(v); \
} while (0)
#define MLX5_SET64(typ, p, fld, v) do { \
BUILD_BUG_ON(__mlx5_bit_off(typ, fld) % 64); \
__MLX5_SET64(typ, p, fld, v); \
} while (0)
#define MLX5_ARRAY_SET64(typ, p, fld, idx, v) do { \
BUILD_BUG_ON(__mlx5_bit_off(typ, fld) % 64); \
__MLX5_SET64(typ, p, fld[idx], v); \
} while (0)
#define MLX5_ARRAY_GET64(typ, p, fld, idx) \
({ \
BUILD_BUG_ON(__mlx5_bit_off(typ, fld) % 64); \
be64_to_cpu( \
*((__be64 *)(p) + __mlx5_64_off(typ, fld) + (idx))); \
})
#define MLX5_GET64(typ, p, fld) be64_to_cpu(*((__be64 *)(p) + __mlx5_64_off(typ, fld)))
#define MLX5_GET64_PR(typ, p, fld) ({ \
u64 ___t = MLX5_GET64(typ, p, fld); \
pr_debug(#fld " = 0x%llx\n", ___t); \
___t; \
})
/* 16-bit field accessors */
#define MLX5_GET16(typ, p, fld) ((be16_to_cpu(*((__be16 *)(p) +\
__mlx5_16_off(typ, fld))) >> __mlx5_16_bit_off(typ, fld)) & \
__mlx5_mask16(typ, fld))
#define MLX5_SET16(typ, p, fld, v) do { \
u16 _v = v; \
BUILD_BUG_ON(__mlx5_st_sz_bits(typ) % 16); \
*((__be16 *)(p) + __mlx5_16_off(typ, fld)) = \
cpu_to_be16((be16_to_cpu(*((__be16 *)(p) + __mlx5_16_off(typ, fld))) & \
(~__mlx5_16_mask(typ, fld))) | (((_v) & __mlx5_mask16(typ, fld)) \
<< __mlx5_16_bit_off(typ, fld))); \
} while (0)
/* Big endian getters */
#define MLX5_GET64_BE(typ, p, fld) (*((__be64 *)(p) +\
__mlx5_64_off(typ, fld)))
#define MLX5_GET_BE(type_t, typ, p, fld) ({ \
type_t tmp; \
switch (sizeof(tmp)) { \
case sizeof(u8): \
tmp = (__force type_t)MLX5_GET(typ, p, fld); \
break; \
case sizeof(u16): \
tmp = (__force type_t)cpu_to_be16(MLX5_GET(typ, p, fld)); \
break; \
case sizeof(u32): \
tmp = (__force type_t)cpu_to_be32(MLX5_GET(typ, p, fld)); \
break; \
case sizeof(u64): \
tmp = (__force type_t)MLX5_GET64_BE(typ, p, fld); \
break; \
} \
tmp; \
})
/*
* Use READ_ONCE/WRITE_ONCE for a single field that hardware may read/write
* unpredictably, mostly owner bits. All other bits in the DW must be stable.
* Usually a dma_wmb() will be required before a write and a dma_rmb() after a
* read.
*/
#define MLX5_GET_ONCE(typ, p, fld) \
((be32_to_cpu(READ_ONCE(*((__be32 *)(p) + __mlx5_dw_off(typ, fld)))) >> \
__mlx5_dw_bit_off(typ, fld)) & \
__mlx5_mask(typ, fld))
#define MLX5_SET_ONCE(typ, p, fld, v) \
do { \
u32 _v = v; \
__be32 *_dw = (__be32 *)(p) + __mlx5_dw_off(typ, fld); \
BUILD_BUG_ON(__mlx5_st_sz_bits(typ) % 32); \
WRITE_ONCE(*_dw, \
cpu_to_be32((be32_to_cpu(READ_ONCE(*_dw)) & \
(~__mlx5_dw_mask(typ, fld))) | \
(((_v) & __mlx5_mask(typ, fld)) \
<< __mlx5_dw_bit_off(typ, fld)))); \
} while (0)
/* Access MMIO registers, usually the init segment, using IFC structs. */
#define MLX5_GET_MMIO(typ, p, fld) \
((ioread32be(((__be32 __iomem *)(p) + __mlx5_dw_off(typ, fld))) >> \
__mlx5_dw_bit_off(typ, fld)) & \
__mlx5_mask(typ, fld))
/* The set is not relaxed so there is an integrated dma_wmb(). */
#define MLX5_SET_MMIO(typ, p, fld, v) \
do { \
u32 _v = v; \
void __iomem *_dw = \
((__be32 __iomem *)(p) + __mlx5_dw_off(typ, fld)); \
if (__mlx5_bit_sz(typ, fld) == 32) \
iowrite32be(_v, _dw); \
else \
iowrite32be((ioread32be(_dw) & \
(~__mlx5_dw_mask(typ, fld))) | \
((_v & __mlx5_mask(typ, fld)) \
<< __mlx5_dw_bit_off(typ, fld)), \
_dw); \
} while (0)
#endif /* MLX5_IFC_MACROS_H */

41
include/linux/vfio_pci_core.h
View File

@ -8,6 +8,7 @@
* Author: Tom Lyon, pugs@cisco.com
*/
#include <linux/device.h>
#include <linux/mutex.h>
#include <linux/pci.h>
#include <linux/vfio.h>
@ -16,6 +17,7 @@
#include <linux/types.h>
#include <linux/uuid.h>
#include <linux/notifier.h>
#include <linux/pm.h>
#ifndef VFIO_PCI_CORE_H
#define VFIO_PCI_CORE_H
@ -168,7 +170,6 @@ int vfio_pci_core_init_dev(struct vfio_device *core_vdev);
void vfio_pci_core_release_dev(struct vfio_device *core_vdev);
int vfio_pci_core_register_device(struct vfio_pci_core_device *vdev);
void vfio_pci_core_unregister_device(struct vfio_pci_core_device *vdev);
extern const struct pci_error_handlers vfio_pci_core_err_handlers;
int vfio_pci_core_sriov_configure(struct vfio_pci_core_device *vdev,
int nr_virtfn);
long vfio_pci_core_ioctl(struct vfio_device *core_vdev, unsigned int cmd,
@ -193,8 +194,6 @@ int vfio_pci_core_match_token_uuid(struct vfio_device *core_vdev,
int vfio_pci_core_enable(struct vfio_pci_core_device *vdev);
void vfio_pci_core_disable(struct vfio_pci_core_device *vdev);
void vfio_pci_core_finish_enable(struct vfio_pci_core_device *vdev);
pci_ers_result_t vfio_pci_core_aer_err_detected(struct pci_dev *pdev,
pci_channel_state_t state);
ssize_t vfio_pci_core_do_io_rw(struct vfio_pci_core_device *vdev, bool test_mem,
void __iomem *io, char __user *buf,
loff_t off, size_t count, size_t x_start,
@ -260,4 +259,40 @@ vfio_pci_core_get_iomap(struct vfio_pci_core_device *vdev, unsigned int bar)
int vfio_pci_dma_buf_iommufd_map(struct dma_buf_attachment *attachment,
struct phys_vec *phys);
int vfio_pci_core_runtime_suspend(struct vfio_pci_core_device *vdev);
int vfio_pci_core_runtime_resume(struct vfio_pci_core_device *vdev);
pci_ers_result_t vfio_pci_core_aer_err_detected(struct vfio_pci_core_device *vdev);
/*
* Per-driver PM/AER trampolines. driver_data points to the driver's private
* structure, which embeds a struct vfio_pci_core_device.
*
* @name: prefix for <name>_pm_ops and <name>_aer_err_detected
* @drv_type: type of the driver's private structure, including struct
* @member: member containing the struct vfio_pci_core_device
*/
#define VFIO_PCI_CORE_DEFINE_CALLBACKS(name, drv_type, member) \
static int __maybe_unused name##_pm_suspend(struct device *dev) \
{ \
drv_type *drvdata = dev_get_drvdata(dev); \
\
return vfio_pci_core_runtime_suspend(&drvdata->member); \
} \
static int __maybe_unused name##_pm_resume(struct device *dev) \
{ \
drv_type *drvdata = dev_get_drvdata(dev); \
\
return vfio_pci_core_runtime_resume(&drvdata->member); \
} \
static const struct dev_pm_ops name##_pm_ops = { \
SET_RUNTIME_PM_OPS(name##_pm_suspend, name##_pm_resume, NULL) \
}; \
static pci_ers_result_t name##_aer_err_detected(struct pci_dev *pdev, \
pci_channel_state_t state) \
{ \
drv_type *drvdata = pci_get_drvdata(pdev); \
\
return vfio_pci_core_aer_err_detected(&drvdata->member); \
}
#endif /* VFIO_PCI_CORE_H */

8
include/linux/vgaarb.h
View File

@ -33,7 +33,8 @@ struct pci_dev *vga_default_device(void);
void vga_set_default_device(struct pci_dev *pdev);
int vga_remove_vgacon(struct pci_dev *pdev);
int vga_client_register(struct pci_dev *pdev,
unsigned int (*set_decode)(struct pci_dev *pdev, bool state));
unsigned int (*set_decode)(void *data, bool state),
void *data);
#else /* CONFIG_VGA_ARB */
static inline void vga_set_legacy_decoding(struct pci_dev *pdev,
unsigned int decodes)
@ -59,7 +60,8 @@ static inline int vga_remove_vgacon(struct pci_dev *pdev)
return 0;
}
static inline int vga_client_register(struct pci_dev *pdev,
unsigned int (*set_decode)(struct pci_dev *pdev, bool state))
unsigned int (*set_decode)(void *data, bool state),
void *data)
{
return 0;
}
@ -97,7 +99,7 @@ static inline int vga_get_uninterruptible(struct pci_dev *pdev,
static inline void vga_client_unregister(struct pci_dev *pdev)
{
vga_client_register(pdev, NULL);
vga_client_register(pdev, NULL, NULL);
}
#endif /* LINUX_VGA_H */

4
tools/arch/arm64/include/asm/barrier.h
View File

@ -24,6 +24,10 @@
#define smp_wmb() asm volatile("dmb ishst" ::: "memory")
#define smp_rmb() asm volatile("dmb ishld" ::: "memory")
/* DMA barriers use outer-shareable (osh) for device visibility */
#define dma_rmb() asm volatile("dmb oshld" ::: "memory")
#define dma_wmb() asm volatile("dmb oshst" ::: "memory")
#define smp_store_release(p, v) \
do { \
union { typeof(*p) __val; char __c[1]; } __u = \

5
tools/arch/x86/include/asm/barrier.h
View File

@ -43,4 +43,9 @@ do { \
___p1; \
})
#endif /* defined(__x86_64__) */
/* x86 is DMA-coherent so DMA barriers are just compiler barriers */
#define dma_rmb() barrier()
#define dma_wmb() barrier()
#endif /* _TOOLS_LINUX_ASM_X86_BARRIER_H */

28
tools/include/asm-generic/io.h
View File

@ -479,4 +479,32 @@ static inline void writesq(volatile void __iomem *addr, const void *buffer,
}
#endif
/*
* ioread/iowrite for big-endian MMIO registers.
*/
#ifndef ioread32be
#define ioread32be ioread32be
static inline u32 ioread32be(const volatile void __iomem *addr)
{
return bswap_32(readl(addr));
}
#endif
#ifndef iowrite32be
#define iowrite32be iowrite32be
static inline void iowrite32be(u32 value, volatile void __iomem *addr)
{
writel(bswap_32(value), addr);
}
#endif
#ifndef iowrite64be
#define iowrite64be iowrite64be
static inline void iowrite64be(u64 value, volatile void __iomem *addr)
{
writeq(bswap_64(value), addr);
}
#endif
#endif /* _TOOLS_ASM_GENERIC_IO_H */

8
tools/include/asm/barrier.h
View File

@ -47,6 +47,14 @@
# define smp_mb() mb()
#endif
#ifndef dma_rmb
# define dma_rmb() rmb()
#endif
#ifndef dma_wmb
# define dma_wmb() wmb()
#endif
#ifndef smp_store_release
# define smp_store_release(p, v) \
do { \

10
tools/include/linux/stddef.h Normal file
View File

@ -0,0 +1,10 @@
/* SPDX-License-Identifier: GPL-2.0 */
#ifndef _TOOLS_LINUX_STDDEF_H
#define _TOOLS_LINUX_STDDEF_H
#include_next <linux/stddef.h>
#define DECLARE_FLEX_ARRAY(TYPE, NAME) \
__DECLARE_FLEX_ARRAY(TYPE, NAME)
#endif /* _TOOLS_LINUX_STDDEF_H */

4
tools/testing/selftests/kvm/include/arm64/processor.h
View File

@ -12,6 +12,7 @@
#include <linux/stringify.h>
#include <linux/types.h>
#include <asm/barrier.h>
#include <asm/brk-imm.h>
#include <asm/esr.h>
#include <asm/sysreg.h>
@ -191,11 +192,8 @@ static inline void cpu_relax(void)
#define dsb(opt) asm volatile("dsb " #opt : : : "memory")
#define dmb(opt) asm volatile("dmb " #opt : : : "memory")
#define dma_wmb() dmb(oshst)
#define __iowmb() dma_wmb()
#define dma_rmb() dmb(oshld)
#define __iormb(v) \
({ \
unsigned long tmp; \

2
tools/testing/selftests/kvm/irq_test.c
View File

@ -6,7 +6,6 @@
#include "proc_util.h"
#include <libvfio.h>
#include <linux/sizes.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
@ -101,7 +100,6 @@ static int vfio_setup_msi(struct vfio_pci_device *device)
allocator = iova_allocator_init(device->iommu);
region = &device->driver.region;
region->size = SZ_2M;
region->iova = iova_allocator_alloc(allocator, region->size);
region->vaddr = kvm_mmap(region->size, prot, flags, -1);
TEST_ASSERT(region->vaddr != MAP_FAILED, "mmap() failed\n");

1
tools/testing/selftests/vfio/.gitignore vendored
View File

@ -8,3 +8,4 @@
!*.mk
!.gitignore
!Makefile
!settings

1
tools/testing/selftests/vfio/lib/drivers/dsa/dsa.c
View File

@ -418,6 +418,7 @@ static void dsa_send_msi(struct vfio_pci_device *device)
const struct vfio_pci_driver_ops dsa_ops = {
.name = "dsa",
.region_size = sizeof(struct dsa_state),
.probe = dsa_probe,
.init = dsa_init,
.remove = dsa_remove,

1
tools/testing/selftests/vfio/lib/drivers/igb/igb.c
View File

@ -576,6 +576,7 @@ static void igb_send_msi(struct vfio_pci_device *device)
const struct vfio_pci_driver_ops igb_ops = {
.name = "igb",
.region_size = sizeof(struct igb),
.probe = igb_probe,
.init = igb_init,
.remove = igb_remove,

1
tools/testing/selftests/vfio/lib/drivers/ioat/ioat.c
View File

@ -226,6 +226,7 @@ static void ioat_send_msi(struct vfio_pci_device *device)
const struct vfio_pci_driver_ops ioat_ops = {
.name = "ioat",
.region_size = sizeof(struct ioat_state),
.probe = ioat_probe,
.init = ioat_init,
.remove = ioat_remove,

1928
tools/testing/selftests/vfio/lib/drivers/mlx5/mlx5.c Normal file
View File

File diff suppressed because it is too large Load Diff

114
tools/testing/selftests/vfio/lib/drivers/mlx5/mlx5_hw.h Normal file
View File

@ -0,0 +1,114 @@
/* SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB */
/*
* mlx5 VFIO selftest driver - HW definitions
*
* Typed wrappers, constants, and helpers for programming mlx5 hardware
* via the VFIO selftest framework. Most HW constants and all MLX5_SET/GET
* macros come from the kernel headers (mlx5_ifc.h, mlx5_ifc_macros.h).
*/
#ifndef SELFTESTS_VFIO_MLX5_HW_H
#define SELFTESTS_VFIO_MLX5_HW_H
#include <linux/io.h>
#include <linux/build_bug.h>
#include <vdso/bits.h>
#include "mlx5_ifc.h"
#include "mlx5_ifc_macros.h"
/*
* Typed HW object wrappers for driver region arrays.
*
* The IFC _bits structs have sizeof == num_bits (not bytes), so they cannot
* be used as array elements. These wrappers provide byte-sized types.
*/
#define MLX5ST_MAKE_DATA32(name) \
struct mlx5st_##name { \
u32 data[MLX5_ST_SZ_DW(name)]; \
}
#define MLX5ST_MAKE_DATA64(name) \
struct mlx5st_##name { \
u64 data[MLX5_ST_SZ_QW(name)]; \
}
MLX5ST_MAKE_DATA32(initial_seg);
MLX5ST_MAKE_DATA64(cmd_queue_entry);
MLX5ST_MAKE_DATA64(cmd_if_box);
MLX5ST_MAKE_DATA64(wqe_ctrl_seg);
MLX5ST_MAKE_DATA64(wqe_raddr_seg);
MLX5ST_MAKE_DATA64(wqe_data_seg);
MLX5ST_MAKE_DATA64(cqe64) __aligned(64);
MLX5ST_MAKE_DATA64(eqe);
/*
* Mailbox blocks: 512 data + 64 header = 576 bytes, but the
* next_pointer field stores bits [31:10], requiring 1024-byte alignment.
*/
#define CMD_MBOX_SIZE (2 * MLX5_HW_PAGE_SIZE)
#define CMD_MBOX_STRIDE 1024
#define CMD_MBOX_NENT (CMD_MBOX_SIZE / CMD_MBOX_STRIDE)
/* Stride-aligned mailbox entry — block + padding to 1024 bytes */
struct mlx5st_mbox_entry {
struct mlx5st_cmd_if_box block;
} __aligned(CMD_MBOX_STRIDE);
#define MLX5_CMD_INLINE_SZ \
MLX5_FLD_SZ_BYTES(cmd_queue_entry, command_input_inline_data)
/* Command interface mailbox block (512 data + 64 header) */
#define MLX5_CMD_DATA_BLOCK_SIZE MLX5_FLD_SZ_BYTES(cmd_if_box, mailbox_data)
/* RDMA Write WQE — one basic block: ctrl + raddr + data + padding */
struct mlx5st_send_wqe {
struct mlx5st_wqe_ctrl_seg ctrl;
struct mlx5st_wqe_raddr_seg raddr;
struct mlx5st_wqe_data_seg data;
} __aligned(64);
static_assert(sizeof(struct mlx5st_send_wqe) == 64,
"send WQE segments must fit in one BB");
/* DS = number of 16-byte segments in the WQE (ctrl + raddr + data) */
#define MLX5_RDMA_WRITE_DS 3
/* Doorbell record — two __be32 in a 64-byte aligned pair */
struct mlx5st_dbrec {
__be32 recv_counter;
__be32 send_counter;
} __aligned(64);
/* UAR BlueFlame buffer offsets within a UAR page */
#define MLX5_BF_OFFSET 0x800
#define MLX5_BF_SIZE 0x100
/* CQ doorbell offset within UAR page */
#define MLX5_CQ_DOORBELL_OFFSET 0x20
/* EQ doorbell offset within UAR page */
#define MLX5_EQ_DOORBELL_OFFSET 0x40
#define MLX5_HW_PAGE_SIZE 4096
/*
* Test parameters
*/
#define SQ_WQE_CNT 16
#define LOG_SQ_SIZE 4
#define CQ_CQE_CNT 16
#define LOG_CQ_SIZE 4
#define EQ_NENT 64
#define LOG_EQ_SIZE 6
#define MSI_EQ_NENT 16
#define LOG_MSI_EQ_SIZE 4
#define MSI_VECTOR 0
#define MAX_FW_PAGES 8192
#define MAX_FW_PAGES_PER_CMD 512
#define MLX5_CMD_TIMEOUT_MS 5000
static inline u32 mlx5st_idx_to_mkey(u32 mkey_idx)
{
return mkey_idx << 8;
}
#endif /* SELFTESTS_VFIO_MLX5_HW_H */

1
tools/testing/selftests/vfio/lib/drivers/mlx5/mlx5_ifc.h Symbolic link
View File

@ -0,0 +1 @@
../../../../../../../include/linux/mlx5/mlx5_ifc.h

1
tools/testing/selftests/vfio/lib/drivers/mlx5/mlx5_ifc_fpga.h Symbolic link
View File

@ -0,0 +1 @@
../../../../../../../include/linux/mlx5/mlx5_ifc_fpga.h

1
tools/testing/selftests/vfio/lib/drivers/mlx5/mlx5_ifc_macros.h Symbolic link
View File

@ -0,0 +1 @@
../../../../../../../include/linux/mlx5/mlx5_ifc_macros.h

1
tools/testing/selftests/vfio/lib/drivers/nv_falcon/nv_falcon.c
View File

@ -775,6 +775,7 @@ static int nv_falcon_memcpy_wait(struct vfio_pci_device *device)
const struct vfio_pci_driver_ops nv_falcon_ops = {
.name = "nv_falcon",
.region_size = sizeof(struct gpu_device),
.probe = nv_falcon_probe,
.init = nv_falcon_init,
.remove = nv_falcon_remove,

11
tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_device.h
View File

@ -40,6 +40,17 @@ struct vfio_pci_device {
struct vfio_pci_device *vfio_pci_device_alloc(const char *bdf, struct iommu *iommu);
void vfio_pci_device_free(struct vfio_pci_device *device);
#ifdef DEBUG
#define dev_dbg dev_info
#else
#define dev_dbg(_dev, _fmt, ...) \
do { \
if (0) \
dev_info(_dev, _fmt, ##__VA_ARGS__); \
} while (0)
#endif
struct vfio_pci_device *vfio_pci_device_init(const char *bdf, struct iommu *iommu);
void vfio_pci_device_cleanup(struct vfio_pci_device *device);

6
tools/testing/selftests/vfio/lib/include/libvfio/vfio_pci_driver.h
View File

@ -9,6 +9,12 @@ struct vfio_pci_device;
struct vfio_pci_driver_ops {
const char *name;
/*
* Size of the driver's state structure overlaid on
* device->driver.region.vaddr
*/
u64 region_size;
/**
* @probe() - Check if the driver supports the given device.
*

7
tools/testing/selftests/vfio/lib/iova_allocator.c
View File

@ -13,8 +13,10 @@
#include <linux/iommufd.h>
#include <linux/limits.h>
#include <linux/log2.h>
#include <linux/mman.h>
#include <linux/overflow.h>
#include <linux/sizes.h>
#include <linux/types.h>
#include <linux/vfio.h>
@ -50,7 +52,10 @@ void iova_allocator_cleanup(struct iova_allocator *allocator)
iova_t iova_allocator_alloc(struct iova_allocator *allocator, size_t size)
{
VFIO_ASSERT_GT(size, 0, "Invalid size arg, zero\n");
VFIO_ASSERT_EQ(size & (size - 1), 0, "Invalid size arg, non-power-of-2\n");
VFIO_ASSERT_LE(size, rounddown_pow_of_two(SIZE_MAX),
"Invalid size arg, too large (%zu)\n", size);
size = roundup_pow_of_two(size);
for (;;) {
struct iommu_iova_range *range;

1
tools/testing/selftests/vfio/lib/libvfio.mk
View File

@ -15,6 +15,7 @@ LIBVFIO_C += drivers/ioat/ioat.c
LIBVFIO_C += drivers/dsa/dsa.c
endif
LIBVFIO_C += drivers/mlx5/mlx5.c
LIBVFIO_C += drivers/nv_falcon/nv_falcon.c
LIBVFIO_C += drivers/igb/igb.c

1
tools/testing/selftests/vfio/lib/sysfs.c
View File

@ -1,5 +1,6 @@
// SPDX-License-Identifier: GPL-2.0-only
#include <fcntl.h>
#include <libgen.h>
#include <unistd.h>
#include <stdlib.h>
#include <string.h>

6
tools/testing/selftests/vfio/lib/vfio_pci_driver.c
View File

@ -6,6 +6,7 @@
extern struct vfio_pci_driver_ops dsa_ops;
extern struct vfio_pci_driver_ops ioat_ops;
#endif
extern struct vfio_pci_driver_ops mlx5st_ops;
extern struct vfio_pci_driver_ops nv_falcon_ops;
extern struct vfio_pci_driver_ops igb_ops;
@ -14,6 +15,7 @@ static struct vfio_pci_driver_ops *driver_ops[] = {
&dsa_ops,
&ioat_ops,
#endif
&mlx5st_ops,
&nv_falcon_ops,
&igb_ops,
};
@ -32,6 +34,10 @@ void vfio_pci_driver_probe(struct vfio_pci_device *device)
continue;
device->driver.ops = ops;
VFIO_ASSERT_NE(ops->region_size, 0);
device->driver.region.size =
round_up(ops->region_size, getpagesize());
}
}

5
tools/testing/selftests/vfio/settings Normal file
View File

@ -0,0 +1,5 @@
# Extend kselftest default 45s timeout based on worst case test runtime,
# currently vfio:vfio_pci_driver_test running on physical NIC, battling
# SMIs due to invoked firmware-first error handling from bad DMAs and
# device wedging. Observed worst case ~450s, extend to 600s.
timeout=600

5
tools/testing/selftests/vfio/vfio_pci_driver_test.c
View File

@ -87,7 +87,8 @@ FIXTURE_SETUP(vfio_pci_driver_test)
driver = &self->device->driver;
region_setup(self->iommu, self->iova_allocator, &self->memcpy_region, SZ_1G);
region_setup(self->iommu, self->iova_allocator, &driver->region, SZ_2M);
region_setup(self->iommu, self->iova_allocator, &driver->region,
driver->region.size);
/* Any IOVA that doesn't overlap memcpy_region and driver->region. */
self->unmapped_iova = iova_allocator_alloc(self->iova_allocator, SZ_1G);
@ -190,7 +191,7 @@ TEST_F(vfio_pci_driver_test, send_msi)
ASSERT_EQ(1, value);
}
TEST_F(vfio_pci_driver_test, mix_and_match)
TEST_F_TIMEOUT(vfio_pci_driver_test, mix_and_match, 90)
{
u64 value;
int i;

35
tools/testing/selftests/vfio/vfio_pci_sriov_uapi_test.c
View File

@ -157,6 +157,41 @@ TEST_F(vfio_pci_sriov_uapi_test, override_token)
ASSERT_COND_VF_CREATION(ret);
}
TEST(failed_second_open_does_not_clobber_token)
{
struct vfio_pci_device *pf = NULL, *pf_second_fd = NULL, *vf = NULL;
struct iommu *iommu;
int ret;
iommu = iommu_init(MODE_IOMMUFD);
/* Create and bind PF using UUID_1 */
ret = device_init(pf_bdf, iommu, UUID_1, &pf);
ASSERT_EQ(ret, 0);
/*
* Attempt to open the same PF again and bind it with a *different*
* token (UUID_2). This must fail with -EBUSY because the cdev path
* only supports a single open per device. Enforce it with EXPECT_EQ()
* so the clobber assertion below still runs if the errno differs.
*/
ret = device_init(pf_bdf, iommu, UUID_2, &pf_second_fd);
EXPECT_EQ(ret, -EBUSY);
/*
* Attempt to initialize a VF using the original PF token (UUID_1).
* If the failed open above clobbered the PF's token (i.e. updated it to
* UUID_2), this VF initialization will fail.
*/
ret = device_init(vf_bdf, iommu, UUID_1, &vf);
ASSERT_EQ(ret, 0);
device_cleanup(vf);
device_cleanup(pf_second_fd);
device_cleanup(pf);
iommu_cleanup(iommu);
}
static void vf_teardown(void)
{
/*