github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-06-07 14:04:54 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph

Merge 818dbde78e ("Merge tag 'scsi-misc' of git://git.kernel.org/pub/scm/linux/kernel/git/jejb/scsi") into android-mainline

Browse Source 
Steps on the way to 5.8-rc1

Signed-off-by: Greg Kroah-Hartman <gregkh@google.com>
Change-Id: I3beac21380f95fa9eaa9eb4c708c47c9c6addf11
This commit is contained in:
Greg Kroah-Hartman 2020-06-24 07:11:16 +02:00
commit b378328f2a
449 changed files with 33734 additions and 15388 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

46
Documentation/ABI/testing/sysfs-block-rnbd Normal file
View File

@ -0,0 +1,46 @@
What: /sys/block/rnbd<N>/rnbd/unmap_device
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: To unmap a volume, "normal" or "force" has to be written to:
/sys/block/rnbd<N>/rnbd/unmap_device
When "normal" is used, the operation will fail with EBUSY if any process
is using the device. When "force" is used, the device is also unmapped
when device is in use. All I/Os that are in progress will fail.
Example:
# echo "normal" > /sys/block/rnbd0/rnbd/unmap_device
What: /sys/block/rnbd<N>/rnbd/state
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: The file contains the current state of the block device. The state file
returns "open" when the device is successfully mapped from the server
and accepting I/O requests. When the connection to the server gets
disconnected in case of an error (e.g. link failure), the state file
returns "closed" and all I/O requests submitted to it will fail with -EIO.
What: /sys/block/rnbd<N>/rnbd/session
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: RNBD uses RTRS session to transport the data between client and
server. The entry "session" contains the name of the session, that
was used to establish the RTRS session. It's the same name that
was passed as server parameter to the map_device entry.
What: /sys/block/rnbd<N>/rnbd/mapping_path
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: Contains the path that was passed as "device_path" to the map_device
operation.
What: /sys/block/rnbd<N>/rnbd/access_mode
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: Contains the device access mode: ro, rw or migration.

111
Documentation/ABI/testing/sysfs-class-rnbd-client Normal file
View File

@ -0,0 +1,111 @@
What: /sys/class/rnbd-client
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: Provide information about RNBD-client.
All sysfs files that are not read-only provide the usage information on read:
Example:
# cat /sys/class/rnbd-client/ctl/map_device
> Usage: echo "sessname=<name of the rtrs session> path=<[srcaddr,]dstaddr>
> [path=<[srcaddr,]dstaddr>] device_path=<full path on remote side>
> [access_mode=<ro|rw|migration>] > map_device
>
> addr ::= [ ip:<ipv4> | ip:<ipv6> | gid:<gid> ]
What: /sys/class/rnbd-client/ctl/map_device
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: Expected format is the following:
sessname=<name of the rtrs session>
path=<[srcaddr,]dstaddr> [path=<[srcaddr,]dstaddr> ...]
device_path=<full path on remote side>
[access_mode=<ro|rw|migration>]
Where:
sessname: accepts a string not bigger than 256 chars, which identifies
a given session on the client and on the server.
I.e. "clt_hostname-srv_hostname" could be a natural choice.
path: describes a connection between the client and the server by
specifying destination and, when required, the source address.
The addresses are to be provided in the following format:
ip:<IPv6>
ip:<IPv4>
gid:<GID>
for example:
path=ip:10.0.0.66
The single addr is treated as the destination.
The connection will be established to this server from any client IP address.
path=ip:10.0.0.66,ip:10.0.1.66
First addr is the source address and the second is the destination.
If multiple "path=" options are specified multiple connection
will be established and data will be sent according to
the selected multipath policy (see RTRS mp_policy sysfs entry description).
device_path: Path to the block device on the server side. Path is specified
relative to the directory on server side configured in the
'dev_search_path' module parameter of the rnbd_server.
The rnbd_server prepends the <device_path> received from client
with <dev_search_path> and tries to open the
<dev_search_path>/<device_path> block device. On success,
a /dev/rnbd<N> device file, a /sys/block/rnbd_client/rnbd<N>/
directory and an entry in /sys/class/rnbd-client/ctl/devices
will be created.
If 'dev_search_path' contains '%SESSNAME%', then each session can
have different devices namespace, e.g. server was configured with
the following parameter "dev_search_path=/run/rnbd-devs/%SESSNAME%",
client has this string "sessname=blya device_path=sda", then server
will try to open: /run/rnbd-devs/blya/sda.
access_mode: the access_mode parameter specifies if the device is to be
mapped as "ro" read-only or "rw" read-write. The server allows
a device to be exported in rw mode only once. The "migration"
access mode has to be specified if a second mapping in read-write
mode is desired.
By default "rw" is used.
Exit Codes:
If the device is already mapped it will fail with EEXIST. If the input
has an invalid format it will return EINVAL. If the device path cannot
be found on the server, it will fail with ENOENT.
Finding device file after mapping
---------------------------------
After mapping, the device file can be found by:
o The symlink /sys/class/rnbd-client/ctl/devices/<device_id>
points to /sys/block/<dev-name>. The last part of the symlink destination
is the same as the device name. By extracting the last part of the
path the path to the device /dev/<dev-name> can be build.
o /dev/block/$(cat /sys/class/rnbd-client/ctl/devices/<device_id>/dev)
How to find the <device_id> of the device is described on the next
section.
What: /sys/class/rnbd-client/ctl/devices/
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: For each device mapped on the client a new symbolic link is created as
/sys/class/rnbd-client/ctl/devices/<device_id>, which points
to the block device created by rnbd (/sys/block/rnbd<N>/).
The <device_id> of each device is created as follows:
- If the 'device_path' provided during mapping contains slashes ("/"),
they are replaced by exclamation mark ("!") and used as as the
<device_id>. Otherwise, the <device_id> will be the same as the
"device_path" provided.

50
Documentation/ABI/testing/sysfs-class-rnbd-server Normal file
View File

@ -0,0 +1,50 @@
What: /sys/class/rnbd-server
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: provide information about RNBD-server.
What: /sys/class/rnbd-server/ctl/
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: When a client maps a device, a directory entry with the name of the
block device is created under /sys/class/rnbd-server/ctl/devices/.
What: /sys/class/rnbd-server/ctl/devices/<device_name>/block_dev
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: Is a symlink to the sysfs entry of the exported device.
Example:
block_dev -> ../../../../class/block/ram0
What: /sys/class/rnbd-server/ctl/devices/<device_name>/sessions/
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: For each client a particular device is exported to, following directory will be
created:
/sys/class/rnbd-server/ctl/devices/<device_name>/sessions/<session-name>/
When the device is unmapped by that client, the directory will be removed.
What: /sys/class/rnbd-server/ctl/devices/<device_name>/sessions/<session-name>/read_only
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: Contains '1' if device is mapped read-only, otherwise '0'.
What: /sys/class/rnbd-server/ctl/devices/<device_name>/sessions/<session-name>/mapping_path
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: Contains the relative device path provided by the user during mapping.
What: /sys/class/rnbd-server/ctl/devices/<device_name>/sessions/<session-name>/access_mode
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: Contains the device access mode: ro, rw or migration.

131
Documentation/ABI/testing/sysfs-class-rtrs-client Normal file
View File

@ -0,0 +1,131 @@
What: /sys/class/rtrs-client
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: When a user of RTRS API creates a new session, a directory entry with
the name of that session is created under /sys/class/rtrs-client/<session-name>/
What: /sys/class/rtrs-client/<session-name>/add_path
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: RW, adds a new path (connection) to an existing session. Expected format is the
following:
<[source addr,]destination addr>
*addr ::= [ ip:<ipv4|ipv6> | gid:<gid> ]
What: /sys/class/rtrs-client/<session-name>/max_reconnect_attempts
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: Maximum number reconnect attempts the client should make before giving up
after connection breaks unexpectedly.
What: /sys/class/rtrs-client/<session-name>/mp_policy
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: Multipath policy specifies which path should be selected on each IO:
round-robin (0):
select path in per CPU round-robin manner.
min-inflight (1):
select path with minimum inflights.
What: /sys/class/rtrs-client/<session-name>/paths/
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: Each path belonging to a given session is listed here by its source and
destination address. When a new path is added to a session by writing to
the "add_path" entry, a directory <src@dst> is created.
What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/state
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: RO, Contains "connected" if the session is connected to the peer and fully
functional. Otherwise the file contains "disconnected"
What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/reconnect
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: Write "1" to the file in order to reconnect the path.
Operation is blocking and returns 0 if reconnect was successful.
What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/disconnect
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: Write "1" to the file in order to disconnect the path.
Operation blocks until RTRS path is disconnected.
What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/remove_path
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: Write "1" to the file in order to disconnected and remove the path
from the session. Operation blocks until the path is disconnected
and removed from the session.
What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/hca_name
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: RO, Contains the the name of HCA the connection established on.
What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/hca_port
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: RO, Contains the port number of active port traffic is going through.
What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/src_addr
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: RO, Contains the source address of the path
What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/dst_addr
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: RO, Contains the destination address of the path
What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/reset_all
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: RW, Read will return usage help, write 0 will clear all the statistics.
What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/cpu_migration
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: RTRS expects that each HCA IRQ is pinned to a separate CPU. If it's
not the case, the processing of an I/O response could be processed on a
different CPU than where it was originally submitted. This file shows
how many interrupts where generated on a non expected CPU.
"from:" is the CPU on which the IRQ was expected, but not generated.
"to:" is the CPU on which the IRQ was generated, but not expected.
What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/reconnects
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: Contains 2 unsigned int values, the first one records number of successful
reconnects in the path lifetime, the second one records number of failed
reconnects in the path lifetime.
What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/rdma
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: Contains statistics regarding rdma operations and inflight operations.
The output consists of 6 values:
<read-count> <read-total-size> <write-count> <write-total-size> \
<inflights> <failovered>

53
Documentation/ABI/testing/sysfs-class-rtrs-server Normal file
View File

@ -0,0 +1,53 @@
What: /sys/class/rtrs-server
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: When a user of RTRS API creates a new session on a client side, a
directory entry with the name of that session is created in here.
What: /sys/class/rtrs-server/<session-name>/paths/
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: When new path is created by writing to "add_path" entry on client side,
a directory entry named as <source address>@<destination address> is created
on server.
What: /sys/class/rtrs-server/<session-name>/paths/<src@dst>/disconnect
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: When "1" is written to the file, the RTRS session is being disconnected.
Operations is non-blocking and returns control immediately to the caller.
What: /sys/class/rtrs-server/<session-name>/paths/<src@dst>/hca_name
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: RO, Contains the the name of HCA the connection established on.
What: /sys/class/rtrs-server/<session-name>/paths/<src@dst>/hca_port
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: RO, Contains the port number of active port traffic is going through.
What: /sys/class/rtrs-server/<session-name>/paths/<src@dst>/src_addr
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: RO, Contains the source address of the path
What: /sys/class/rtrs-server/<session-name>/paths/<src@dst>/dst_addr
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: RO, Contains the destination address of the path
What: /sys/class/rtrs-server/<session-name>/paths/<src@dst>/stats/rdma
Date: Feb 2020
KernelVersion: 5.7
Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
Description: Contains statistics regarding rdma operations and inflight operations.
The output consists of 5 values:
<read-count> <read-total-size> <write-count> <write-total-size> <inflights>

111
Documentation/admin-guide/gpio/gpio-aggregator.rst Normal file
View File

@ -0,0 +1,111 @@
.. SPDX-License-Identifier: GPL-2.0-only
GPIO Aggregator
===============
The GPIO Aggregator provides a mechanism to aggregate GPIOs, and expose them as
a new gpio_chip. This supports the following use cases.
Aggregating GPIOs using Sysfs
-----------------------------
GPIO controllers are exported to userspace using /dev/gpiochip* character
devices. Access control to these devices is provided by standard UNIX file
system permissions, on an all-or-nothing basis: either a GPIO controller is
accessible for a user, or it is not.
The GPIO Aggregator provides access control for a set of one or more GPIOs, by
aggregating them into a new gpio_chip, which can be assigned to a group or user
using standard UNIX file ownership and permissions. Furthermore, this
simplifies and hardens exporting GPIOs to a virtual machine, as the VM can just
grab the full GPIO controller, and no longer needs to care about which GPIOs to
grab and which not, reducing the attack surface.
Aggregated GPIO controllers are instantiated and destroyed by writing to
write-only attribute files in sysfs.
/sys/bus/platform/drivers/gpio-aggregator/
"new_device" ...
Userspace may ask the kernel to instantiate an aggregated GPIO
controller by writing a string describing the GPIOs to
aggregate to the "new_device" file, using the format
.. code-block:: none
[<gpioA>] [<gpiochipB> <offsets>] ...
Where:
"<gpioA>" ...
is a GPIO line name,
"<gpiochipB>" ...
is a GPIO chip label, and
"<offsets>" ...
is a comma-separated list of GPIO offsets and/or
GPIO offset ranges denoted by dashes.
Example: Instantiate a new GPIO aggregator by aggregating GPIO
line 19 of "e6052000.gpio" and GPIO lines 20-21 of
"e6050000.gpio" into a new gpio_chip:
.. code-block:: sh
$ echo 'e6052000.gpio 19 e6050000.gpio 20-21' > new_device
"delete_device" ...
Userspace may ask the kernel to destroy an aggregated GPIO
controller after use by writing its device name to the
"delete_device" file.
Example: Destroy the previously-created aggregated GPIO
controller, assumed to be "gpio-aggregator.0":
.. code-block:: sh
$ echo gpio-aggregator.0 > delete_device
Generic GPIO Driver
-------------------
The GPIO Aggregator can also be used as a generic driver for a simple
GPIO-operated device described in DT, without a dedicated in-kernel driver.
This is useful in industrial control, and is not unlike e.g. spidev, which
allows the user to communicate with an SPI device from userspace.
Binding a device to the GPIO Aggregator is performed either by modifying the
gpio-aggregator driver, or by writing to the "driver_override" file in Sysfs.
Example: If "door" is a GPIO-operated device described in DT, using its own
compatible value::
door {
compatible = "myvendor,mydoor";
gpios = <&gpio2 19 GPIO_ACTIVE_HIGH>,
<&gpio2 20 GPIO_ACTIVE_LOW>;
gpio-line-names = "open", "lock";
};
it can be bound to the GPIO Aggregator by either:
1. Adding its compatible value to ``gpio_aggregator_dt_ids[]``,
2. Binding manually using "driver_override":
.. code-block:: sh
$ echo gpio-aggregator > /sys/bus/platform/devices/door/driver_override
$ echo door > /sys/bus/platform/drivers/gpio-aggregator/bind
After that, a new gpiochip "door" has been created:
.. code-block:: sh
$ gpioinfo door
gpiochip12 - 2 lines:
line 0: "open" unused input active-high
line 1: "lock" unused input active-high

1
Documentation/admin-guide/gpio/index.rst
View File

@ -7,6 +7,7 @@ gpio
.. toctree::
:maxdepth: 1
gpio-aggregator
sysfs
.. only:: subproject and html

70
Documentation/devicetree/bindings/gpio/renesas,em-gio.yaml Normal file
View File

@ -0,0 +1,70 @@
# SPDX-License-Identifier: (GPL-2.0-only OR BSD-2-Clause)
%YAML 1.2
---
$id: http://devicetree.org/schemas/gpio/renesas,em-gio.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
title: Renesas EMMA Mobile General Purpose I/O Interface
maintainers:
- Magnus Damm <magnus.damm@gmail.com>
properties:
compatible:
const: renesas,em-gio
reg:
items:
- description: First set of contiguous registers
- description: Second set of contiguous registers
interrupts:
items:
- description: Interrupt for the first set of 16 GPIO ports
- description: Interrupt for the second set of 16 GPIO ports
gpio-controller: true
'#gpio-cells':
const: 2
gpio-ranges:
maxItems: 1
ngpios:
minimum: 1
maximum: 32
interrupt-controller: true
'#interrupt-cells':
const: 2
required:
- compatible
- reg
- interrupts
- gpio-controller
- '#gpio-cells'
- gpio-ranges
- ngpios
- interrupt-controller
- '#interrupt-cells'
additionalProperties: false
examples:
- |
#include <dt-bindings/interrupt-controller/arm-gic.h>
gpio0: gpio@e0050000 {
compatible = "renesas,em-gio";
reg = <0xe0050000 0x2c>, <0xe0050040 0x20>;
interrupts = <GIC_SPI 67 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 68 IRQ_TYPE_LEVEL_HIGH>;
gpio-controller;
#gpio-cells = <2>;
gpio-ranges = <&pfc 0 0 32>;
ngpios = <32>;
interrupt-controller;
#interrupt-cells = <2>;
};

134
Documentation/devicetree/bindings/gpio/snps,dw-apb-gpio.yaml Normal file
View File

@ -0,0 +1,134 @@
# SPDX-License-Identifier: (GPL-2.0 OR BSD-2-Clause)
%YAML 1.2
---
$id: http://devicetree.org/schemas/gpio/snps,dw-apb-gpio.yaml#
$schema: http://devicetree.org/meta-schemas/core.yaml#
title: Synopsys DesignWare APB GPIO controller
description: |
Synopsys DesignWare GPIO controllers have a configurable number of ports,
each of which are intended to be represented as child nodes with the generic
GPIO-controller properties as desribed in this bindings file.
maintainers:
- Hoan Tran <hoan@os.amperecomputing.com>
- Serge Semin <fancer.lancer@gmail.com>
properties:
$nodename:
pattern: "^gpio@[0-9a-f]+$"
compatible:
const: snps,dw-apb-gpio
"#address-cells":
const: 1
"#size-cells":
const: 0
reg:
maxItems: 1
clocks:
minItems: 1
items:
- description: APB interface clock source
- description: DW GPIO debounce reference clock source
clock-names:
minItems: 1
items:
- const: bus
- const: db
resets:
maxItems: 1
patternProperties:
"^gpio-(port|controller)@[0-9a-f]+$":
type: object
properties:
compatible:
const: snps,dw-apb-gpio-port
reg:
maxItems: 1
gpio-controller: true
'#gpio-cells':
const: 2
snps,nr-gpios:
description: The number of GPIO pins exported by the port.
default: 32
allOf:
- $ref: /schemas/types.yaml#/definitions/uint32
- minimum: 1
maximum: 32
interrupts:
description: |
The interrupts to the parent controller raised when GPIOs generate
the interrupts. If the controller provides one combined interrupt
for all GPIOs, specify a single interrupt. If the controller provides
one interrupt for each GPIO, provide a list of interrupts that
correspond to each of the GPIO pins.
minItems: 1
maxItems: 32
interrupt-controller: true
'#interrupt-cells':
const: 2
required:
- compatible
- reg
- gpio-controller
- '#gpio-cells'
dependencies:
interrupt-controller: [ interrupts ]
additionalProperties: false
additionalProperties: false
required:
- compatible
- reg
- "#address-cells"
- "#size-cells"
examples:
- |
gpio: gpio@20000 {
compatible = "snps,dw-apb-gpio";
reg = <0x20000 0x1000>;
#address-cells = <1>;
#size-cells = <0>;
porta: gpio-port@0 {
compatible = "snps,dw-apb-gpio-port";
reg = <0>;
gpio-controller;
#gpio-cells = <2>;
snps,nr-gpios = <8>;
interrupt-controller;
#interrupt-cells = <2>;
interrupt-parent = <&vic1>;
interrupts = <0>;
};
portb: gpio-port@1 {
compatible = "snps,dw-apb-gpio-port";
reg = <1>;
gpio-controller;
#gpio-cells = <2>;
snps,nr-gpios = <8>;
};
};
...

65
Documentation/devicetree/bindings/gpio/snps-dwapb-gpio.txt
View File

@ -1,65 +0,0 @@
* Synopsys DesignWare APB GPIO controller
Required properties:
- compatible : Should contain "snps,dw-apb-gpio"
- reg : Address and length of the register set for the device.
- #address-cells : should be 1 (for addressing port subnodes).
- #size-cells : should be 0 (port subnodes).
The GPIO controller has a configurable number of ports, each of which are
represented as child nodes with the following properties:
Required properties:
- compatible : "snps,dw-apb-gpio-port"
- gpio-controller : Marks the device node as a gpio controller.
- #gpio-cells : Should be two. The first cell is the pin number and
the second cell is used to specify the gpio polarity:
0 = active high
1 = active low
- reg : The integer port index of the port, a single cell.
Optional properties:
- interrupt-controller : The first port may be configured to be an interrupt
controller.
- #interrupt-cells : Specifies the number of cells needed to encode an
interrupt. Shall be set to 2. The first cell defines the interrupt number,
the second encodes the triger flags encoded as described in
Documentation/devicetree/bindings/interrupt-controller/interrupts.txt
- interrupts : The interrupts to the parent controller raised when GPIOs
generate the interrupts. If the controller provides one combined interrupt
for all GPIOs, specify a single interrupt. If the controller provides one
interrupt for each GPIO, provide a list of interrupts that correspond to each
of the GPIO pins. When specifying multiple interrupts, if any are unconnected,
use the interrupts-extended property to specify the interrupts and set the
interrupt controller handle for unused interrupts to 0.
- snps,nr-gpios : The number of pins in the port, a single cell.
- resets : Reset line for the controller.
Example:
gpio: gpio@20000 {
compatible = "snps,dw-apb-gpio";
reg = <0x20000 0x1000>;
#address-cells = <1>;
#size-cells = <0>;
porta: gpio@0 {
compatible = "snps,dw-apb-gpio-port";
gpio-controller;
#gpio-cells = <2>;
snps,nr-gpios = <8>;
reg = <0>;
interrupt-controller;
#interrupt-cells = <2>;
interrupt-parent = <&vic1>;
interrupts = <0>;
};
portb: gpio@1 {
compatible = "snps,dw-apb-gpio-port";
gpio-controller;
#gpio-cells = <2>;
snps,nr-gpios = <8>;
reg = <1>;
};
};

15
Documentation/driver-api/gpio/board.rst
View File

@ -113,13 +113,15 @@ files that desire to do so need to include the following header::
GPIOs are mapped by the means of tables of lookups, containing instances of the
gpiod_lookup structure. Two macros are defined to help declaring such mappings::
GPIO_LOOKUP(chip_label, chip_hwnum, con_id, flags)
GPIO_LOOKUP_IDX(chip_label, chip_hwnum, con_id, idx, flags)
GPIO_LOOKUP(key, chip_hwnum, con_id, flags)
GPIO_LOOKUP_IDX(key, chip_hwnum, con_id, idx, flags)
where
- chip_label is the label of the gpiod_chip instance providing the GPIO
- chip_hwnum is the hardware number of the GPIO within the chip
- key is either the label of the gpiod_chip instance providing the GPIO, or
the GPIO line name
- chip_hwnum is the hardware number of the GPIO within the chip, or U16_MAX
to indicate that key is a GPIO line name
- con_id is the name of the GPIO function from the device point of view. It
can be NULL, in which case it will match any function.
- idx is the index of the GPIO within the function.
@ -135,7 +137,10 @@ where
In the future, these flags might be extended to support more properties.
Note that GPIO_LOOKUP() is just a shortcut to GPIO_LOOKUP_IDX() where idx = 0.
Note that:
1. GPIO line names are not guaranteed to be globally unique, so the first
match found will be used.
2. GPIO_LOOKUP() is just a shortcut to GPIO_LOOKUP_IDX() where idx = 0.
A lookup table can then be defined as follows, with an empty entry defining its
end. The 'dev_id' field of the table is the identifier of the device that will

3
Documentation/driver-api/infiniband.rst
View File

@ -37,9 +37,6 @@ InfiniBand core interfaces
.. kernel-doc:: drivers/infiniband/core/ud_header.c
:export:
.. kernel-doc:: drivers/infiniband/core/fmr_pool.c
:export:
.. kernel-doc:: drivers/infiniband/core/umem.c
:export:

2
Documentation/infiniband/core_locking.rst
View File

@ -22,7 +22,6 @@ Sleeping and interrupt context
- post_recv
- poll_cq
- req_notify_cq
- map_phys_fmr
which may not sleep and must be callable from any context.
@ -36,7 +35,6 @@ Sleeping and interrupt context
- ib_post_send
- ib_post_recv
- ib_req_notify_cq
- ib_map_phys_fmr
are therefore safe to call from any context.

32
MAINTAINERS
View File

@ -7283,6 +7283,13 @@ F: Documentation/firmware-guide/acpi/gpio-properties.rst
F: drivers/gpio/gpiolib-acpi.c
F: drivers/gpio/gpiolib-acpi.h
GPIO AGGREGATOR
M: Geert Uytterhoeven <geert+renesas@glider.be>
L: linux-gpio@vger.kernel.org
S: Supported
F: Documentation/admin-guide/gpio/gpio-aggregator.rst
F: drivers/gpio/gpio-aggregator.c
GPIO IR Transmitter
M: Sean Young <sean@mess.org>
L: linux-media@vger.kernel.org
@ -7296,6 +7303,12 @@ S: Maintained
F: drivers/gpio/gpio-mockup.c
F: tools/testing/selftests/gpio/
GPIO REGMAP
R: Michael Walle <michael@walle.cc>
S: Maintained
F: drivers/gpio/gpio-regmap.c
F: include/linux/gpio/regmap.h
GPIO SUBSYSTEM
M: Linus Walleij <linus.walleij@linaro.org>
M: Bartosz Golaszewski <bgolaszewski@baylibre.com>
@ -10038,7 +10051,7 @@ F: drivers/hid/hid-lg-g15.c
LSILOGIC MPT FUSION DRIVERS (FC/SAS/SPI)
M: Sathya Prakash <sathya.prakash@broadcom.com>
M: Chaitra P B <chaitra.basappa@broadcom.com>
M: Sreekanth Reddy <sreekanth.reddy@broadcom.com>
M: Suganath Prabu Subramani <suganath-prabu.subramani@broadcom.com>
L: MPT-FusionLinux.pdl@broadcom.com
L: linux-scsi@vger.kernel.org
@ -14572,6 +14585,13 @@ F: arch/riscv/
N: riscv
K: riscv
RNBD BLOCK DRIVERS
M: Danil Kipnis <danil.kipnis@cloud.ionos.com>
M: Jack Wang <jinpu.wang@cloud.ionos.com>
L: linux-block@vger.kernel.org
S: Maintained
F: drivers/block/rnbd/
ROCCAT DRIVERS
M: Stefan Achatz <erazor_de@users.sourceforge.net>
S: Maintained
@ -14709,6 +14729,13 @@ S: Maintained
T: git git://git.kernel.org/pub/scm/linux/kernel/git/jes/linux.git rtl8xxxu-devel
F: drivers/net/wireless/realtek/rtl8xxxu/
RTRS TRANSPORT DRIVERS
M: Danil Kipnis <danil.kipnis@cloud.ionos.com>
M: Jack Wang <jinpu.wang@cloud.ionos.com>
L: linux-rdma@vger.kernel.org
S: Maintained
F: drivers/infiniband/ulp/rtrs/
RXRPC SOCKETS (AF_RXRPC)
M: David Howells <dhowells@redhat.com>
L: linux-afs@lists.infradead.org
@ -16374,9 +16401,10 @@ F: drivers/tty/serial/8250/8250_lpss.c
SYNOPSYS DESIGNWARE APB GPIO DRIVER
M: Hoan Tran <hoan@os.amperecomputing.com>
M: Serge Semin <fancer.lancer@gmail.com>
L: linux-gpio@vger.kernel.org
S: Maintained
F: Documentation/devicetree/bindings/gpio/snps-dwapb-gpio.txt
F: Documentation/devicetree/bindings/gpio/snps,dw-apb-gpio.yaml
F: drivers/gpio/gpio-dwapb.c
SYNOPSYS DESIGNWARE AXI DMAC DRIVER

27
arch/arm/kernel/elf.c
View File

@ -78,13 +78,32 @@ void elf_set_personality(const struct elf32_hdr *x)
EXPORT_SYMBOL(elf_set_personality);
/*
* Set READ_IMPLIES_EXEC if:
* - the binary requires an executable stack
* - we're running on a CPU which doesn't support NX.
* An executable for which elf_read_implies_exec() returns TRUE will
* have the READ_IMPLIES_EXEC personality flag set automatically.
*
* The decision process for determining the results are:
*
*              CPU: | lacks NX*  | has NX |
* ELF:              |            |           |
* ---------------------|------------|------------|
* missing PT_GNU_STACK | exec-all   | exec-all  |
* PT_GNU_STACK == RWX  | exec-all   | exec-stack |
* PT_GNU_STACK == RW   | exec-all  | exec-none |
*
* exec-all : all PROT_READ user mappings are executable, except when
* backed by files on a noexec-filesystem.
* exec-none : only PROT_EXEC user mappings are executable.
* exec-stack: only the stack and PROT_EXEC user mappings are executable.
*
* *this column has no architectural effect: NX markings are ignored by
* hardware, but may have behavioral effects when "wants X" collides with
* "cannot be X" constraints in memory permission flags, as in
* https://lkml.kernel.org/r/20190418055759.GA3155@mellanox.com
*
*/
int arm_elf_read_implies_exec(int executable_stack)
{
if (executable_stack != EXSTACK_DISABLE_X)
if (executable_stack == EXSTACK_DEFAULT)
return 1;
if (cpu_architecture() < CPU_ARCH_ARMv6)
return 1;

23
arch/arm64/include/asm/elf.h
View File

@ -96,7 +96,28 @@
*/
#define elf_check_arch(x) ((x)->e_machine == EM_AARCH64)
#define elf_read_implies_exec(ex,stk) (stk != EXSTACK_DISABLE_X)
/*
* An executable for which elf_read_implies_exec() returns TRUE will
* have the READ_IMPLIES_EXEC personality flag set automatically.
*
* The decision process for determining the results are:
*
*              CPU*: | arm32    | arm64 |
* ELF:              |            |            |
* ---------------------|------------|------------|
* missing PT_GNU_STACK | exec-all   | exec-none  |
* PT_GNU_STACK == RWX  | exec-stack | exec-stack |
* PT_GNU_STACK == RW   | exec-none | exec-none |
*
* exec-all : all PROT_READ user mappings are executable, except when
* backed by files on a noexec-filesystem.
* exec-none : only PROT_EXEC user mappings are executable.
* exec-stack: only the stack and PROT_EXEC user mappings are executable.
*
* *all arm64 CPUs support NX, so there is no "lacks NX" column.
*
*/
#define compat_elf_read_implies_exec(ex, stk) (stk == EXSTACK_DEFAULT)
#define CORE_DUMP_USE_REGSET
#define ELF_EXEC_PAGESIZE PAGE_SIZE

22
arch/x86/include/asm/elf.h
View File

@ -281,9 +281,29 @@ extern u32 elf_hwcap2;
/*
* An executable for which elf_read_implies_exec() returns TRUE will
* have the READ_IMPLIES_EXEC personality flag set automatically.
*
* The decision process for determining the results are:
*
*              CPU: | lacks NX*  | has NX, ia32     | has NX, x86_64 |
* ELF:              |            |                  |                |
* ---------------------|------------|------------------|----------------|
* missing PT_GNU_STACK | exec-all   | exec-all         | exec-none      |
* PT_GNU_STACK == RWX  | exec-stack | exec-stack       | exec-stack     |
* PT_GNU_STACK == RW   | exec-none  | exec-none        | exec-none      |
*
* exec-all : all PROT_READ user mappings are executable, except when
* backed by files on a noexec-filesystem.
* exec-none : only PROT_EXEC user mappings are executable.
* exec-stack: only the stack and PROT_EXEC user mappings are executable.
*
* *this column has no architectural effect: NX markings are ignored by
* hardware, but may have behavioral effects when "wants X" collides with
* "cannot be X" constraints in memory permission flags, as in
* https://lkml.kernel.org/r/20190418055759.GA3155@mellanox.com
*
*/
#define elf_read_implies_exec(ex, executable_stack) \
(executable_stack != EXSTACK_DISABLE_X)
(mmap_is_ia32() && executable_stack == EXSTACK_DEFAULT)
struct task_struct;

2
drivers/block/Kconfig
View File

@ -458,4 +458,6 @@ config BLK_DEV_RSXX
To compile this driver as a module, choose M here: the
module will be called rsxx.
source "drivers/block/rnbd/Kconfig"
endif # BLK_DEV

1
drivers/block/Makefile
View File

@ -39,6 +39,7 @@ obj-$(CONFIG_BLK_DEV_PCIESSD_MTIP32XX) += mtip32xx/
obj-$(CONFIG_BLK_DEV_RSXX) += rsxx/
obj-$(CONFIG_ZRAM) += zram/
obj-$(CONFIG_BLK_DEV_RNBD) += rnbd/
obj-$(CONFIG_BLK_DEV_NULL_BLK) += null_blk.o
null_blk-objs := null_blk_main.o

28
drivers/block/rnbd/Kconfig Normal file
View File

@ -0,0 +1,28 @@
# SPDX-License-Identifier: GPL-2.0-or-later
config BLK_DEV_RNBD
bool
config BLK_DEV_RNBD_CLIENT
tristate "RDMA Network Block Device driver client"
depends on INFINIBAND_RTRS_CLIENT
select BLK_DEV_RNBD
help
RNBD client is a network block device driver using rdma transport.
RNBD client allows for mapping of a remote block devices over
RTRS protocol from a target system where RNBD server is running.
If unsure, say N.
config BLK_DEV_RNBD_SERVER
tristate "RDMA Network Block Device driver server"
depends on INFINIBAND_RTRS_SERVER
select BLK_DEV_RNBD
help
RNBD server is the server side of RNBD using rdma transport.
RNBD server allows for exporting local block devices to a remote client
over RTRS protocol.
If unsure, say N.

15
drivers/block/rnbd/Makefile Normal file
View File

@ -0,0 +1,15 @@
# SPDX-License-Identifier: GPL-2.0-or-later
ccflags-y := -I$(srctree)/drivers/infiniband/ulp/rtrs
rnbd-client-y := rnbd-clt.o \
rnbd-clt-sysfs.o \
rnbd-common.o
rnbd-server-y := rnbd-common.o \
rnbd-srv.o \
rnbd-srv-dev.o \
rnbd-srv-sysfs.o
obj-$(CONFIG_BLK_DEV_RNBD_CLIENT) += rnbd-client.o
obj-$(CONFIG_BLK_DEV_RNBD_SERVER) += rnbd-server.o

92
drivers/block/rnbd/README Normal file
View File

@ -0,0 +1,92 @@
********************************
RDMA Network Block Device (RNBD)
********************************
Introduction
------------
RNBD (RDMA Network Block Device) is a pair of kernel modules
(client and server) that allow for remote access of a block device on
the server over RTRS protocol using the RDMA (InfiniBand, RoCE, iWARP)
transport. After being mapped, the remote block devices can be accessed
on the client side as local block devices.
I/O is transferred between client and server by the RTRS transport
modules. The administration of RNBD and RTRS modules is done via
sysfs entries.
Requirements
------------
RTRS kernel modules
Quick Start
-----------
Server side:
# modprobe rnbd_server
Client side:
# modprobe rnbd_client
# echo "sessname=blya path=ip:10.50.100.66 device_path=/dev/ram0" > \
/sys/devices/virtual/rnbd-client/ctl/map_device
Where "sessname=" is a session name, a string to identify the session
on client and on server sides; "path=" is a destination IP address or
a pair of a source and a destination IPs, separated by comma. Multiple
"path=" options can be specified in order to use multipath (see RTRS
description for details); "device_path=" is the block device to be
mapped from the server side. After the session to the server machine is
established, the mapped device will appear on the client side under
/dev/rnbd<N>.
RNBD-Server Module Parameters
=============================
dev_search_path
---------------
When a device is mapped from the client, the server generates the path
to the block device on the server side by concatenating dev_search_path
and the "device_path" that was specified in the map_device operation.
The default dev_search_path is: "/".
dev_search_path option can also contain %SESSNAME% in order to provide
different device namespaces for different sessions. See "device_path"
option for details.
============================
Protocol (rnbd/rnbd-proto.h)
============================
1. Before mapping first device from a given server, client sends an
RNBD_MSG_SESS_INFO to the server. Server responds with
RNBD_MSG_SESS_INFO_RSP. Currently the messages only contain the protocol
version for backward compatibility.
2. Client requests to open a device by sending RNBD_MSG_OPEN message. This
contains the path to the device and access mode (read-only or writable).
Server responds to the message with RNBD_MSG_OPEN_RSP. This contains
a 32 bit device id to be used for IOs and device "geometry" related
information: side, max_hw_sectors, etc.
3. Client attaches RNBD_MSG_IO to each IO message send to a device. This
message contains device id, provided by server in his rnbd_msg_open_rsp,
sector to be accessed, read-write flags and bi_size.
4. Client closes a device by sending RNBD_MSG_CLOSE which contains only the
device id provided by the server.
=========================================
Contributors List(in alphabetical order)
=========================================
Danil Kipnis <danil.kipnis@profitbricks.com>
Fabian Holler <mail@fholler.de>
Guoqing Jiang <guoqing.jiang@cloud.ionos.com>
Jack Wang <jinpu.wang@profitbricks.com>
Kleber Souza <kleber.souza@profitbricks.com>
Lutz Pogrell <lutz.pogrell@cloud.ionos.com>
Milind Dumbare <Milind.dumbare@gmail.com>
Roman Penyaev <roman.penyaev@profitbricks.com>

639
drivers/block/rnbd/rnbd-clt-sysfs.c Normal file
View File

@ -0,0 +1,639 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* RDMA Network Block Driver
*
* Copyright (c) 2014 - 2018 ProfitBricks GmbH. All rights reserved.
* Copyright (c) 2018 - 2019 1&1 IONOS Cloud GmbH. All rights reserved.
* Copyright (c) 2019 - 2020 1&1 IONOS SE. All rights reserved.
*/
#undef pr_fmt
#define pr_fmt(fmt) KBUILD_MODNAME " L" __stringify(__LINE__) ": " fmt
#include <linux/types.h>
#include <linux/ctype.h>
#include <linux/parser.h>
#include <linux/module.h>
#include <linux/in6.h>
#include <linux/fs.h>
#include <linux/uaccess.h>
#include <linux/device.h>
#include <rdma/ib.h>
#include <rdma/rdma_cm.h>
#include "rnbd-clt.h"
static struct device *rnbd_dev;
static struct class *rnbd_dev_class;
static struct kobject *rnbd_devs_kobj;
enum {
RNBD_OPT_ERR = 0,
RNBD_OPT_DEST_PORT = 1 << 0,
RNBD_OPT_PATH = 1 << 1,
RNBD_OPT_DEV_PATH = 1 << 2,
RNBD_OPT_ACCESS_MODE = 1 << 3,
RNBD_OPT_SESSNAME = 1 << 6,
};
static const unsigned int rnbd_opt_mandatory[] = {
RNBD_OPT_PATH,
RNBD_OPT_DEV_PATH,
RNBD_OPT_SESSNAME,
};
static const match_table_t rnbd_opt_tokens = {
{RNBD_OPT_PATH, "path=%s" },
{RNBD_OPT_DEV_PATH, "device_path=%s"},
{RNBD_OPT_DEST_PORT, "dest_port=%d" },
{RNBD_OPT_ACCESS_MODE, "access_mode=%s"},
{RNBD_OPT_SESSNAME, "sessname=%s" },
{RNBD_OPT_ERR, NULL },
};
struct rnbd_map_options {
char *sessname;
struct rtrs_addr *paths;
size_t *path_cnt;
char *pathname;
u16 *dest_port;
enum rnbd_access_mode *access_mode;
};
static int rnbd_clt_parse_map_options(const char *buf, size_t max_path_cnt,
struct rnbd_map_options *opt)
{
char *options, *sep_opt;
char *p;
substring_t args[MAX_OPT_ARGS];
int opt_mask = 0;
int token;
int ret = -EINVAL;
int i, dest_port;
int p_cnt = 0;
options = kstrdup(buf, GFP_KERNEL);
if (!options)
return -ENOMEM;
sep_opt = strstrip(options);
while ((p = strsep(&sep_opt, " ")) != NULL) {
if (!*p)
continue;
token = match_token(p, rnbd_opt_tokens, args);
opt_mask |= token;
switch (token) {
case RNBD_OPT_SESSNAME:
p = match_strdup(args);
if (!p) {
ret = -ENOMEM;
goto out;
}
if (strlen(p) > NAME_MAX) {
pr_err("map_device: sessname too long\n");
ret = -EINVAL;
kfree(p);
goto out;
}
strlcpy(opt->sessname, p, NAME_MAX);
kfree(p);
break;
case RNBD_OPT_PATH:
if (p_cnt >= max_path_cnt) {
pr_err("map_device: too many (> %zu) paths provided\n",
max_path_cnt);
ret = -ENOMEM;
goto out;
}
p = match_strdup(args);
if (!p) {
ret = -ENOMEM;
goto out;
}
ret = rtrs_addr_to_sockaddr(p, strlen(p),
*opt->dest_port,
&opt->paths[p_cnt]);
if (ret) {
pr_err("Can't parse path %s: %d\n", p, ret);
kfree(p);
goto out;
}
p_cnt++;
kfree(p);
break;
case RNBD_OPT_DEV_PATH:
p = match_strdup(args);
if (!p) {
ret = -ENOMEM;
goto out;
}
if (strlen(p) > NAME_MAX) {
pr_err("map_device: Device path too long\n");
ret = -EINVAL;
kfree(p);
goto out;
}
strlcpy(opt->pathname, p, NAME_MAX);
kfree(p);
break;
case RNBD_OPT_DEST_PORT:
if (match_int(args, &dest_port) || dest_port < 0 ||
dest_port > 65535) {
pr_err("bad destination port number parameter '%d'\n",
dest_port);
ret = -EINVAL;
goto out;
}
*opt->dest_port = dest_port;
break;
case RNBD_OPT_ACCESS_MODE:
p = match_strdup(args);
if (!p) {
ret = -ENOMEM;
goto out;
}
if (!strcmp(p, "ro")) {
*opt->access_mode = RNBD_ACCESS_RO;
} else if (!strcmp(p, "rw")) {
*opt->access_mode = RNBD_ACCESS_RW;
} else if (!strcmp(p, "migration")) {
*opt->access_mode = RNBD_ACCESS_MIGRATION;
} else {
pr_err("map_device: Invalid access_mode: '%s'\n",
p);
ret = -EINVAL;
kfree(p);
goto out;
}
kfree(p);
break;
default:
pr_err("map_device: Unknown parameter or missing value '%s'\n",
p);
ret = -EINVAL;
goto out;
}
}
for (i = 0; i < ARRAY_SIZE(rnbd_opt_mandatory); i++) {
if ((opt_mask & rnbd_opt_mandatory[i])) {
ret = 0;
} else {
pr_err("map_device: Parameters missing\n");
ret = -EINVAL;
break;
}
}
out:
*opt->path_cnt = p_cnt;
kfree(options);
return ret;
}
static ssize_t state_show(struct kobject *kobj,
struct kobj_attribute *attr, char *page)
{
struct rnbd_clt_dev *dev;
dev = container_of(kobj, struct rnbd_clt_dev, kobj);
switch (dev->dev_state) {
case DEV_STATE_INIT:
return snprintf(page, PAGE_SIZE, "init\n");
case DEV_STATE_MAPPED:
/* TODO fix cli tool before changing to proper state */
return snprintf(page, PAGE_SIZE, "open\n");
case DEV_STATE_MAPPED_DISCONNECTED:
/* TODO fix cli tool before changing to proper state */
return snprintf(page, PAGE_SIZE, "closed\n");
case DEV_STATE_UNMAPPED:
return snprintf(page, PAGE_SIZE, "unmapped\n");
default:
return snprintf(page, PAGE_SIZE, "unknown\n");
}
}
static struct kobj_attribute rnbd_clt_state_attr = __ATTR_RO(state);
static ssize_t mapping_path_show(struct kobject *kobj,
struct kobj_attribute *attr, char *page)
{
struct rnbd_clt_dev *dev;
dev = container_of(kobj, struct rnbd_clt_dev, kobj);
return scnprintf(page, PAGE_SIZE, "%s\n", dev->pathname);
}
static struct kobj_attribute rnbd_clt_mapping_path_attr =
__ATTR_RO(mapping_path);
static ssize_t access_mode_show(struct kobject *kobj,
struct kobj_attribute *attr, char *page)
{
struct rnbd_clt_dev *dev;
dev = container_of(kobj, struct rnbd_clt_dev, kobj);
return snprintf(page, PAGE_SIZE, "%s\n",
rnbd_access_mode_str(dev->access_mode));
}
static struct kobj_attribute rnbd_clt_access_mode =
__ATTR_RO(access_mode);
static ssize_t rnbd_clt_unmap_dev_show(struct kobject *kobj,
struct kobj_attribute *attr, char *page)
{
return scnprintf(page, PAGE_SIZE, "Usage: echo <normal|force> > %s\n",
attr->attr.name);
}
static ssize_t rnbd_clt_unmap_dev_store(struct kobject *kobj,
struct kobj_attribute *attr,
const char *buf, size_t count)
{
struct rnbd_clt_dev *dev;
char *opt, *options;
bool force;
int err;
opt = kstrdup(buf, GFP_KERNEL);
if (!opt)
return -ENOMEM;
options = strstrip(opt);
dev = container_of(kobj, struct rnbd_clt_dev, kobj);
if (sysfs_streq(options, "normal")) {
force = false;
} else if (sysfs_streq(options, "force")) {
force = true;
} else {
rnbd_clt_err(dev,
"unmap_device: Invalid value: %s\n",
options);
err = -EINVAL;
goto out;
}
rnbd_clt_info(dev, "Unmapping device, option: %s.\n",
force ? "force" : "normal");
/*
* We take explicit module reference only for one reason: do not
* race with lockless rnbd_destroy_sessions().
*/
if (!try_module_get(THIS_MODULE)) {
err = -ENODEV;
goto out;
}
err = rnbd_clt_unmap_device(dev, force, &attr->attr);
if (err) {
if (err != -EALREADY)
rnbd_clt_err(dev, "unmap_device: %d\n", err);
goto module_put;
}
/*
* Here device can be vanished!
*/
err = count;
module_put:
module_put(THIS_MODULE);
out:
kfree(opt);
return err;
}
static struct kobj_attribute rnbd_clt_unmap_device_attr =
__ATTR(unmap_device, 0644, rnbd_clt_unmap_dev_show,
rnbd_clt_unmap_dev_store);
static ssize_t rnbd_clt_resize_dev_show(struct kobject *kobj,
struct kobj_attribute *attr,
char *page)
{
return scnprintf(page, PAGE_SIZE,
"Usage: echo <new size in sectors> > %s\n",
attr->attr.name);
}
static ssize_t rnbd_clt_resize_dev_store(struct kobject *kobj,
struct kobj_attribute *attr,
const char *buf, size_t count)
{
int ret;
unsigned long sectors;
struct rnbd_clt_dev *dev;
dev = container_of(kobj, struct rnbd_clt_dev, kobj);
ret = kstrtoul(buf, 0, &sectors);
if (ret)
return ret;
ret = rnbd_clt_resize_disk(dev, (size_t)sectors);
if (ret)
return ret;
return count;
}
static struct kobj_attribute rnbd_clt_resize_dev_attr =
__ATTR(resize, 0644, rnbd_clt_resize_dev_show,
rnbd_clt_resize_dev_store);
static ssize_t rnbd_clt_remap_dev_show(struct kobject *kobj,
struct kobj_attribute *attr, char *page)
{
return scnprintf(page, PAGE_SIZE, "Usage: echo <1> > %s\n",
attr->attr.name);
}
static ssize_t rnbd_clt_remap_dev_store(struct kobject *kobj,
struct kobj_attribute *attr,
const char *buf, size_t count)
{
struct rnbd_clt_dev *dev;
char *opt, *options;
int err;
opt = kstrdup(buf, GFP_KERNEL);
if (!opt)
return -ENOMEM;
options = strstrip(opt);
dev = container_of(kobj, struct rnbd_clt_dev, kobj);
if (!sysfs_streq(options, "1")) {
rnbd_clt_err(dev,
"remap_device: Invalid value: %s\n",
options);
err = -EINVAL;
goto out;
}
err = rnbd_clt_remap_device(dev);
if (likely(!err))
err = count;
out:
kfree(opt);
return err;
}
static struct kobj_attribute rnbd_clt_remap_device_attr =
__ATTR(remap_device, 0644, rnbd_clt_remap_dev_show,
rnbd_clt_remap_dev_store);
static ssize_t session_show(struct kobject *kobj, struct kobj_attribute *attr,
char *page)
{
struct rnbd_clt_dev *dev;
dev = container_of(kobj, struct rnbd_clt_dev, kobj);
return scnprintf(page, PAGE_SIZE, "%s\n", dev->sess->sessname);
}
static struct kobj_attribute rnbd_clt_session_attr =
__ATTR_RO(session);
static struct attribute *rnbd_dev_attrs[] = {
&rnbd_clt_unmap_device_attr.attr,
&rnbd_clt_resize_dev_attr.attr,
&rnbd_clt_remap_device_attr.attr,
&rnbd_clt_mapping_path_attr.attr,
&rnbd_clt_state_attr.attr,
&rnbd_clt_session_attr.attr,
&rnbd_clt_access_mode.attr,
NULL,
};
void rnbd_clt_remove_dev_symlink(struct rnbd_clt_dev *dev)
{
/*
* The module unload rnbd_client_exit path is racing with unmapping of
* the last single device from the sysfs manually
* i.e. rnbd_clt_unmap_dev_store() leading to a sysfs warning because
* of sysfs link already was removed already.
*/
if (strlen(dev->blk_symlink_name) && try_module_get(THIS_MODULE)) {
sysfs_remove_link(rnbd_devs_kobj, dev->blk_symlink_name);
module_put(THIS_MODULE);
}
}
static struct kobj_type rnbd_dev_ktype = {
.sysfs_ops = &kobj_sysfs_ops,
.default_attrs = rnbd_dev_attrs,
};
static int rnbd_clt_add_dev_kobj(struct rnbd_clt_dev *dev)
{
int ret;
struct kobject *gd_kobj = &disk_to_dev(dev->gd)->kobj;
ret = kobject_init_and_add(&dev->kobj, &rnbd_dev_ktype, gd_kobj, "%s",
"rnbd");
if (ret)
rnbd_clt_err(dev, "Failed to create device sysfs dir, err: %d\n",
ret);
return ret;
}
static ssize_t rnbd_clt_map_device_show(struct kobject *kobj,
struct kobj_attribute *attr,
char *page)
{
return scnprintf(page, PAGE_SIZE,
"Usage: echo \"[dest_port=server port number] sessname=<name of the rtrs session> path=<[srcaddr@]dstaddr> [path=<[srcaddr@]dstaddr>] device_path=<full path on remote side> [access_mode=<ro|rw|migration>]\" > %s\n\naddr ::= [ ip:<ipv4> | ip:<ipv6> | gid:<gid> ]\n",
attr->attr.name);
}
static int rnbd_clt_get_path_name(struct rnbd_clt_dev *dev, char *buf,
size_t len)
{
int ret;
char pathname[NAME_MAX], *s;
strlcpy(pathname, dev->pathname, sizeof(pathname));
while ((s = strchr(pathname, '/')))
s[0] = '!';
ret = snprintf(buf, len, "%s", pathname);
if (ret >= len)
return -ENAMETOOLONG;
return 0;
}
static int rnbd_clt_add_dev_symlink(struct rnbd_clt_dev *dev)
{
struct kobject *gd_kobj = &disk_to_dev(dev->gd)->kobj;
int ret;
ret = rnbd_clt_get_path_name(dev, dev->blk_symlink_name,
sizeof(dev->blk_symlink_name));
if (ret) {
rnbd_clt_err(dev, "Failed to get /sys/block symlink path, err: %d\n",
ret);
goto out_err;
}
ret = sysfs_create_link(rnbd_devs_kobj, gd_kobj,
dev->blk_symlink_name);
if (ret) {
rnbd_clt_err(dev, "Creating /sys/block symlink failed, err: %d\n",
ret);
goto out_err;
}
return 0;
out_err:
dev->blk_symlink_name[0] = '\0';
return ret;
}
static ssize_t rnbd_clt_map_device_store(struct kobject *kobj,
struct kobj_attribute *attr,
const char *buf, size_t count)
{
struct rnbd_clt_dev *dev;
struct rnbd_map_options opt;
int ret;
char pathname[NAME_MAX];
char sessname[NAME_MAX];
enum rnbd_access_mode access_mode = RNBD_ACCESS_RW;
u16 port_nr = RTRS_PORT;
struct sockaddr_storage *addrs;
struct rtrs_addr paths[6];
size_t path_cnt;
opt.sessname = sessname;
opt.paths = paths;
opt.path_cnt = &path_cnt;
opt.pathname = pathname;
opt.dest_port = &port_nr;
opt.access_mode = &access_mode;
addrs = kcalloc(ARRAY_SIZE(paths) * 2, sizeof(*addrs), GFP_KERNEL);
if (!addrs)
return -ENOMEM;
for (path_cnt = 0; path_cnt < ARRAY_SIZE(paths); path_cnt++) {
paths[path_cnt].src = &addrs[path_cnt * 2];
paths[path_cnt].dst = &addrs[path_cnt * 2 + 1];
}
ret = rnbd_clt_parse_map_options(buf, ARRAY_SIZE(paths), &opt);
if (ret)
goto out;
pr_info("Mapping device %s on session %s, (access_mode: %s)\n",
pathname, sessname,
rnbd_access_mode_str(access_mode));
dev = rnbd_clt_map_device(sessname, paths, path_cnt, port_nr, pathname,
access_mode);
if (IS_ERR(dev)) {
ret = PTR_ERR(dev);
goto out;
}
ret = rnbd_clt_add_dev_kobj(dev);
if (ret)
goto unmap_dev;
ret = rnbd_clt_add_dev_symlink(dev);
if (ret)
goto unmap_dev;
kfree(addrs);
return count;
unmap_dev:
rnbd_clt_unmap_device(dev, true, NULL);
out:
kfree(addrs);
return ret;
}
static struct kobj_attribute rnbd_clt_map_device_attr =
__ATTR(map_device, 0644,
rnbd_clt_map_device_show, rnbd_clt_map_device_store);
static struct attribute *default_attrs[] = {
&rnbd_clt_map_device_attr.attr,
NULL,
};
static struct attribute_group default_attr_group = {
.attrs = default_attrs,
};
static const struct attribute_group *default_attr_groups[] = {
&default_attr_group,
NULL,
};
int rnbd_clt_create_sysfs_files(void)
{
int err;
rnbd_dev_class = class_create(THIS_MODULE, "rnbd-client");
if (IS_ERR(rnbd_dev_class))
return PTR_ERR(rnbd_dev_class);
rnbd_dev = device_create_with_groups(rnbd_dev_class, NULL,
MKDEV(0, 0), NULL,
default_attr_groups, "ctl");
if (IS_ERR(rnbd_dev)) {
err = PTR_ERR(rnbd_dev);
goto cls_destroy;
}
rnbd_devs_kobj = kobject_create_and_add("devices", &rnbd_dev->kobj);
if (!rnbd_devs_kobj) {
err = -ENOMEM;
goto dev_destroy;
}
return 0;
dev_destroy:
device_destroy(rnbd_dev_class, MKDEV(0, 0));
cls_destroy:
class_destroy(rnbd_dev_class);
return err;
}
void rnbd_clt_destroy_default_group(void)
{
sysfs_remove_group(&rnbd_dev->kobj, &default_attr_group);
}
void rnbd_clt_destroy_sysfs_files(void)
{
kobject_del(rnbd_devs_kobj);
kobject_put(rnbd_devs_kobj);
device_destroy(rnbd_dev_class, MKDEV(0, 0));
class_destroy(rnbd_dev_class);
}

1729
drivers/block/rnbd/rnbd-clt.c Normal file
View File

File diff suppressed because it is too large Load Diff

156
drivers/block/rnbd/rnbd-clt.h Normal file
View File

@ -0,0 +1,156 @@
/* SPDX-License-Identifier: GPL-2.0-or-later */
/*
* RDMA Network Block Driver
*
* Copyright (c) 2014 - 2018 ProfitBricks GmbH. All rights reserved.
* Copyright (c) 2018 - 2019 1&1 IONOS Cloud GmbH. All rights reserved.
* Copyright (c) 2019 - 2020 1&1 IONOS SE. All rights reserved.
*/
#ifndef RNBD_CLT_H
#define RNBD_CLT_H
#include <linux/wait.h>
#include <linux/in.h>
#include <linux/inet.h>
#include <linux/blk-mq.h>
#include <linux/refcount.h>
#include <rtrs.h>
#include "rnbd-proto.h"
#include "rnbd-log.h"
/* Max. number of segments per IO request, Mellanox Connect X ~ Connect X5,
* choose minimial 30 for all, minus 1 for internal protocol, so 29.
*/
#define BMAX_SEGMENTS 29
/* time in seconds between reconnect tries, default to 30 s */
#define RECONNECT_DELAY 30
/*
* Number of times to reconnect on error before giving up, 0 for * disabled,
* -1 for forever
*/
#define MAX_RECONNECTS -1
enum rnbd_clt_dev_state {
DEV_STATE_INIT,
DEV_STATE_MAPPED,
DEV_STATE_MAPPED_DISCONNECTED,
DEV_STATE_UNMAPPED,
};
struct rnbd_iu_comp {
wait_queue_head_t wait;
int errno;
};
struct rnbd_iu {
union {
struct request *rq; /* for block io */
void *buf; /* for user messages */
};
struct rtrs_permit *permit;
union {
/* use to send msg associated with a dev */
struct rnbd_clt_dev *dev;
/* use to send msg associated with a sess */
struct rnbd_clt_session *sess;
};
struct scatterlist sglist[BMAX_SEGMENTS];
struct work_struct work;
int errno;
struct rnbd_iu_comp comp;
atomic_t refcount;
};
struct rnbd_cpu_qlist {
struct list_head requeue_list;
spinlock_t requeue_lock;
unsigned int cpu;
};
struct rnbd_clt_session {
struct list_head list;
struct rtrs_clt *rtrs;
wait_queue_head_t rtrs_waitq;
bool rtrs_ready;
struct rnbd_cpu_qlist __percpu
*cpu_queues;
DECLARE_BITMAP(cpu_queues_bm, NR_CPUS);
int __percpu *cpu_rr; /* per-cpu var for CPU round-robin */
atomic_t busy;
int queue_depth;
u32 max_io_size;
struct blk_mq_tag_set tag_set;
struct mutex lock; /* protects state and devs_list */
struct list_head devs_list; /* list of struct rnbd_clt_dev */
refcount_t refcount;
char sessname[NAME_MAX];
u8 ver; /* protocol version */
};
/**
* Submission queues.
*/
struct rnbd_queue {
struct list_head requeue_list;
unsigned long in_list;
struct rnbd_clt_dev *dev;
struct blk_mq_hw_ctx *hctx;
};
struct rnbd_clt_dev {
struct rnbd_clt_session *sess;
struct request_queue *queue;
struct rnbd_queue *hw_queues;
u32 device_id;
/* local Idr index - used to track minor number allocations. */
u32 clt_device_id;
struct mutex lock;
enum rnbd_clt_dev_state dev_state;
char pathname[NAME_MAX];
enum rnbd_access_mode access_mode;
bool read_only;
bool rotational;
u32 max_hw_sectors;
u32 max_write_same_sectors;
u32 max_discard_sectors;
u32 discard_granularity;
u32 discard_alignment;
u16 secure_discard;
u16 physical_block_size;
u16 logical_block_size;
u16 max_segments;
size_t nsectors;
u64 size; /* device size in bytes */
struct list_head list;
struct gendisk *gd;
struct kobject kobj;
char blk_symlink_name[NAME_MAX];
refcount_t refcount;
struct work_struct unmap_on_rmmod_work;
};
/* rnbd-clt.c */
struct rnbd_clt_dev *rnbd_clt_map_device(const char *sessname,
struct rtrs_addr *paths,
size_t path_cnt, u16 port_nr,
const char *pathname,
enum rnbd_access_mode access_mode);
int rnbd_clt_unmap_device(struct rnbd_clt_dev *dev, bool force,
const struct attribute *sysfs_self);
int rnbd_clt_remap_device(struct rnbd_clt_dev *dev);
int rnbd_clt_resize_disk(struct rnbd_clt_dev *dev, size_t newsize);
/* rnbd-clt-sysfs.c */
int rnbd_clt_create_sysfs_files(void);
void rnbd_clt_destroy_sysfs_files(void);
void rnbd_clt_destroy_default_group(void);
void rnbd_clt_remove_dev_symlink(struct rnbd_clt_dev *dev);
#endif /* RNBD_CLT_H */

23
drivers/block/rnbd/rnbd-common.c Normal file
View File

@ -0,0 +1,23 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* RDMA Network Block Driver
*
* Copyright (c) 2014 - 2018 ProfitBricks GmbH. All rights reserved.
* Copyright (c) 2018 - 2019 1&1 IONOS Cloud GmbH. All rights reserved.
* Copyright (c) 2019 - 2020 1&1 IONOS SE. All rights reserved.
*/
#include "rnbd-proto.h"
const char *rnbd_access_mode_str(enum rnbd_access_mode mode)
{
switch (mode) {
case RNBD_ACCESS_RO:
return "ro";
case RNBD_ACCESS_RW:
return "rw";
case RNBD_ACCESS_MIGRATION:
return "migration";
default:
return "unknown";
}
}

41
drivers/block/rnbd/rnbd-log.h Normal file
View File

@ -0,0 +1,41 @@
/* SPDX-License-Identifier: GPL-2.0-or-later */
/*
* RDMA Network Block Driver
*
* Copyright (c) 2014 - 2018 ProfitBricks GmbH. All rights reserved.
* Copyright (c) 2018 - 2019 1&1 IONOS Cloud GmbH. All rights reserved.
* Copyright (c) 2019 - 2020 1&1 IONOS SE. All rights reserved.
*/
#ifndef RNBD_LOG_H
#define RNBD_LOG_H
#include "rnbd-clt.h"
#include "rnbd-srv.h"
#define rnbd_clt_log(fn, dev, fmt, ...) ( \
fn("<%s@%s> " fmt, (dev)->pathname, \
(dev)->sess->sessname, \
##__VA_ARGS__))
#define rnbd_srv_log(fn, dev, fmt, ...) ( \
fn("<%s@%s>: " fmt, (dev)->pathname, \
(dev)->sess->sessname, ##__VA_ARGS__))
#define rnbd_clt_err(dev, fmt, ...) \
rnbd_clt_log(pr_err, dev, fmt, ##__VA_ARGS__)
#define rnbd_clt_err_rl(dev, fmt, ...) \
rnbd_clt_log(pr_err_ratelimited, dev, fmt, ##__VA_ARGS__)
#define rnbd_clt_info(dev, fmt, ...) \
rnbd_clt_log(pr_info, dev, fmt, ##__VA_ARGS__)
#define rnbd_clt_info_rl(dev, fmt, ...) \
rnbd_clt_log(pr_info_ratelimited, dev, fmt, ##__VA_ARGS__)
#define rnbd_srv_err(dev, fmt, ...) \
rnbd_srv_log(pr_err, dev, fmt, ##__VA_ARGS__)
#define rnbd_srv_err_rl(dev, fmt, ...) \
rnbd_srv_log(pr_err_ratelimited, dev, fmt, ##__VA_ARGS__)
#define rnbd_srv_info(dev, fmt, ...) \
rnbd_srv_log(pr_info, dev, fmt, ##__VA_ARGS__)
#define rnbd_srv_info_rl(dev, fmt, ...) \
rnbd_srv_log(pr_info_ratelimited, dev, fmt, ##__VA_ARGS__)
#endif /* RNBD_LOG_H */

303
drivers/block/rnbd/rnbd-proto.h Normal file
View File

@ -0,0 +1,303 @@
/* SPDX-License-Identifier: GPL-2.0-or-later */
/*
* RDMA Network Block Driver
*
* Copyright (c) 2014 - 2018 ProfitBricks GmbH. All rights reserved.
* Copyright (c) 2018 - 2019 1&1 IONOS Cloud GmbH. All rights reserved.
* Copyright (c) 2019 - 2020 1&1 IONOS SE. All rights reserved.
*/
#ifndef RNBD_PROTO_H
#define RNBD_PROTO_H
#include <linux/types.h>
#include <linux/blkdev.h>
#include <linux/limits.h>
#include <linux/inet.h>
#include <linux/in.h>
#include <linux/in6.h>
#include <rdma/ib.h>
#define RNBD_PROTO_VER_MAJOR 2
#define RNBD_PROTO_VER_MINOR 0
/* The default port number the RTRS server is listening on. */
#define RTRS_PORT 1234
/**
* enum rnbd_msg_types - RNBD message types
* @RNBD_MSG_SESS_INFO: initial session info from client to server
* @RNBD_MSG_SESS_INFO_RSP: initial session info from server to client
* @RNBD_MSG_OPEN: open (map) device request
* @RNBD_MSG_OPEN_RSP: response to an @RNBD_MSG_OPEN
* @RNBD_MSG_IO: block IO request operation
* @RNBD_MSG_CLOSE: close (unmap) device request
*/
enum rnbd_msg_type {
RNBD_MSG_SESS_INFO,
RNBD_MSG_SESS_INFO_RSP,
RNBD_MSG_OPEN,
RNBD_MSG_OPEN_RSP,
RNBD_MSG_IO,
RNBD_MSG_CLOSE,
};
/**
* struct rnbd_msg_hdr - header of RNBD messages
* @type: Message type, valid values see: enum rnbd_msg_types
*/
struct rnbd_msg_hdr {
__le16 type;
__le16 __padding;
};
/**
* We allow to map RO many times and RW only once. We allow to map yet another
* time RW, if MIGRATION is provided (second RW export can be required for
* example for VM migration)
*/
enum rnbd_access_mode {
RNBD_ACCESS_RO,
RNBD_ACCESS_RW,
RNBD_ACCESS_MIGRATION,
};
/**
* struct rnbd_msg_sess_info - initial session info from client to server
* @hdr: message header
* @ver: RNBD protocol version
*/
struct rnbd_msg_sess_info {
struct rnbd_msg_hdr hdr;
u8 ver;
u8 reserved[31];
};
/**
* struct rnbd_msg_sess_info_rsp - initial session info from server to client
* @hdr: message header
* @ver: RNBD protocol version
*/
struct rnbd_msg_sess_info_rsp {
struct rnbd_msg_hdr hdr;
u8 ver;
u8 reserved[31];
};
/**
* struct rnbd_msg_open - request to open a remote device.
* @hdr: message header
* @access_mode: the mode to open remote device, valid values see:
* enum rnbd_access_mode
* @device_name: device path on remote side
*/
struct rnbd_msg_open {
struct rnbd_msg_hdr hdr;
u8 access_mode;
u8 resv1;
s8 dev_name[NAME_MAX];
u8 reserved[3];
};
/**
* struct rnbd_msg_close - request to close a remote device.
* @hdr: message header
* @device_id: device_id on server side to identify the device
*/
struct rnbd_msg_close {
struct rnbd_msg_hdr hdr;
__le32 device_id;
};
/**
* struct rnbd_msg_open_rsp - response message to RNBD_MSG_OPEN
* @hdr: message header
* @device_id: device_id on server side to identify the device
* @nsectors: number of sectors in the usual 512b unit
* @max_hw_sectors: max hardware sectors in the usual 512b unit
* @max_write_same_sectors: max sectors for WRITE SAME in the 512b unit
* @max_discard_sectors: max. sectors that can be discarded at once in 512b
* unit.
* @discard_granularity: size of the internal discard allocation unit in bytes
* @discard_alignment: offset from internal allocation assignment in bytes
* @physical_block_size: physical block size device supports in bytes
* @logical_block_size: logical block size device supports in bytes
* @max_segments: max segments hardware support in one transfer
* @secure_discard: supports secure discard
* @rotation: is a rotational disc?
*/
struct rnbd_msg_open_rsp {
struct rnbd_msg_hdr hdr;
__le32 device_id;
__le64 nsectors;
__le32 max_hw_sectors;
__le32 max_write_same_sectors;
__le32 max_discard_sectors;
__le32 discard_granularity;
__le32 discard_alignment;
__le16 physical_block_size;
__le16 logical_block_size;
__le16 max_segments;
__le16 secure_discard;
u8 rotational;
u8 reserved[11];
};
/**
* struct rnbd_msg_io - message for I/O read/write
* @hdr: message header
* @device_id: device_id on server side to find the right device
* @sector: bi_sector attribute from struct bio
* @rw: valid values are defined in enum rnbd_io_flags
* @bi_size: number of bytes for I/O read/write
* @prio: priority
*/
struct rnbd_msg_io {
struct rnbd_msg_hdr hdr;
__le32 device_id;
__le64 sector;
__le32 rw;
__le32 bi_size;
__le16 prio;
};
#define RNBD_OP_BITS 8
#define RNBD_OP_MASK ((1 << RNBD_OP_BITS) - 1)
/**
* enum rnbd_io_flags - RNBD request types from rq_flag_bits
* @RNBD_OP_READ: read sectors from the device
* @RNBD_OP_WRITE: write sectors to the device
* @RNBD_OP_FLUSH: flush the volatile write cache
* @RNBD_OP_DISCARD: discard sectors
* @RNBD_OP_SECURE_ERASE: securely erase sectors
* @RNBD_OP_WRITE_SAME: write the same sectors many times
* @RNBD_F_SYNC: request is sync (sync write or read)
* @RNBD_F_FUA: forced unit access
*/
enum rnbd_io_flags {
/* Operations */
RNBD_OP_READ = 0,
RNBD_OP_WRITE = 1,
RNBD_OP_FLUSH = 2,
RNBD_OP_DISCARD = 3,
RNBD_OP_SECURE_ERASE = 4,
RNBD_OP_WRITE_SAME = 5,
RNBD_OP_LAST,
/* Flags */
RNBD_F_SYNC = 1<<(RNBD_OP_BITS + 0),
RNBD_F_FUA = 1<<(RNBD_OP_BITS + 1),
RNBD_F_ALL = (RNBD_F_SYNC | RNBD_F_FUA)
};
static inline u32 rnbd_op(u32 flags)
{
return flags & RNBD_OP_MASK;
}
static inline u32 rnbd_flags(u32 flags)
{
return flags & ~RNBD_OP_MASK;
}
static inline bool rnbd_flags_supported(u32 flags)
{
u32 op;
op = rnbd_op(flags);
flags = rnbd_flags(flags);
if (op >= RNBD_OP_LAST)
return false;
if (flags & ~RNBD_F_ALL)
return false;
return true;
}
static inline u32 rnbd_to_bio_flags(u32 rnbd_opf)
{
u32 bio_opf;
switch (rnbd_op(rnbd_opf)) {
case RNBD_OP_READ:
bio_opf = REQ_OP_READ;
break;
case RNBD_OP_WRITE:
bio_opf = REQ_OP_WRITE;
break;
case RNBD_OP_FLUSH:
bio_opf = REQ_OP_FLUSH | REQ_PREFLUSH;
break;
case RNBD_OP_DISCARD:
bio_opf = REQ_OP_DISCARD;
break;
case RNBD_OP_SECURE_ERASE:
bio_opf = REQ_OP_SECURE_ERASE;
break;
case RNBD_OP_WRITE_SAME:
bio_opf = REQ_OP_WRITE_SAME;
break;
default:
WARN(1, "Unknown RNBD type: %d (flags %d)\n",
rnbd_op(rnbd_opf), rnbd_opf);
bio_opf = 0;
}
if (rnbd_opf & RNBD_F_SYNC)
bio_opf |= REQ_SYNC;
if (rnbd_opf & RNBD_F_FUA)
bio_opf |= REQ_FUA;
return bio_opf;
}
static inline u32 rq_to_rnbd_flags(struct request *rq)
{
u32 rnbd_opf;
switch (req_op(rq)) {
case REQ_OP_READ:
rnbd_opf = RNBD_OP_READ;
break;
case REQ_OP_WRITE:
rnbd_opf = RNBD_OP_WRITE;
break;
case REQ_OP_DISCARD:
rnbd_opf = RNBD_OP_DISCARD;
break;
case REQ_OP_SECURE_ERASE:
rnbd_opf = RNBD_OP_SECURE_ERASE;
break;
case REQ_OP_WRITE_SAME:
rnbd_opf = RNBD_OP_WRITE_SAME;
break;
case REQ_OP_FLUSH:
rnbd_opf = RNBD_OP_FLUSH;
break;
default:
WARN(1, "Unknown request type %d (flags %llu)\n",
req_op(rq), (unsigned long long)rq->cmd_flags);
rnbd_opf = 0;
}
if (op_is_sync(rq->cmd_flags))
rnbd_opf |= RNBD_F_SYNC;
if (op_is_flush(rq->cmd_flags))
rnbd_opf |= RNBD_F_FUA;
return rnbd_opf;
}
const char *rnbd_access_mode_str(enum rnbd_access_mode mode);
#endif /* RNBD_PROTO_H */

134
drivers/block/rnbd/rnbd-srv-dev.c Normal file
View File

@ -0,0 +1,134 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* RDMA Network Block Driver
*
* Copyright (c) 2014 - 2018 ProfitBricks GmbH. All rights reserved.
* Copyright (c) 2018 - 2019 1&1 IONOS Cloud GmbH. All rights reserved.
* Copyright (c) 2019 - 2020 1&1 IONOS SE. All rights reserved.
*/
#undef pr_fmt
#define pr_fmt(fmt) KBUILD_MODNAME " L" __stringify(__LINE__) ": " fmt
#include "rnbd-srv-dev.h"
#include "rnbd-log.h"
struct rnbd_dev *rnbd_dev_open(const char *path, fmode_t flags,
struct bio_set *bs)
{
struct rnbd_dev *dev;
int ret;
dev = kzalloc(sizeof(*dev), GFP_KERNEL);
if (!dev)
return ERR_PTR(-ENOMEM);
dev->blk_open_flags = flags;
dev->bdev = blkdev_get_by_path(path, flags, THIS_MODULE);
ret = PTR_ERR_OR_ZERO(dev->bdev);
if (ret)
goto err;
dev->blk_open_flags = flags;
bdevname(dev->bdev, dev->name);
dev->ibd_bio_set = bs;
return dev;
err:
kfree(dev);
return ERR_PTR(ret);
}
void rnbd_dev_close(struct rnbd_dev *dev)
{
blkdev_put(dev->bdev, dev->blk_open_flags);
kfree(dev);
}
static void rnbd_dev_bi_end_io(struct bio *bio)
{
struct rnbd_dev_blk_io *io = bio->bi_private;
rnbd_endio(io->priv, blk_status_to_errno(bio->bi_status));
bio_put(bio);
}
/**
* rnbd_bio_map_kern - map kernel address into bio
* @data: pointer to buffer to map
* @bs: bio_set to use.
* @len: length in bytes
* @gfp_mask: allocation flags for bio allocation
*
* Map the kernel address into a bio suitable for io to a block
* device. Returns an error pointer in case of error.
*/
static struct bio *rnbd_bio_map_kern(void *data, struct bio_set *bs,
unsigned int len, gfp_t gfp_mask)
{
unsigned long kaddr = (unsigned long)data;
unsigned long end = (kaddr + len + PAGE_SIZE - 1) >> PAGE_SHIFT;
unsigned long start = kaddr >> PAGE_SHIFT;
const int nr_pages = end - start;
int offset, i;
struct bio *bio;
bio = bio_alloc_bioset(gfp_mask, nr_pages, bs);
if (!bio)
return ERR_PTR(-ENOMEM);
offset = offset_in_page(kaddr);
for (i = 0; i < nr_pages; i++) {
unsigned int bytes = PAGE_SIZE - offset;
if (len <= 0)
break;
if (bytes > len)
bytes = len;
if (bio_add_page(bio, virt_to_page(data), bytes,
offset) < bytes) {
/* we don't support partial mappings */
bio_put(bio);
return ERR_PTR(-EINVAL);
}
data += bytes;
len -= bytes;
offset = 0;
}
bio->bi_end_io = bio_put;
return bio;
}
int rnbd_dev_submit_io(struct rnbd_dev *dev, sector_t sector, void *data,
size_t len, u32 bi_size, enum rnbd_io_flags flags,
short prio, void *priv)
{
struct rnbd_dev_blk_io *io;
struct bio *bio;
/* Generate bio with pages pointing to the rdma buffer */
bio = rnbd_bio_map_kern(data, dev->ibd_bio_set, len, GFP_KERNEL);
if (IS_ERR(bio))
return PTR_ERR(bio);
io = container_of(bio, struct rnbd_dev_blk_io, bio);
io->dev = dev;
io->priv = priv;
bio->bi_end_io = rnbd_dev_bi_end_io;
bio->bi_private = io;
bio->bi_opf = rnbd_to_bio_flags(flags);
bio->bi_iter.bi_sector = sector;
bio->bi_iter.bi_size = bi_size;
bio_set_prio(bio, prio);
bio_set_dev(bio, dev->bdev);
submit_bio(bio);
return 0;
}

92
drivers/block/rnbd/rnbd-srv-dev.h Normal file
View File

@ -0,0 +1,92 @@
/* SPDX-License-Identifier: GPL-2.0-or-later */
/*
* RDMA Network Block Driver
*
* Copyright (c) 2014 - 2018 ProfitBricks GmbH. All rights reserved.
* Copyright (c) 2018 - 2019 1&1 IONOS Cloud GmbH. All rights reserved.
* Copyright (c) 2019 - 2020 1&1 IONOS SE. All rights reserved.
*/
#ifndef RNBD_SRV_DEV_H
#define RNBD_SRV_DEV_H
#include <linux/fs.h>
#include "rnbd-proto.h"
struct rnbd_dev {
struct block_device *bdev;
struct bio_set *ibd_bio_set;
fmode_t blk_open_flags;
char name[BDEVNAME_SIZE];
};
struct rnbd_dev_blk_io {
struct rnbd_dev *dev;
void *priv;
/* have to be last member for front_pad usage of bioset_init */
struct bio bio;
};
/**
* rnbd_dev_open() - Open a device
* @flags: open flags
* @bs: bio_set to use during block io,
*/
struct rnbd_dev *rnbd_dev_open(const char *path, fmode_t flags,
struct bio_set *bs);
/**
* rnbd_dev_close() - Close a device
*/
void rnbd_dev_close(struct rnbd_dev *dev);
void rnbd_endio(void *priv, int error);
static inline int rnbd_dev_get_max_segs(const struct rnbd_dev *dev)
{
return queue_max_segments(bdev_get_queue(dev->bdev));
}
static inline int rnbd_dev_get_max_hw_sects(const struct rnbd_dev *dev)
{
return queue_max_hw_sectors(bdev_get_queue(dev->bdev));
}
static inline int rnbd_dev_get_secure_discard(const struct rnbd_dev *dev)
{
return blk_queue_secure_erase(bdev_get_queue(dev->bdev));
}
static inline int rnbd_dev_get_max_discard_sects(const struct rnbd_dev *dev)
{
if (!blk_queue_discard(bdev_get_queue(dev->bdev)))
return 0;
return blk_queue_get_max_sectors(bdev_get_queue(dev->bdev),
REQ_OP_DISCARD);
}
static inline int rnbd_dev_get_discard_granularity(const struct rnbd_dev *dev)
{
return bdev_get_queue(dev->bdev)->limits.discard_granularity;
}
static inline int rnbd_dev_get_discard_alignment(const struct rnbd_dev *dev)
{
return bdev_get_queue(dev->bdev)->limits.discard_alignment;
}
/**
* rnbd_dev_submit_io() - Submit an I/O to the disk
* @dev: device to that the I/O is submitted
* @sector: address to read/write data to
* @data: I/O data to write or buffer to read I/O date into
* @len: length of @data
* @bi_size: Amount of data that will be read/written
* @prio: IO priority
* @priv: private data passed to @io_fn
*/
int rnbd_dev_submit_io(struct rnbd_dev *dev, sector_t sector, void *data,
size_t len, u32 bi_size, enum rnbd_io_flags flags,
short prio, void *priv);
#endif /* RNBD_SRV_DEV_H */

215
drivers/block/rnbd/rnbd-srv-sysfs.c Normal file
View File

@ -0,0 +1,215 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* RDMA Network Block Driver
*
* Copyright (c) 2014 - 2018 ProfitBricks GmbH. All rights reserved.
* Copyright (c) 2018 - 2019 1&1 IONOS Cloud GmbH. All rights reserved.
* Copyright (c) 2019 - 2020 1&1 IONOS SE. All rights reserved.
*/
#undef pr_fmt
#define pr_fmt(fmt) KBUILD_MODNAME " L" __stringify(__LINE__) ": " fmt
#include <uapi/linux/limits.h>
#include <linux/kobject.h>
#include <linux/sysfs.h>
#include <linux/stat.h>
#include <linux/genhd.h>
#include <linux/list.h>
#include <linux/moduleparam.h>
#include <linux/device.h>
#include "rnbd-srv.h"
static struct device *rnbd_dev;
static struct class *rnbd_dev_class;
static struct kobject *rnbd_devs_kobj;
static void rnbd_srv_dev_release(struct kobject *kobj)
{
struct rnbd_srv_dev *dev;
dev = container_of(kobj, struct rnbd_srv_dev, dev_kobj);
kfree(dev);
}
static struct kobj_type dev_ktype = {
.sysfs_ops = &kobj_sysfs_ops,
.release = rnbd_srv_dev_release
};
int rnbd_srv_create_dev_sysfs(struct rnbd_srv_dev *dev,
struct block_device *bdev,
const char *dev_name)
{
struct kobject *bdev_kobj;
int ret;
ret = kobject_init_and_add(&dev->dev_kobj, &dev_ktype,
rnbd_devs_kobj, dev_name);
if (ret)
return ret;
dev->dev_sessions_kobj = kobject_create_and_add("sessions",
&dev->dev_kobj);
if (!dev->dev_sessions_kobj)
goto put_dev_kobj;
bdev_kobj = &disk_to_dev(bdev->bd_disk)->kobj;
ret = sysfs_create_link(&dev->dev_kobj, bdev_kobj, "block_dev");
if (ret)
goto put_sess_kobj;
return 0;
put_sess_kobj:
kobject_put(dev->dev_sessions_kobj);
put_dev_kobj:
kobject_put(&dev->dev_kobj);
return ret;
}
void rnbd_srv_destroy_dev_sysfs(struct rnbd_srv_dev *dev)
{
sysfs_remove_link(&dev->dev_kobj, "block_dev");
kobject_del(dev->dev_sessions_kobj);
kobject_put(dev->dev_sessions_kobj);
kobject_del(&dev->dev_kobj);
kobject_put(&dev->dev_kobj);
}
static ssize_t read_only_show(struct kobject *kobj, struct kobj_attribute *attr,
char *page)
{
struct rnbd_srv_sess_dev *sess_dev;
sess_dev = container_of(kobj, struct rnbd_srv_sess_dev, kobj);
return scnprintf(page, PAGE_SIZE, "%d\n",
!(sess_dev->open_flags & FMODE_WRITE));
}
static struct kobj_attribute rnbd_srv_dev_session_ro_attr =
__ATTR_RO(read_only);
static ssize_t access_mode_show(struct kobject *kobj,
struct kobj_attribute *attr,
char *page)
{
struct rnbd_srv_sess_dev *sess_dev;
sess_dev = container_of(kobj, struct rnbd_srv_sess_dev, kobj);
return scnprintf(page, PAGE_SIZE, "%s\n",
rnbd_access_mode_str(sess_dev->access_mode));
}
static struct kobj_attribute rnbd_srv_dev_session_access_mode_attr =
__ATTR_RO(access_mode);
static ssize_t mapping_path_show(struct kobject *kobj,
struct kobj_attribute *attr, char *page)
{
struct rnbd_srv_sess_dev *sess_dev;
sess_dev = container_of(kobj, struct rnbd_srv_sess_dev, kobj);
return scnprintf(page, PAGE_SIZE, "%s\n", sess_dev->pathname);
}
static struct kobj_attribute rnbd_srv_dev_session_mapping_path_attr =
__ATTR_RO(mapping_path);
static struct attribute *rnbd_srv_default_dev_sessions_attrs[] = {
&rnbd_srv_dev_session_access_mode_attr.attr,
&rnbd_srv_dev_session_ro_attr.attr,
&rnbd_srv_dev_session_mapping_path_attr.attr,
NULL,
};
static struct attribute_group rnbd_srv_default_dev_session_attr_group = {
.attrs = rnbd_srv_default_dev_sessions_attrs,
};
void rnbd_srv_destroy_dev_session_sysfs(struct rnbd_srv_sess_dev *sess_dev)
{
sysfs_remove_group(&sess_dev->kobj,
&rnbd_srv_default_dev_session_attr_group);
kobject_del(&sess_dev->kobj);
kobject_put(&sess_dev->kobj);
}
static void rnbd_srv_sess_dev_release(struct kobject *kobj)
{
struct rnbd_srv_sess_dev *sess_dev;
sess_dev = container_of(kobj, struct rnbd_srv_sess_dev, kobj);
rnbd_destroy_sess_dev(sess_dev);
}
static struct kobj_type rnbd_srv_sess_dev_ktype = {
.sysfs_ops = &kobj_sysfs_ops,
.release = rnbd_srv_sess_dev_release,
};
int rnbd_srv_create_dev_session_sysfs(struct rnbd_srv_sess_dev *sess_dev)
{
int ret;
ret = kobject_init_and_add(&sess_dev->kobj, &rnbd_srv_sess_dev_ktype,
sess_dev->dev->dev_sessions_kobj, "%s",
sess_dev->sess->sessname);
if (ret)
return ret;
ret = sysfs_create_group(&sess_dev->kobj,
&rnbd_srv_default_dev_session_attr_group);
if (ret)
goto err;
return 0;
err:
kobject_put(&sess_dev->kobj);
return ret;
}
int rnbd_srv_create_sysfs_files(void)
{
int err;
rnbd_dev_class = class_create(THIS_MODULE, "rnbd-server");
if (IS_ERR(rnbd_dev_class))
return PTR_ERR(rnbd_dev_class);
rnbd_dev = device_create(rnbd_dev_class, NULL,
MKDEV(0, 0), NULL, "ctl");
if (IS_ERR(rnbd_dev)) {
err = PTR_ERR(rnbd_dev);
goto cls_destroy;
}
rnbd_devs_kobj = kobject_create_and_add("devices", &rnbd_dev->kobj);
if (!rnbd_devs_kobj) {
err = -ENOMEM;
goto dev_destroy;
}
return 0;
dev_destroy:
device_destroy(rnbd_dev_class, MKDEV(0, 0));
cls_destroy:
class_destroy(rnbd_dev_class);
return err;
}
void rnbd_srv_destroy_sysfs_files(void)
{
kobject_del(rnbd_devs_kobj);
kobject_put(rnbd_devs_kobj);
device_destroy(rnbd_dev_class, MKDEV(0, 0));
class_destroy(rnbd_dev_class);
}

844
drivers/block/rnbd/rnbd-srv.c Normal file
View File

@ -0,0 +1,844 @@
// SPDX-License-Identifier: GPL-2.0-or-later
/*
* RDMA Network Block Driver
*
* Copyright (c) 2014 - 2018 ProfitBricks GmbH. All rights reserved.
* Copyright (c) 2018 - 2019 1&1 IONOS Cloud GmbH. All rights reserved.
* Copyright (c) 2019 - 2020 1&1 IONOS SE. All rights reserved.
*/
#undef pr_fmt
#define pr_fmt(fmt) KBUILD_MODNAME " L" __stringify(__LINE__) ": " fmt
#include <linux/module.h>
#include <linux/blkdev.h>
#include "rnbd-srv.h"
#include "rnbd-srv-dev.h"
MODULE_DESCRIPTION("RDMA Network Block Device Server");
MODULE_LICENSE("GPL");
static u16 port_nr = RTRS_PORT;
module_param_named(port_nr, port_nr, ushort, 0444);
MODULE_PARM_DESC(port_nr,
"The port number the server is listening on (default: "
__stringify(RTRS_PORT)")");
#define DEFAULT_DEV_SEARCH_PATH "/"
static char dev_search_path[PATH_MAX] = DEFAULT_DEV_SEARCH_PATH;
static int dev_search_path_set(const char *val, const struct kernel_param *kp)
{
const char *p = strrchr(val, '\n') ? : val + strlen(val);
if (strlen(val) >= sizeof(dev_search_path))
return -EINVAL;
snprintf(dev_search_path, sizeof(dev_search_path), "%.*s",
(int)(p - val), val);
pr_info("dev_search_path changed to '%s'\n", dev_search_path);
return 0;
}
static struct kparam_string dev_search_path_kparam_str = {
.maxlen = sizeof(dev_search_path),
.string = dev_search_path
};
static const struct kernel_param_ops dev_search_path_ops = {
.set = dev_search_path_set,
.get = param_get_string,
};
module_param_cb(dev_search_path, &dev_search_path_ops,
&dev_search_path_kparam_str, 0444);
MODULE_PARM_DESC(dev_search_path,
"Sets the dev_search_path. When a device is mapped this path is prepended to the device path from the map device operation. If %SESSNAME% is specified in a path, then device will be searched in a session namespace. (default: "
DEFAULT_DEV_SEARCH_PATH ")");
static DEFINE_MUTEX(sess_lock);
static DEFINE_SPINLOCK(dev_lock);
static LIST_HEAD(sess_list);
static LIST_HEAD(dev_list);
struct rnbd_io_private {
struct rtrs_srv_op *id;
struct rnbd_srv_sess_dev *sess_dev;
};
static void rnbd_sess_dev_release(struct kref *kref)
{
struct rnbd_srv_sess_dev *sess_dev;
sess_dev = container_of(kref, struct rnbd_srv_sess_dev, kref);
complete(sess_dev->destroy_comp);
}
static inline void rnbd_put_sess_dev(struct rnbd_srv_sess_dev *sess_dev)
{
kref_put(&sess_dev->kref, rnbd_sess_dev_release);
}
void rnbd_endio(void *priv, int error)
{
struct rnbd_io_private *rnbd_priv = priv;
struct rnbd_srv_sess_dev *sess_dev = rnbd_priv->sess_dev;
rnbd_put_sess_dev(sess_dev);
rtrs_srv_resp_rdma(rnbd_priv->id, error);
kfree(priv);
}
static struct rnbd_srv_sess_dev *
rnbd_get_sess_dev(int dev_id, struct rnbd_srv_session *srv_sess)
{
struct rnbd_srv_sess_dev *sess_dev;
int ret = 0;
rcu_read_lock();
sess_dev = xa_load(&srv_sess->index_idr, dev_id);
if (likely(sess_dev))
ret = kref_get_unless_zero(&sess_dev->kref);
rcu_read_unlock();
if (!sess_dev || !ret)
return ERR_PTR(-ENXIO);
return sess_dev;
}
static int process_rdma(struct rtrs_srv *sess,
struct rnbd_srv_session *srv_sess,
struct rtrs_srv_op *id, void *data, u32 datalen,
const void *usr, size_t usrlen)
{
const struct rnbd_msg_io *msg = usr;
struct rnbd_io_private *priv;
struct rnbd_srv_sess_dev *sess_dev;
u32 dev_id;
int err;
priv = kmalloc(sizeof(*priv), GFP_KERNEL);
if (!priv)
return -ENOMEM;
dev_id = le32_to_cpu(msg->device_id);
sess_dev = rnbd_get_sess_dev(dev_id, srv_sess);
if (IS_ERR(sess_dev)) {
pr_err_ratelimited("Got I/O request on session %s for unknown device id %d\n",
srv_sess->sessname, dev_id);
err = -ENOTCONN;
goto err;
}
priv->sess_dev = sess_dev;
priv->id = id;
err = rnbd_dev_submit_io(sess_dev->rnbd_dev, le64_to_cpu(msg->sector),
data, datalen, le32_to_cpu(msg->bi_size),
le32_to_cpu(msg->rw),
srv_sess->ver < RNBD_PROTO_VER_MAJOR ||
usrlen < sizeof(*msg) ?
0 : le16_to_cpu(msg->prio), priv);
if (unlikely(err)) {
rnbd_srv_err(sess_dev, "Submitting I/O to device failed, err: %d\n",
err);
goto sess_dev_put;
}
return 0;
sess_dev_put:
rnbd_put_sess_dev(sess_dev);
err:
kfree(priv);
return err;
}
static void destroy_device(struct rnbd_srv_dev *dev)
{
WARN_ONCE(!list_empty(&dev->sess_dev_list),
"Device %s is being destroyed but still in use!\n",
dev->id);
spin_lock(&dev_lock);
list_del(&dev->list);
spin_unlock(&dev_lock);
mutex_destroy(&dev->lock);
if (dev->dev_kobj.state_in_sysfs)
/*
* Destroy kobj only if it was really created.
*/
rnbd_srv_destroy_dev_sysfs(dev);
else
kfree(dev);
}
static void destroy_device_cb(struct kref *kref)
{
struct rnbd_srv_dev *dev;
dev = container_of(kref, struct rnbd_srv_dev, kref);
destroy_device(dev);
}
static void rnbd_put_srv_dev(struct rnbd_srv_dev *dev)
{
kref_put(&dev->kref, destroy_device_cb);
}
void rnbd_destroy_sess_dev(struct rnbd_srv_sess_dev *sess_dev)
{
DECLARE_COMPLETION_ONSTACK(dc);
xa_erase(&sess_dev->sess->index_idr, sess_dev->device_id);
synchronize_rcu();
sess_dev->destroy_comp = &dc;
rnbd_put_sess_dev(sess_dev);
wait_for_completion(&dc); /* wait for inflights to drop to zero */
rnbd_dev_close(sess_dev->rnbd_dev);
list_del(&sess_dev->sess_list);
mutex_lock(&sess_dev->dev->lock);
list_del(&sess_dev->dev_list);
if (sess_dev->open_flags & FMODE_WRITE)
sess_dev->dev->open_write_cnt--;
mutex_unlock(&sess_dev->dev->lock);
rnbd_put_srv_dev(sess_dev->dev);
rnbd_srv_info(sess_dev, "Device closed\n");
kfree(sess_dev);
}
static void destroy_sess(struct rnbd_srv_session *srv_sess)
{
struct rnbd_srv_sess_dev *sess_dev, *tmp;
if (list_empty(&srv_sess->sess_dev_list))
goto out;
mutex_lock(&srv_sess->lock);
list_for_each_entry_safe(sess_dev, tmp, &srv_sess->sess_dev_list,
sess_list)
rnbd_srv_destroy_dev_session_sysfs(sess_dev);
mutex_unlock(&srv_sess->lock);
out:
xa_destroy(&srv_sess->index_idr);
bioset_exit(&srv_sess->sess_bio_set);
pr_info("RTRS Session %s disconnected\n", srv_sess->sessname);
mutex_lock(&sess_lock);
list_del(&srv_sess->list);
mutex_unlock(&sess_lock);
mutex_destroy(&srv_sess->lock);
kfree(srv_sess);
}
static int create_sess(struct rtrs_srv *rtrs)
{
struct rnbd_srv_session *srv_sess;
char sessname[NAME_MAX];
int err;
err = rtrs_srv_get_sess_name(rtrs, sessname, sizeof(sessname));
if (err) {
pr_err("rtrs_srv_get_sess_name(%s): %d\n", sessname, err);
return err;
}
srv_sess = kzalloc(sizeof(*srv_sess), GFP_KERNEL);
if (!srv_sess)
return -ENOMEM;
srv_sess->queue_depth = rtrs_srv_get_queue_depth(rtrs);
err = bioset_init(&srv_sess->sess_bio_set, srv_sess->queue_depth,
offsetof(struct rnbd_dev_blk_io, bio),
BIOSET_NEED_BVECS);
if (err) {
pr_err("Allocating srv_session for session %s failed\n",
sessname);
kfree(srv_sess);
return err;
}
xa_init_flags(&srv_sess->index_idr, XA_FLAGS_ALLOC);
INIT_LIST_HEAD(&srv_sess->sess_dev_list);
mutex_init(&srv_sess->lock);
mutex_lock(&sess_lock);
list_add(&srv_sess->list, &sess_list);
mutex_unlock(&sess_lock);
srv_sess->rtrs = rtrs;
strlcpy(srv_sess->sessname, sessname, sizeof(srv_sess->sessname));
rtrs_srv_set_sess_priv(rtrs, srv_sess);
return 0;
}
static int rnbd_srv_link_ev(struct rtrs_srv *rtrs,
enum rtrs_srv_link_ev ev, void *priv)
{
struct rnbd_srv_session *srv_sess = priv;
switch (ev) {
case RTRS_SRV_LINK_EV_CONNECTED:
return create_sess(rtrs);
case RTRS_SRV_LINK_EV_DISCONNECTED:
if (WARN_ON_ONCE(!srv_sess))
return -EINVAL;
destroy_sess(srv_sess);
return 0;
default:
pr_warn("Received unknown RTRS session event %d from session %s\n",
ev, srv_sess->sessname);
return -EINVAL;
}
}
static int process_msg_close(struct rtrs_srv *rtrs,
struct rnbd_srv_session *srv_sess,
void *data, size_t datalen, const void *usr,
size_t usrlen)
{
const struct rnbd_msg_close *close_msg = usr;
struct rnbd_srv_sess_dev *sess_dev;
sess_dev = rnbd_get_sess_dev(le32_to_cpu(close_msg->device_id),
srv_sess);
if (IS_ERR(sess_dev))
return 0;
rnbd_put_sess_dev(sess_dev);
mutex_lock(&srv_sess->lock);
rnbd_srv_destroy_dev_session_sysfs(sess_dev);
mutex_unlock(&srv_sess->lock);
return 0;
}
static int process_msg_open(struct rtrs_srv *rtrs,
struct rnbd_srv_session *srv_sess,
const void *msg, size_t len,
void *data, size_t datalen);
static int process_msg_sess_info(struct rtrs_srv *rtrs,
struct rnbd_srv_session *srv_sess,
const void *msg, size_t len,
void *data, size_t datalen);
static int rnbd_srv_rdma_ev(struct rtrs_srv *rtrs, void *priv,
struct rtrs_srv_op *id, int dir,
void *data, size_t datalen, const void *usr,
size_t usrlen)
{
struct rnbd_srv_session *srv_sess = priv;
const struct rnbd_msg_hdr *hdr = usr;
int ret = 0;
u16 type;
if (WARN_ON_ONCE(!srv_sess))
return -ENODEV;
type = le16_to_cpu(hdr->type);
switch (type) {
case RNBD_MSG_IO:
return process_rdma(rtrs, srv_sess, id, data, datalen, usr,
usrlen);
case RNBD_MSG_CLOSE:
ret = process_msg_close(rtrs, srv_sess, data, datalen,
usr, usrlen);
break;
case RNBD_MSG_OPEN:
ret = process_msg_open(rtrs, srv_sess, usr, usrlen,
data, datalen);
break;
case RNBD_MSG_SESS_INFO:
ret = process_msg_sess_info(rtrs, srv_sess, usr, usrlen,
data, datalen);
break;
default:
pr_warn("Received unexpected message type %d with dir %d from session %s\n",
type, dir, srv_sess->sessname);
return -EINVAL;
}
rtrs_srv_resp_rdma(id, ret);
return 0;
}
static struct rnbd_srv_sess_dev
*rnbd_sess_dev_alloc(struct rnbd_srv_session *srv_sess)
{
struct rnbd_srv_sess_dev *sess_dev;
int error;
sess_dev = kzalloc(sizeof(*sess_dev), GFP_KERNEL);
if (!sess_dev)
return ERR_PTR(-ENOMEM);
error = xa_alloc(&srv_sess->index_idr, &sess_dev->device_id, sess_dev,
xa_limit_32b, GFP_NOWAIT);
if (error < 0) {
pr_warn("Allocating idr failed, err: %d\n", error);
kfree(sess_dev);
return ERR_PTR(error);
}
return sess_dev;
}
static struct rnbd_srv_dev *rnbd_srv_init_srv_dev(const char *id)
{
struct rnbd_srv_dev *dev;
dev = kzalloc(sizeof(*dev), GFP_KERNEL);
if (!dev)
return ERR_PTR(-ENOMEM);
strlcpy(dev->id, id, sizeof(dev->id));
kref_init(&dev->kref);
INIT_LIST_HEAD(&dev->sess_dev_list);
mutex_init(&dev->lock);
return dev;
}
static struct rnbd_srv_dev *
rnbd_srv_find_or_add_srv_dev(struct rnbd_srv_dev *new_dev)
{
struct rnbd_srv_dev *dev;
spin_lock(&dev_lock);
list_for_each_entry(dev, &dev_list, list) {
if (!strncmp(dev->id, new_dev->id, sizeof(dev->id))) {
if (!kref_get_unless_zero(&dev->kref))
/*
* We lost the race, device is almost dead.
* Continue traversing to find a valid one.
*/
continue;
spin_unlock(&dev_lock);
return dev;
}
}
list_add(&new_dev->list, &dev_list);
spin_unlock(&dev_lock);
return new_dev;
}
static int rnbd_srv_check_update_open_perm(struct rnbd_srv_dev *srv_dev,
struct rnbd_srv_session *srv_sess,
enum rnbd_access_mode access_mode)
{
int ret = -EPERM;
mutex_lock(&srv_dev->lock);
switch (access_mode) {
case RNBD_ACCESS_RO:
ret = 0;
break;
case RNBD_ACCESS_RW:
if (srv_dev->open_write_cnt == 0) {
srv_dev->open_write_cnt++;
ret = 0;
} else {
pr_err("Mapping device '%s' for session %s with RW permissions failed. Device already opened as 'RW' by %d client(s), access mode %s.\n",
srv_dev->id, srv_sess->sessname,
srv_dev->open_write_cnt,
rnbd_access_mode_str(access_mode));
}
break;
case RNBD_ACCESS_MIGRATION:
if (srv_dev->open_write_cnt < 2) {
srv_dev->open_write_cnt++;
ret = 0;
} else {
pr_err("Mapping device '%s' for session %s with migration permissions failed. Device already opened as 'RW' by %d client(s), access mode %s.\n",
srv_dev->id, srv_sess->sessname,
srv_dev->open_write_cnt,
rnbd_access_mode_str(access_mode));
}
break;
default:
pr_err("Received mapping request for device '%s' on session %s with invalid access mode: %d\n",
srv_dev->id, srv_sess->sessname, access_mode);
ret = -EINVAL;
}
mutex_unlock(&srv_dev->lock);
return ret;
}
static struct rnbd_srv_dev *
rnbd_srv_get_or_create_srv_dev(struct rnbd_dev *rnbd_dev,
struct rnbd_srv_session *srv_sess,
enum rnbd_access_mode access_mode)
{
int ret;
struct rnbd_srv_dev *new_dev, *dev;
new_dev = rnbd_srv_init_srv_dev(rnbd_dev->name);
if (IS_ERR(new_dev))
return new_dev;
dev = rnbd_srv_find_or_add_srv_dev(new_dev);
if (dev != new_dev)
kfree(new_dev);
ret = rnbd_srv_check_update_open_perm(dev, srv_sess, access_mode);
if (ret) {
rnbd_put_srv_dev(dev);
return ERR_PTR(ret);
}
return dev;
}
static void rnbd_srv_fill_msg_open_rsp(struct rnbd_msg_open_rsp *rsp,
struct rnbd_srv_sess_dev *sess_dev)
{
struct rnbd_dev *rnbd_dev = sess_dev->rnbd_dev;
rsp->hdr.type = cpu_to_le16(RNBD_MSG_OPEN_RSP);
rsp->device_id =
cpu_to_le32(sess_dev->device_id);
rsp->nsectors =
cpu_to_le64(get_capacity(rnbd_dev->bdev->bd_disk));
rsp->logical_block_size =
cpu_to_le16(bdev_logical_block_size(rnbd_dev->bdev));
rsp->physical_block_size =
cpu_to_le16(bdev_physical_block_size(rnbd_dev->bdev));
rsp->max_segments =
cpu_to_le16(rnbd_dev_get_max_segs(rnbd_dev));
rsp->max_hw_sectors =
cpu_to_le32(rnbd_dev_get_max_hw_sects(rnbd_dev));
rsp->max_write_same_sectors =
cpu_to_le32(bdev_write_same(rnbd_dev->bdev));
rsp->max_discard_sectors =
cpu_to_le32(rnbd_dev_get_max_discard_sects(rnbd_dev));
rsp->discard_granularity =
cpu_to_le32(rnbd_dev_get_discard_granularity(rnbd_dev));
rsp->discard_alignment =
cpu_to_le32(rnbd_dev_get_discard_alignment(rnbd_dev));
rsp->secure_discard =
cpu_to_le16(rnbd_dev_get_secure_discard(rnbd_dev));
rsp->rotational =
!blk_queue_nonrot(bdev_get_queue(rnbd_dev->bdev));
}
static struct rnbd_srv_sess_dev *
rnbd_srv_create_set_sess_dev(struct rnbd_srv_session *srv_sess,
const struct rnbd_msg_open *open_msg,
struct rnbd_dev *rnbd_dev, fmode_t open_flags,
struct rnbd_srv_dev *srv_dev)
{
struct rnbd_srv_sess_dev *sdev = rnbd_sess_dev_alloc(srv_sess);
if (IS_ERR(sdev))
return sdev;
kref_init(&sdev->kref);
strlcpy(sdev->pathname, open_msg->dev_name, sizeof(sdev->pathname));
sdev->rnbd_dev = rnbd_dev;
sdev->sess = srv_sess;
sdev->dev = srv_dev;
sdev->open_flags = open_flags;
sdev->access_mode = open_msg->access_mode;
return sdev;
}
static char *rnbd_srv_get_full_path(struct rnbd_srv_session *srv_sess,
const char *dev_name)
{
char *full_path;
char *a, *b;
full_path = kmalloc(PATH_MAX, GFP_KERNEL);
if (!full_path)
return ERR_PTR(-ENOMEM);
/*
* Replace %SESSNAME% with a real session name in order to
* create device namespace.
*/
a = strnstr(dev_search_path, "%SESSNAME%", sizeof(dev_search_path));
if (a) {
int len = a - dev_search_path;
len = snprintf(full_path, PATH_MAX, "%.*s/%s/%s", len,
dev_search_path, srv_sess->sessname, dev_name);
if (len >= PATH_MAX) {
pr_err("Too long path: %s, %s, %s\n",
dev_search_path, srv_sess->sessname, dev_name);
kfree(full_path);
return ERR_PTR(-EINVAL);
}
} else {
snprintf(full_path, PATH_MAX, "%s/%s",
dev_search_path, dev_name);
}
/* eliminitate duplicated slashes */
a = strchr(full_path, '/');
b = a;
while (*b != '\0') {
if (*b == '/' && *a == '/') {
b++;
} else {
a++;
*a = *b;
b++;
}
}
a++;
*a = '\0';
return full_path;
}
static int process_msg_sess_info(struct rtrs_srv *rtrs,
struct rnbd_srv_session *srv_sess,
const void *msg, size_t len,
void *data, size_t datalen)
{
const struct rnbd_msg_sess_info *sess_info_msg = msg;
struct rnbd_msg_sess_info_rsp *rsp = data;
srv_sess->ver = min_t(u8, sess_info_msg->ver, RNBD_PROTO_VER_MAJOR);
pr_debug("Session %s using protocol version %d (client version: %d, server version: %d)\n",
srv_sess->sessname, srv_sess->ver,
sess_info_msg->ver, RNBD_PROTO_VER_MAJOR);
rsp->hdr.type = cpu_to_le16(RNBD_MSG_SESS_INFO_RSP);
rsp->ver = srv_sess->ver;
return 0;
}
/**
* find_srv_sess_dev() - a dev is already opened by this name
* @srv_sess: the session to search.
* @dev_name: string containing the name of the device.
*
* Return struct rnbd_srv_sess_dev if srv_sess already opened the dev_name
* NULL if the session didn't open the device yet.
*/
static struct rnbd_srv_sess_dev *
find_srv_sess_dev(struct rnbd_srv_session *srv_sess, const char *dev_name)
{
struct rnbd_srv_sess_dev *sess_dev;
if (list_empty(&srv_sess->sess_dev_list))
return NULL;
list_for_each_entry(sess_dev, &srv_sess->sess_dev_list, sess_list)
if (!strcmp(sess_dev->pathname, dev_name))
return sess_dev;
return NULL;
}
static int process_msg_open(struct rtrs_srv *rtrs,
struct rnbd_srv_session *srv_sess,
const void *msg, size_t len,
void *data, size_t datalen)
{
int ret;
struct rnbd_srv_dev *srv_dev;
struct rnbd_srv_sess_dev *srv_sess_dev;
const struct rnbd_msg_open *open_msg = msg;
fmode_t open_flags;
char *full_path;
struct rnbd_dev *rnbd_dev;
struct rnbd_msg_open_rsp *rsp = data;
pr_debug("Open message received: session='%s' path='%s' access_mode=%d\n",
srv_sess->sessname, open_msg->dev_name,
open_msg->access_mode);
open_flags = FMODE_READ;
if (open_msg->access_mode != RNBD_ACCESS_RO)
open_flags |= FMODE_WRITE;
mutex_lock(&srv_sess->lock);
srv_sess_dev = find_srv_sess_dev(srv_sess, open_msg->dev_name);
if (srv_sess_dev)
goto fill_response;
if ((strlen(dev_search_path) + strlen(open_msg->dev_name))
>= PATH_MAX) {
pr_err("Opening device for session %s failed, device path too long. '%s/%s' is longer than PATH_MAX (%d)\n",
srv_sess->sessname, dev_search_path, open_msg->dev_name,
PATH_MAX);
ret = -EINVAL;
goto reject;
}
if (strstr(open_msg->dev_name, "..")) {
pr_err("Opening device for session %s failed, device path %s contains relative path ..\n",
srv_sess->sessname, open_msg->dev_name);
ret = -EINVAL;
goto reject;
}
full_path = rnbd_srv_get_full_path(srv_sess, open_msg->dev_name);
if (IS_ERR(full_path)) {
ret = PTR_ERR(full_path);
pr_err("Opening device '%s' for client %s failed, failed to get device full path, err: %d\n",
open_msg->dev_name, srv_sess->sessname, ret);
goto reject;
}
rnbd_dev = rnbd_dev_open(full_path, open_flags,
&srv_sess->sess_bio_set);
if (IS_ERR(rnbd_dev)) {
pr_err("Opening device '%s' on session %s failed, failed to open the block device, err: %ld\n",
full_path, srv_sess->sessname, PTR_ERR(rnbd_dev));
ret = PTR_ERR(rnbd_dev);
goto free_path;
}
srv_dev = rnbd_srv_get_or_create_srv_dev(rnbd_dev, srv_sess,
open_msg->access_mode);
if (IS_ERR(srv_dev)) {
pr_err("Opening device '%s' on session %s failed, creating srv_dev failed, err: %ld\n",
full_path, srv_sess->sessname, PTR_ERR(srv_dev));
ret = PTR_ERR(srv_dev);
goto rnbd_dev_close;
}
srv_sess_dev = rnbd_srv_create_set_sess_dev(srv_sess, open_msg,
rnbd_dev, open_flags,
srv_dev);
if (IS_ERR(srv_sess_dev)) {
pr_err("Opening device '%s' on session %s failed, creating sess_dev failed, err: %ld\n",
full_path, srv_sess->sessname, PTR_ERR(srv_sess_dev));
ret = PTR_ERR(srv_sess_dev);
goto srv_dev_put;
}
/* Create the srv_dev sysfs files if they haven't been created yet. The
* reason to delay the creation is not to create the sysfs files before
* we are sure the device can be opened.
*/
mutex_lock(&srv_dev->lock);
if (!srv_dev->dev_kobj.state_in_sysfs) {
ret = rnbd_srv_create_dev_sysfs(srv_dev, rnbd_dev->bdev,
rnbd_dev->name);
if (ret) {
mutex_unlock(&srv_dev->lock);
rnbd_srv_err(srv_sess_dev,
"Opening device failed, failed to create device sysfs files, err: %d\n",
ret);
goto free_srv_sess_dev;
}
}
ret = rnbd_srv_create_dev_session_sysfs(srv_sess_dev);
if (ret) {
mutex_unlock(&srv_dev->lock);
rnbd_srv_err(srv_sess_dev,
"Opening device failed, failed to create dev client sysfs files, err: %d\n",
ret);
goto free_srv_sess_dev;
}
list_add(&srv_sess_dev->dev_list, &srv_dev->sess_dev_list);
mutex_unlock(&srv_dev->lock);
list_add(&srv_sess_dev->sess_list, &srv_sess->sess_dev_list);
rnbd_srv_info(srv_sess_dev, "Opened device '%s'\n", srv_dev->id);
kfree(full_path);
fill_response:
rnbd_srv_fill_msg_open_rsp(rsp, srv_sess_dev);
mutex_unlock(&srv_sess->lock);
return 0;
free_srv_sess_dev:
xa_erase(&srv_sess->index_idr, srv_sess_dev->device_id);
synchronize_rcu();
kfree(srv_sess_dev);
srv_dev_put:
if (open_msg->access_mode != RNBD_ACCESS_RO) {
mutex_lock(&srv_dev->lock);
srv_dev->open_write_cnt--;
mutex_unlock(&srv_dev->lock);
}
rnbd_put_srv_dev(srv_dev);
rnbd_dev_close:
rnbd_dev_close(rnbd_dev);
free_path:
kfree(full_path);
reject:
mutex_unlock(&srv_sess->lock);
return ret;
}
static struct rtrs_srv_ctx *rtrs_ctx;
static struct rtrs_srv_ops rtrs_ops;
static int __init rnbd_srv_init_module(void)
{
int err;
BUILD_BUG_ON(sizeof(struct rnbd_msg_hdr) != 4);
BUILD_BUG_ON(sizeof(struct rnbd_msg_sess_info) != 36);
BUILD_BUG_ON(sizeof(struct rnbd_msg_sess_info_rsp) != 36);
BUILD_BUG_ON(sizeof(struct rnbd_msg_open) != 264);
BUILD_BUG_ON(sizeof(struct rnbd_msg_close) != 8);
BUILD_BUG_ON(sizeof(struct rnbd_msg_open_rsp) != 56);
rtrs_ops = (struct rtrs_srv_ops) {
.rdma_ev = rnbd_srv_rdma_ev,
.link_ev = rnbd_srv_link_ev,
};
rtrs_ctx = rtrs_srv_open(&rtrs_ops, port_nr);
if (IS_ERR(rtrs_ctx)) {
err = PTR_ERR(rtrs_ctx);
pr_err("rtrs_srv_open(), err: %d\n", err);
return err;
}
err = rnbd_srv_create_sysfs_files();
if (err) {
pr_err("rnbd_srv_create_sysfs_files(), err: %d\n", err);
rtrs_srv_close(rtrs_ctx);
return err;
}
return 0;
}
static void __exit rnbd_srv_cleanup_module(void)
{
rtrs_srv_close(rtrs_ctx);
WARN_ON(!list_empty(&sess_list));
rnbd_srv_destroy_sysfs_files();
}
module_init(rnbd_srv_init_module);
module_exit(rnbd_srv_cleanup_module);

78
drivers/block/rnbd/rnbd-srv.h Normal file
View File

@ -0,0 +1,78 @@
/* SPDX-License-Identifier: GPL-2.0-or-later */
/*
* RDMA Network Block Driver
*
* Copyright (c) 2014 - 2018 ProfitBricks GmbH. All rights reserved.
* Copyright (c) 2018 - 2019 1&1 IONOS Cloud GmbH. All rights reserved.
* Copyright (c) 2019 - 2020 1&1 IONOS SE. All rights reserved.
*/
#ifndef RNBD_SRV_H
#define RNBD_SRV_H
#include <linux/types.h>
#include <linux/idr.h>
#include <linux/kref.h>
#include <rtrs.h>
#include "rnbd-proto.h"
#include "rnbd-log.h"
struct rnbd_srv_session {
/* Entry inside global sess_list */
struct list_head list;
struct rtrs_srv *rtrs;
char sessname[NAME_MAX];
int queue_depth;
struct bio_set sess_bio_set;
struct xarray index_idr;
/* List of struct rnbd_srv_sess_dev */
struct list_head sess_dev_list;
struct mutex lock;
u8 ver;
};
struct rnbd_srv_dev {
/* Entry inside global dev_list */
struct list_head list;
struct kobject dev_kobj;
struct kobject *dev_sessions_kobj;
struct kref kref;
char id[NAME_MAX];
/* List of rnbd_srv_sess_dev structs */
struct list_head sess_dev_list;
struct mutex lock;
int open_write_cnt;
};
/* Structure which binds N devices and N sessions */
struct rnbd_srv_sess_dev {
/* Entry inside rnbd_srv_dev struct */
struct list_head dev_list;
/* Entry inside rnbd_srv_session struct */
struct list_head sess_list;
struct rnbd_dev *rnbd_dev;
struct rnbd_srv_session *sess;
struct rnbd_srv_dev *dev;
struct kobject kobj;
u32 device_id;
fmode_t open_flags;
struct kref kref;
struct completion *destroy_comp;
char pathname[NAME_MAX];
enum rnbd_access_mode access_mode;
};
/* rnbd-srv-sysfs.c */
int rnbd_srv_create_dev_sysfs(struct rnbd_srv_dev *dev,
struct block_device *bdev,
const char *dir_name);
void rnbd_srv_destroy_dev_sysfs(struct rnbd_srv_dev *dev);
int rnbd_srv_create_dev_session_sysfs(struct rnbd_srv_sess_dev *sess_dev);
void rnbd_srv_destroy_dev_session_sysfs(struct rnbd_srv_sess_dev *sess_dev);
int rnbd_srv_create_sysfs_files(void);
void rnbd_srv_destroy_sysfs_files(void);
void rnbd_destroy_sess_dev(struct rnbd_srv_sess_dev *sess_dev);
#endif /* RNBD_SRV_H */

21
drivers/char/ipmi/bt-bmc.c
View File

@ -399,15 +399,15 @@ static int bt_bmc_config_irq(struct bt_bmc *bt_bmc,
struct device *dev = &pdev->dev;
int rc;
bt_bmc->irq = platform_get_irq(pdev, 0);
if (!bt_bmc->irq)
return -ENODEV;
bt_bmc->irq = platform_get_irq_optional(pdev, 0);
if (bt_bmc->irq < 0)
return bt_bmc->irq;
rc = devm_request_irq(dev, bt_bmc->irq, bt_bmc_irq, IRQF_SHARED,
DEVICE_NAME, bt_bmc);
if (rc < 0) {
dev_warn(dev, "Unable to request IRQ %d\n", bt_bmc->irq);
bt_bmc->irq = 0;
bt_bmc->irq = rc;
return rc;
}
@ -430,9 +430,6 @@ static int bt_bmc_probe(struct platform_device *pdev)
struct device *dev;
int rc;
if (!pdev || !pdev->dev.of_node)
return -ENODEV;
dev = &pdev->dev;
dev_info(dev, "Found bt bmc device\n");
@ -466,9 +463,9 @@ static int bt_bmc_probe(struct platform_device *pdev)
init_waitqueue_head(&bt_bmc->queue);
bt_bmc->miscdev.minor = MISC_DYNAMIC_MINOR,
bt_bmc->miscdev.name = DEVICE_NAME,
bt_bmc->miscdev.fops = &bt_bmc_fops,
bt_bmc->miscdev.parent = dev;
bt_bmc->miscdev.name = DEVICE_NAME,
bt_bmc->miscdev.fops = &bt_bmc_fops,
bt_bmc->miscdev.parent = dev;
rc = misc_register(&bt_bmc->miscdev);
if (rc) {
dev_err(dev, "Unable to register misc device\n");
@ -477,7 +474,7 @@ static int bt_bmc_probe(struct platform_device *pdev)
bt_bmc_config_irq(bt_bmc, pdev);
if (bt_bmc->irq) {
if (bt_bmc->irq >= 0) {
dev_info(dev, "Using IRQ %d\n", bt_bmc->irq);
} else {
dev_info(dev, "No IRQ; using timer\n");
@ -503,7 +500,7 @@ static int bt_bmc_remove(struct platform_device *pdev)
struct bt_bmc *bt_bmc = dev_get_drvdata(&pdev->dev);
misc_deregister(&bt_bmc->miscdev);
if (!bt_bmc->irq)
if (bt_bmc->irq < 0)
del_timer_sync(&bt_bmc->poll_timer);
return 0;
}

9
drivers/char/ipmi/ipmi_msghandler.c
View File

@ -33,6 +33,7 @@
#include <linux/workqueue.h>
#include <linux/uuid.h>
#include <linux/nospec.h>
#include <linux/vmalloc.h>
#define IPMI_DRIVER_VERSION "39.2"
@ -1153,7 +1154,7 @@ static void free_user_work(struct work_struct *work)
remove_work);
cleanup_srcu_struct(&user->release_barrier);
kfree(user);
vfree(user);
}
int ipmi_create_user(unsigned int if_num,
@ -1185,7 +1186,7 @@ int ipmi_create_user(unsigned int if_num,
if (rv)
return rv;
new_user = kmalloc(sizeof(*new_user), GFP_KERNEL);
new_user = vzalloc(sizeof(*new_user));
if (!new_user)
return -ENOMEM;
@ -1232,7 +1233,7 @@ int ipmi_create_user(unsigned int if_num,
out_kfree:
srcu_read_unlock(&ipmi_interfaces_srcu, index);
kfree(new_user);
vfree(new_user);
return rv;
}
EXPORT_SYMBOL(ipmi_create_user);
@ -3171,7 +3172,7 @@ static void guid_handler(struct ipmi_smi *intf, struct ipmi_recv_msg *msg)
goto out;
}
guid_copy(&bmc->fetch_guid, (guid_t *)(msg->msg.data + 1));
import_guid(&bmc->fetch_guid, msg->msg.data + 1);
/*
* Make sure the guid data is available before setting
* dyn_guid_set.

2
drivers/char/ipmi/ipmi_si_platform.c
View File

@ -393,6 +393,8 @@ static int acpi_ipmi_probe(struct platform_device *pdev)
dev_info(io.dev, "%pR regsize %d spacing %d irq %d\n",
res, io.regsize, io.regspacing, io.irq);
request_module("acpi_ipmi");
return ipmi_si_add_smi(&io);
err_free:

24
drivers/char/ipmi/ipmi_ssif.c
View File

@ -189,8 +189,6 @@ struct ssif_addr_info {
struct device *dev;
struct i2c_client *client;
struct i2c_client *added_client;
struct mutex clients_mutex;
struct list_head clients;
@ -1472,6 +1470,7 @@ static bool check_acpi(struct ssif_info *ssif_info, struct device *dev)
if (acpi_handle) {
ssif_info->addr_source = SI_ACPI;
ssif_info->addr_info.acpi_info.acpi_handle = acpi_handle;
request_module("acpi_ipmi");
return true;
}
#endif
@ -1940,21 +1939,6 @@ static int ssif_probe(struct i2c_client *client, const struct i2c_device_id *id)
goto out;
}
static int ssif_adapter_handler(struct device *adev, void *opaque)
{
struct ssif_addr_info *addr_info = opaque;
if (adev->type != &i2c_adapter_type)
return 0;
addr_info->added_client = i2c_new_client_device(to_i2c_adapter(adev),
&addr_info->binfo);
if (!addr_info->adapter_name)
return 1; /* Only try the first I2C adapter by default. */
return 0;
}
static int new_ssif_client(int addr, char *adapter_name,
int debug, int slave_addr,
enum ipmi_addr_src addr_src,
@ -1998,9 +1982,7 @@ static int new_ssif_client(int addr, char *adapter_name,
list_add_tail(&addr_info->link, &ssif_infos);
if (initialized)
i2c_for_each_dev(addr_info, ssif_adapter_handler);
/* Otherwise address list will get it */
/* Address list will get it */
out_unlock:
mutex_unlock(&ssif_infos_mutex);
@ -2120,8 +2102,6 @@ static int ssif_platform_remove(struct platform_device *dev)
return 0;
mutex_lock(&ssif_infos_mutex);
i2c_unregister_device(addr_info->added_client);
list_del(&addr_info->link);
kfree(addr_info);
mutex_unlock(&ssif_infos_mutex);

24
drivers/gpio/Kconfig
View File

@ -73,6 +73,10 @@ config GPIO_GENERIC
depends on HAS_IOMEM # Only for IOMEM drivers
tristate
config GPIO_REGMAP
depends on REGMAP
tristate
# put drivers in the right section, in alphabetical order
# This symbol is selected by both I2C and SPI expanders
@ -422,7 +426,7 @@ config GPIO_OMAP
Say yes here to enable GPIO support for TI OMAP SoCs.
config GPIO_PL061
bool "PrimeCell PL061 GPIO support"
tristate "PrimeCell PL061 GPIO support"
depends on ARM_AMBA
select IRQ_DOMAIN
select GPIOLIB_IRQCHIP
@ -439,7 +443,7 @@ config GPIO_PMIC_EIC_SPRD
config GPIO_PXA
bool "PXA GPIO support"
depends on ARCH_PXA || ARCH_MMP
depends on ARCH_PXA || ARCH_MMP || COMPILE_TEST
help
Say yes here to support the PXA GPIO device
@ -638,7 +642,7 @@ config GPIO_XGENE
config GPIO_XGENE_SB
tristate "APM X-Gene GPIO standby controller support"
depends on ARCH_XGENE && OF_GPIO
depends on (ARCH_XGENE || COMPILE_TEST)
select GPIO_GENERIC
select GPIOLIB_IRQCHIP
select IRQ_DOMAIN_HIERARCHY
@ -952,7 +956,7 @@ config GPIO_PCA953X
config GPIO_PCA953X_IRQ
bool "Interrupt controller support for PCA953x"
depends on GPIO_PCA953X=y
depends on GPIO_PCA953X
select GPIOLIB_IRQCHIP
help
Say yes here to enable the pca953x to be used as an interrupt
@ -1541,6 +1545,18 @@ config GPIO_VIPERBOARD
endmenu
config GPIO_AGGREGATOR
tristate "GPIO Aggregator"
help
Say yes here to enable the GPIO Aggregator, which provides a way to
aggregate existing GPIO lines into a new virtual GPIO chip.
This can serve the following purposes:
- Assign permissions for a collection of GPIO lines to a user,
- Export a collection of GPIO lines to a virtual machine,
- Provide a generic driver for a GPIO-operated device in an
industrial control context, to be operated from userspace using
the GPIO chardev interface.
config GPIO_MOCKUP
tristate "GPIO Testing Driver"
select IRQ_SIM

2
drivers/gpio/Makefile
View File

@ -12,6 +12,7 @@ obj-$(CONFIG_GPIO_SYSFS) += gpiolib-sysfs.o
obj-$(CONFIG_GPIO_ACPI) += gpiolib-acpi.o
# Device drivers. Generally keep list sorted alphabetically
obj-$(CONFIG_GPIO_REGMAP) += gpio-regmap.o
obj-$(CONFIG_GPIO_GENERIC) += gpio-generic.o
# directly supported by gpio-generic
@ -25,6 +26,7 @@ obj-$(CONFIG_GPIO_74XX_MMIO) += gpio-74xx-mmio.o
obj-$(CONFIG_GPIO_ADNP) += gpio-adnp.o
obj-$(CONFIG_GPIO_ADP5520) += gpio-adp5520.o
obj-$(CONFIG_GPIO_ADP5588) += gpio-adp5588.o
obj-$(CONFIG_GPIO_AGGREGATOR) += gpio-aggregator.o
obj-$(CONFIG_GPIO_ALTERA_A10SR) += gpio-altera-a10sr.o
obj-$(CONFIG_GPIO_ALTERA) += gpio-altera.o
obj-$(CONFIG_GPIO_AMD8111) += gpio-amd8111.o

4
drivers/gpio/TODO
View File

@ -99,6 +99,10 @@ similar and probe a proper driver in the gpiolib subsystem.
In some cases it makes sense to create a GPIO chip from the local driver
for a few GPIOs. Those should stay where they are.
At the same time it makes sense to get rid of code duplication in existing or
new coming drivers. For example, gpio-ml-ioh should be incorporated into
gpio-pch. In similar way gpio-intel-mid into gpio-pxa.
Generic MMIO GPIO

568
drivers/gpio/gpio-aggregator.c Normal file
View File

@ -0,0 +1,568 @@
// SPDX-License-Identifier: GPL-2.0-only
//
// GPIO Aggregator
//
// Copyright (C) 2019-2020 Glider bv
#define DRV_NAME "gpio-aggregator"
#define pr_fmt(fmt) DRV_NAME ": " fmt
#include <linux/bitmap.h>
#include <linux/bitops.h>
#include <linux/ctype.h>
#include <linux/gpio/consumer.h>
#include <linux/gpio/driver.h>
#include <linux/gpio/machine.h>
#include <linux/idr.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/mutex.h>
#include <linux/overflow.h>
#include <linux/platform_device.h>
#include <linux/spinlock.h>
#include <linux/string.h>
/*
* GPIO Aggregator sysfs interface
*/
struct gpio_aggregator {
struct gpiod_lookup_table *lookups;
struct platform_device *pdev;
char args[];
};
static DEFINE_MUTEX(gpio_aggregator_lock); /* protects idr */
static DEFINE_IDR(gpio_aggregator_idr);
static char *get_arg(char **args)
{
char *start = *args, *end;
start = skip_spaces(start);
if (!*start)
return NULL;
if (*start == '"') {
/* Quoted arg */
end = strchr(++start, '"');
if (!end)
return ERR_PTR(-EINVAL);
} else {
/* Unquoted arg */
for (end = start; *end && !isspace(*end); end++) ;
}
if (*end)
*end++ = '\0';
*args = end;
return start;
}
static bool isrange(const char *s)
{
size_t n;
if (IS_ERR_OR_NULL(s))
return false;
while (1) {
n = strspn(s, "0123456789");
if (!n)
return false;
s += n;
switch (*s++) {
case '\0':
return true;
case '-':
case ',':
break;
default:
return false;
}
}
}
static int aggr_add_gpio(struct gpio_aggregator *aggr, const char *key,
int hwnum, unsigned int *n)
{
struct gpiod_lookup_table *lookups;
lookups = krealloc(aggr->lookups, struct_size(lookups, table, *n + 2),
GFP_KERNEL);
if (!lookups)
return -ENOMEM;
lookups->table[*n] =
(struct gpiod_lookup)GPIO_LOOKUP_IDX(key, hwnum, NULL, *n, 0);
(*n)++;
memset(&lookups->table[*n], 0, sizeof(lookups->table[*n]));
aggr->lookups = lookups;
return 0;
}
static int aggr_parse(struct gpio_aggregator *aggr)
{
unsigned int first_index, last_index, i, n = 0;
char *name, *offsets, *first, *last, *next;
char *args = aggr->args;
int error;
for (name = get_arg(&args), offsets = get_arg(&args); name;
offsets = get_arg(&args)) {
if (IS_ERR(name)) {
pr_err("Cannot get GPIO specifier: %pe\n", name);
return PTR_ERR(name);
}
if (!isrange(offsets)) {
/* Named GPIO line */
error = aggr_add_gpio(aggr, name, U16_MAX, &n);
if (error)
return error;
name = offsets;
continue;
}
/* GPIO chip + offset(s) */
for (first = offsets; *first; first = next) {
next = strchrnul(first, ',');
if (*next)
*next++ = '\0';
last = strchr(first, '-');
if (last)
*last++ = '\0';
if (kstrtouint(first, 10, &first_index)) {
pr_err("Cannot parse GPIO index %s\n", first);
return -EINVAL;
}
if (!last) {
last_index = first_index;
} else if (kstrtouint(last, 10, &last_index)) {
pr_err("Cannot parse GPIO index %s\n", last);
return -EINVAL;
}
for (i = first_index; i <= last_index; i++) {
error = aggr_add_gpio(aggr, name, i, &n);
if (error)
return error;
}
}
name = get_arg(&args);
}
if (!n) {
pr_err("No GPIOs specified\n");
return -EINVAL;
}
return 0;
}
static ssize_t new_device_store(struct device_driver *driver, const char *buf,
size_t count)
{
struct gpio_aggregator *aggr;
struct platform_device *pdev;
int res, id;
/* kernfs guarantees string termination, so count + 1 is safe */
aggr = kzalloc(sizeof(*aggr) + count + 1, GFP_KERNEL);
if (!aggr)
return -ENOMEM;
memcpy(aggr->args, buf, count + 1);
aggr->lookups = kzalloc(struct_size(aggr->lookups, table, 1),
GFP_KERNEL);
if (!aggr->lookups) {
res = -ENOMEM;
goto free_ga;
}
mutex_lock(&gpio_aggregator_lock);
id = idr_alloc(&gpio_aggregator_idr, aggr, 0, 0, GFP_KERNEL);
mutex_unlock(&gpio_aggregator_lock);
if (id < 0) {
res = id;
goto free_table;
}
aggr->lookups->dev_id = kasprintf(GFP_KERNEL, "%s.%d", DRV_NAME, id);
if (!aggr->lookups->dev_id) {
res = -ENOMEM;
goto remove_idr;
}
res = aggr_parse(aggr);
if (res)
goto free_dev_id;
gpiod_add_lookup_table(aggr->lookups);
pdev = platform_device_register_simple(DRV_NAME, id, NULL, 0);
if (IS_ERR(pdev)) {
res = PTR_ERR(pdev);
goto remove_table;
}
aggr->pdev = pdev;
return count;
remove_table:
gpiod_remove_lookup_table(aggr->lookups);
free_dev_id:
kfree(aggr->lookups->dev_id);
remove_idr:
mutex_lock(&gpio_aggregator_lock);
idr_remove(&gpio_aggregator_idr, id);
mutex_unlock(&gpio_aggregator_lock);
free_table:
kfree(aggr->lookups);
free_ga:
kfree(aggr);
return res;
}
static DRIVER_ATTR_WO(new_device);
static void gpio_aggregator_free(struct gpio_aggregator *aggr)
{
platform_device_unregister(aggr->pdev);
gpiod_remove_lookup_table(aggr->lookups);
kfree(aggr->lookups->dev_id);
kfree(aggr->lookups);
kfree(aggr);
}
static ssize_t delete_device_store(struct device_driver *driver,
const char *buf, size_t count)
{
struct gpio_aggregator *aggr;
unsigned int id;
int error;
if (!str_has_prefix(buf, DRV_NAME "."))
return -EINVAL;
error = kstrtouint(buf + strlen(DRV_NAME "."), 10, &id);
if (error)
return error;
mutex_lock(&gpio_aggregator_lock);
aggr = idr_remove(&gpio_aggregator_idr, id);
mutex_unlock(&gpio_aggregator_lock);
if (!aggr)
return -ENOENT;
gpio_aggregator_free(aggr);
return count;
}
static DRIVER_ATTR_WO(delete_device);
static struct attribute *gpio_aggregator_attrs[] = {
&driver_attr_new_device.attr,
&driver_attr_delete_device.attr,
NULL,
};
ATTRIBUTE_GROUPS(gpio_aggregator);
static int __exit gpio_aggregator_idr_remove(int id, void *p, void *data)
{
gpio_aggregator_free(p);
return 0;
}
static void __exit gpio_aggregator_remove_all(void)
{
mutex_lock(&gpio_aggregator_lock);
idr_for_each(&gpio_aggregator_idr, gpio_aggregator_idr_remove, NULL);
idr_destroy(&gpio_aggregator_idr);
mutex_unlock(&gpio_aggregator_lock);
}
/*
* GPIO Forwarder
*/
struct gpiochip_fwd {
struct gpio_chip chip;
struct gpio_desc **descs;
union {
struct mutex mlock; /* protects tmp[] if can_sleep */
spinlock_t slock; /* protects tmp[] if !can_sleep */
};
unsigned long tmp[]; /* values and descs for multiple ops */
};
static int gpio_fwd_get_direction(struct gpio_chip *chip, unsigned int offset)
{
struct gpiochip_fwd *fwd = gpiochip_get_data(chip);
return gpiod_get_direction(fwd->descs[offset]);
}
static int gpio_fwd_direction_input(struct gpio_chip *chip, unsigned int offset)
{
struct gpiochip_fwd *fwd = gpiochip_get_data(chip);
return gpiod_direction_input(fwd->descs[offset]);
}
static int gpio_fwd_direction_output(struct gpio_chip *chip,
unsigned int offset, int value)
{
struct gpiochip_fwd *fwd = gpiochip_get_data(chip);
return gpiod_direction_output(fwd->descs[offset], value);
}
static int gpio_fwd_get(struct gpio_chip *chip, unsigned int offset)
{
struct gpiochip_fwd *fwd = gpiochip_get_data(chip);
return gpiod_get_value(fwd->descs[offset]);
}
static int gpio_fwd_get_multiple(struct gpio_chip *chip, unsigned long *mask,
unsigned long *bits)
{
struct gpiochip_fwd *fwd = gpiochip_get_data(chip);
unsigned long *values, flags = 0;
struct gpio_desc **descs;
unsigned int i, j = 0;
int error;
if (chip->can_sleep)
mutex_lock(&fwd->mlock);
else
spin_lock_irqsave(&fwd->slock, flags);
/* Both values bitmap and desc pointers are stored in tmp[] */
values = &fwd->tmp[0];
descs = (void *)&fwd->tmp[BITS_TO_LONGS(fwd->chip.ngpio)];
bitmap_clear(values, 0, fwd->chip.ngpio);
for_each_set_bit(i, mask, fwd->chip.ngpio)
descs[j++] = fwd->descs[i];
error = gpiod_get_array_value(j, descs, NULL, values);
if (!error) {
j = 0;
for_each_set_bit(i, mask, fwd->chip.ngpio)
__assign_bit(i, bits, test_bit(j++, values));
}
if (chip->can_sleep)
mutex_unlock(&fwd->mlock);
else
spin_unlock_irqrestore(&fwd->slock, flags);
return error;
}
static void gpio_fwd_set(struct gpio_chip *chip, unsigned int offset, int value)
{
struct gpiochip_fwd *fwd = gpiochip_get_data(chip);
gpiod_set_value(fwd->descs[offset], value);
}
static void gpio_fwd_set_multiple(struct gpio_chip *chip, unsigned long *mask,
unsigned long *bits)
{
struct gpiochip_fwd *fwd = gpiochip_get_data(chip);
unsigned long *values, flags = 0;
struct gpio_desc **descs;
unsigned int i, j = 0;
if (chip->can_sleep)
mutex_lock(&fwd->mlock);
else
spin_lock_irqsave(&fwd->slock, flags);
/* Both values bitmap and desc pointers are stored in tmp[] */
values = &fwd->tmp[0];
descs = (void *)&fwd->tmp[BITS_TO_LONGS(fwd->chip.ngpio)];
for_each_set_bit(i, mask, fwd->chip.ngpio) {
__assign_bit(j, values, test_bit(i, bits));
descs[j++] = fwd->descs[i];
}
gpiod_set_array_value(j, descs, NULL, values);
if (chip->can_sleep)
mutex_unlock(&fwd->mlock);
else
spin_unlock_irqrestore(&fwd->slock, flags);
}
static int gpio_fwd_set_config(struct gpio_chip *chip, unsigned int offset,
unsigned long config)
{
struct gpiochip_fwd *fwd = gpiochip_get_data(chip);
return gpiod_set_config(fwd->descs[offset], config);
}
/**
* gpiochip_fwd_create() - Create a new GPIO forwarder
* @dev: Parent device pointer
* @ngpios: Number of GPIOs in the forwarder.
* @descs: Array containing the GPIO descriptors to forward to.
* This array must contain @ngpios entries, and must not be deallocated
* before the forwarder has been destroyed again.
*
* This function creates a new gpiochip, which forwards all GPIO operations to
* the passed GPIO descriptors.
*
* Return: An opaque object pointer, or an ERR_PTR()-encoded negative error
* code on failure.
*/
static struct gpiochip_fwd *gpiochip_fwd_create(struct device *dev,
unsigned int ngpios,
struct gpio_desc *descs[])
{
const char *label = dev_name(dev);
struct gpiochip_fwd *fwd;
struct gpio_chip *chip;
unsigned int i;
int error;
fwd = devm_kzalloc(dev, struct_size(fwd, tmp,
BITS_TO_LONGS(ngpios) + ngpios), GFP_KERNEL);
if (!fwd)
return ERR_PTR(-ENOMEM);
chip = &fwd->chip;
/*
* If any of the GPIO lines are sleeping, then the entire forwarder
* will be sleeping.
* If any of the chips support .set_config(), then the forwarder will
* support setting configs.
*/
for (i = 0; i < ngpios; i++) {
struct gpio_chip *parent = gpiod_to_chip(descs[i]);
dev_dbg(dev, "%u => gpio-%d\n", i, desc_to_gpio(descs[i]));
if (gpiod_cansleep(descs[i]))
chip->can_sleep = true;
if (parent && parent->set_config)
chip->set_config = gpio_fwd_set_config;
}
chip->label = label;
chip->parent = dev;
chip->owner = THIS_MODULE;
chip->get_direction = gpio_fwd_get_direction;
chip->direction_input = gpio_fwd_direction_input;
chip->direction_output = gpio_fwd_direction_output;
chip->get = gpio_fwd_get;
chip->get_multiple = gpio_fwd_get_multiple;
chip->set = gpio_fwd_set;
chip->set_multiple = gpio_fwd_set_multiple;
chip->base = -1;
chip->ngpio = ngpios;
fwd->descs = descs;
if (chip->can_sleep)
mutex_init(&fwd->mlock);
else
spin_lock_init(&fwd->slock);
error = devm_gpiochip_add_data(dev, chip, fwd);
if (error)
return ERR_PTR(error);
return fwd;
}
/*
* GPIO Aggregator platform device
*/
static int gpio_aggregator_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
struct gpio_desc **descs;
struct gpiochip_fwd *fwd;
int i, n;
n = gpiod_count(dev, NULL);
if (n < 0)
return n;
descs = devm_kmalloc_array(dev, n, sizeof(*descs), GFP_KERNEL);
if (!descs)
return -ENOMEM;
for (i = 0; i < n; i++) {
descs[i] = devm_gpiod_get_index(dev, NULL, i, GPIOD_ASIS);
if (IS_ERR(descs[i]))
return PTR_ERR(descs[i]);
}
fwd = gpiochip_fwd_create(dev, n, descs);
if (IS_ERR(fwd))
return PTR_ERR(fwd);
platform_set_drvdata(pdev, fwd);
return 0;
}
#ifdef CONFIG_OF
static const struct of_device_id gpio_aggregator_dt_ids[] = {
/*
* Add GPIO-operated devices controlled from userspace below,
* or use "driver_override" in sysfs
*/
{},
};
MODULE_DEVICE_TABLE(of, gpio_aggregator_dt_ids);
#endif
static struct platform_driver gpio_aggregator_driver = {
.probe = gpio_aggregator_probe,
.driver = {
.name = DRV_NAME,
.groups = gpio_aggregator_groups,
.of_match_table = of_match_ptr(gpio_aggregator_dt_ids),
},
};
static int __init gpio_aggregator_init(void)
{
return platform_driver_register(&gpio_aggregator_driver);
}
module_init(gpio_aggregator_init);
static void __exit gpio_aggregator_exit(void)
{
gpio_aggregator_remove_all();
platform_driver_unregister(&gpio_aggregator_driver);
}
module_exit(gpio_aggregator_exit);
MODULE_AUTHOR("Geert Uytterhoeven <geert+renesas@glider.be>");
MODULE_DESCRIPTION("GPIO Aggregator");
MODULE_LICENSE("GPL v2");

248
drivers/gpio/gpio-dwapb.c
View File

@ -49,7 +49,9 @@
#define GPIO_EXT_PORTC 0x58
#define GPIO_EXT_PORTD 0x5c
#define DWAPB_DRIVER_NAME "gpio-dwapb"
#define DWAPB_MAX_PORTS 4
#define GPIO_EXT_PORT_STRIDE 0x04 /* register stride 32 bits */
#define GPIO_SWPORT_DR_STRIDE 0x0c /* register stride 3*32 bits */
#define GPIO_SWPORT_DDR_STRIDE 0x0c /* register stride 3*32 bits */
@ -62,6 +64,8 @@
#define GPIO_INTSTATUS_V2 0x3c
#define GPIO_PORTA_EOI_V2 0x40
#define DWAPB_NR_CLOCKS 2
struct dwapb_gpio;
#ifdef CONFIG_PM_SLEEP
@ -97,7 +101,7 @@ struct dwapb_gpio {
struct irq_domain *domain;
unsigned int flags;
struct reset_control *rst;
struct clk *clk;
struct clk_bulk_data clks[DWAPB_NR_CLOCKS];
};
static inline u32 gpio_reg_v2_convert(unsigned int offset)
@ -189,22 +193,21 @@ static void dwapb_toggle_trigger(struct dwapb_gpio *gpio, unsigned int offs)
static u32 dwapb_do_irq(struct dwapb_gpio *gpio)
{
u32 irq_status = dwapb_read(gpio, GPIO_INTSTATUS);
u32 ret = irq_status;
unsigned long irq_status;
irq_hw_number_t hwirq;
while (irq_status) {
int hwirq = fls(irq_status) - 1;
irq_status = dwapb_read(gpio, GPIO_INTSTATUS);
for_each_set_bit(hwirq, &irq_status, 32) {
int gpio_irq = irq_find_mapping(gpio->domain, hwirq);
u32 irq_type = irq_get_trigger_type(gpio_irq);
generic_handle_irq(gpio_irq);
irq_status &= ~BIT(hwirq);
if ((irq_get_trigger_type(gpio_irq) & IRQ_TYPE_SENSE_MASK)
== IRQ_TYPE_EDGE_BOTH)
if ((irq_type & IRQ_TYPE_SENSE_MASK) == IRQ_TYPE_EDGE_BOTH)
dwapb_toggle_trigger(gpio, hwirq);
}
return ret;
return irq_status;
}
static void dwapb_irq_handler(struct irq_desc *desc)
@ -212,10 +215,9 @@ static void dwapb_irq_handler(struct irq_desc *desc)
struct dwapb_gpio *gpio = irq_desc_get_handler_data(desc);
struct irq_chip *chip = irq_desc_get_chip(desc);
chained_irq_enter(chip, desc);
dwapb_do_irq(gpio);
if (chip->irq_eoi)
chip->irq_eoi(irq_desc_get_irq_data(desc));
chained_irq_exit(chip, desc);
}
static void dwapb_irq_enable(struct irq_data *d)
@ -228,7 +230,7 @@ static void dwapb_irq_enable(struct irq_data *d)
spin_lock_irqsave(&gc->bgpio_lock, flags);
val = dwapb_read(gpio, GPIO_INTEN);
val |= BIT(d->hwirq);
val |= BIT(irqd_to_hwirq(d));
dwapb_write(gpio, GPIO_INTEN, val);
spin_unlock_irqrestore(&gc->bgpio_lock, flags);
}
@ -243,46 +245,20 @@ static void dwapb_irq_disable(struct irq_data *d)
spin_lock_irqsave(&gc->bgpio_lock, flags);
val = dwapb_read(gpio, GPIO_INTEN);
val &= ~BIT(d->hwirq);
val &= ~BIT(irqd_to_hwirq(d));
dwapb_write(gpio, GPIO_INTEN, val);
spin_unlock_irqrestore(&gc->bgpio_lock, flags);
}
static int dwapb_irq_reqres(struct irq_data *d)
{
struct irq_chip_generic *igc = irq_data_get_irq_chip_data(d);
struct dwapb_gpio *gpio = igc->private;
struct gpio_chip *gc = &gpio->ports[0].gc;
int ret;
ret = gpiochip_lock_as_irq(gc, irqd_to_hwirq(d));
if (ret) {
dev_err(gpio->dev, "unable to lock HW IRQ %lu for IRQ\n",
irqd_to_hwirq(d));
return ret;
}
return 0;
}
static void dwapb_irq_relres(struct irq_data *d)
{
struct irq_chip_generic *igc = irq_data_get_irq_chip_data(d);
struct dwapb_gpio *gpio = igc->private;
struct gpio_chip *gc = &gpio->ports[0].gc;
gpiochip_unlock_as_irq(gc, irqd_to_hwirq(d));
}
static int dwapb_irq_set_type(struct irq_data *d, u32 type)
{
struct irq_chip_generic *igc = irq_data_get_irq_chip_data(d);
struct dwapb_gpio *gpio = igc->private;
struct gpio_chip *gc = &gpio->ports[0].gc;
int bit = d->hwirq;
irq_hw_number_t bit = irqd_to_hwirq(d);
unsigned long level, polarity, flags;
if (type & ~(IRQ_TYPE_EDGE_RISING | IRQ_TYPE_EDGE_FALLING |
IRQ_TYPE_LEVEL_HIGH | IRQ_TYPE_LEVEL_LOW))
if (type & ~IRQ_TYPE_SENSE_MASK)
return -EINVAL;
spin_lock_irqsave(&gc->bgpio_lock, flags);
@ -328,11 +304,12 @@ static int dwapb_irq_set_wake(struct irq_data *d, unsigned int enable)
struct irq_chip_generic *igc = irq_data_get_irq_chip_data(d);
struct dwapb_gpio *gpio = igc->private;
struct dwapb_context *ctx = gpio->ports[0].ctx;
irq_hw_number_t bit = irqd_to_hwirq(d);
if (enable)
ctx->wake_en |= BIT(d->hwirq);
ctx->wake_en |= BIT(bit);
else
ctx->wake_en &= ~BIT(d->hwirq);
ctx->wake_en &= ~BIT(bit);
return 0;
}
@ -350,9 +327,10 @@ static int dwapb_gpio_set_debounce(struct gpio_chip *gc,
val_deb = dwapb_read(gpio, GPIO_PORTA_DEBOUNCE);
if (debounce)
dwapb_write(gpio, GPIO_PORTA_DEBOUNCE, val_deb | mask);
val_deb |= mask;
else
dwapb_write(gpio, GPIO_PORTA_DEBOUNCE, val_deb & ~mask);
val_deb &= ~mask;
dwapb_write(gpio, GPIO_PORTA_DEBOUNCE, val_deb);
spin_unlock_irqrestore(&gc->bgpio_lock, flags);
@ -373,12 +351,7 @@ static int dwapb_gpio_set_config(struct gpio_chip *gc, unsigned offset,
static irqreturn_t dwapb_irq_handler_mfd(int irq, void *dev_id)
{
u32 worked;
struct dwapb_gpio *gpio = dev_id;
worked = dwapb_do_irq(gpio);
return worked ? IRQ_HANDLED : IRQ_NONE;
return IRQ_RETVAL(dwapb_do_irq(dev_id));
}
static void dwapb_configure_irqs(struct dwapb_gpio *gpio,
@ -388,17 +361,23 @@ static void dwapb_configure_irqs(struct dwapb_gpio *gpio,
struct gpio_chip *gc = &port->gc;
struct fwnode_handle *fwnode = pp->fwnode;
struct irq_chip_generic *irq_gc = NULL;
unsigned int hwirq, ngpio = gc->ngpio;
unsigned int ngpio = gc->ngpio;
struct irq_chip_type *ct;
irq_hw_number_t hwirq;
int err, i;
if (memchr_inv(pp->irq, 0, sizeof(pp->irq)) == NULL) {
dev_warn(gpio->dev, "no IRQ for port%d\n", pp->idx);
return;
}
gpio->domain = irq_domain_create_linear(fwnode, ngpio,
&irq_generic_chip_ops, gpio);
if (!gpio->domain)
return;
err = irq_alloc_domain_generic_chips(gpio->domain, ngpio, 2,
"gpio-dwapb", handle_level_irq,
DWAPB_DRIVER_NAME, handle_bad_irq,
IRQ_NOREQUEST, 0,
IRQ_GC_INIT_NESTED_LOCK);
if (err) {
@ -426,8 +405,6 @@ static void dwapb_configure_irqs(struct dwapb_gpio *gpio,
ct->chip.irq_set_type = dwapb_irq_set_type;
ct->chip.irq_enable = dwapb_irq_enable;
ct->chip.irq_disable = dwapb_irq_disable;
ct->chip.irq_request_resources = dwapb_irq_reqres;
ct->chip.irq_release_resources = dwapb_irq_relres;
#ifdef CONFIG_PM_SLEEP
ct->chip.irq_set_wake = dwapb_irq_set_wake;
#endif
@ -437,6 +414,7 @@ static void dwapb_configure_irqs(struct dwapb_gpio *gpio,
}
irq_gc->chip_types[0].type = IRQ_TYPE_LEVEL_MASK;
irq_gc->chip_types[0].handler = handle_level_irq;
irq_gc->chip_types[1].type = IRQ_TYPE_EDGE_BOTH;
irq_gc->chip_types[1].handler = handle_edge_irq;
@ -444,7 +422,7 @@ static void dwapb_configure_irqs(struct dwapb_gpio *gpio,
int i;
for (i = 0; i < pp->ngpio; i++) {
if (pp->irq[i] >= 0)
if (pp->irq[i])
irq_set_chained_handler_and_data(pp->irq[i],
dwapb_irq_handler, gpio);
}
@ -455,7 +433,7 @@ static void dwapb_configure_irqs(struct dwapb_gpio *gpio,
*/
err = devm_request_irq(gpio->dev, pp->irq[0],
dwapb_irq_handler_mfd,
IRQF_SHARED, "gpio-dwapb-mfd", gpio);
IRQF_SHARED, DWAPB_DRIVER_NAME, gpio);
if (err) {
dev_err(gpio->dev, "error requesting IRQ\n");
irq_domain_remove(gpio->domain);
@ -464,7 +442,7 @@ static void dwapb_configure_irqs(struct dwapb_gpio *gpio,
}
}
for (hwirq = 0 ; hwirq < ngpio ; hwirq++)
for (hwirq = 0; hwirq < ngpio; hwirq++)
irq_create_mapping(gpio->domain, hwirq);
port->gc.to_irq = dwapb_gpio_to_irq;
@ -480,7 +458,7 @@ static void dwapb_irq_teardown(struct dwapb_gpio *gpio)
if (!gpio->domain)
return;
for (hwirq = 0 ; hwirq < ngpio ; hwirq++)
for (hwirq = 0; hwirq < ngpio; hwirq++)
irq_dispose_mapping(irq_find_mapping(gpio->domain, hwirq));
irq_domain_remove(gpio->domain);
@ -505,10 +483,9 @@ static int dwapb_gpio_add_port(struct dwapb_gpio *gpio,
return -ENOMEM;
#endif
dat = gpio->regs + GPIO_EXT_PORTA + (pp->idx * GPIO_EXT_PORT_STRIDE);
set = gpio->regs + GPIO_SWPORTA_DR + (pp->idx * GPIO_SWPORT_DR_STRIDE);
dirout = gpio->regs + GPIO_SWPORTA_DDR +
(pp->idx * GPIO_SWPORT_DDR_STRIDE);
dat = gpio->regs + GPIO_EXT_PORTA + pp->idx * GPIO_EXT_PORT_STRIDE;
set = gpio->regs + GPIO_SWPORTA_DR + pp->idx * GPIO_SWPORT_DR_STRIDE;
dirout = gpio->regs + GPIO_SWPORTA_DDR + pp->idx * GPIO_SWPORT_DDR_STRIDE;
/* This registers 32 GPIO lines per port */
err = bgpio_init(&port->gc, gpio->dev, 4, dat, set, NULL, dirout,
@ -529,40 +506,66 @@ static int dwapb_gpio_add_port(struct dwapb_gpio *gpio,
if (pp->idx == 0)
port->gc.set_config = dwapb_gpio_set_config;
if (pp->has_irq)
/* Only port A can provide interrupts in all configurations of the IP */
if (pp->idx == 0)
dwapb_configure_irqs(gpio, port, pp);
err = gpiochip_add_data(&port->gc, port);
if (err)
if (err) {
dev_err(gpio->dev, "failed to register gpiochip for port%d\n",
port->idx);
else
port->is_registered = true;
return err;
}
/* Add GPIO-signaled ACPI event support */
if (pp->has_irq)
acpi_gpiochip_request_interrupts(&port->gc);
acpi_gpiochip_request_interrupts(&port->gc);
return err;
port->is_registered = true;
return 0;
}
static void dwapb_gpio_unregister(struct dwapb_gpio *gpio)
{
unsigned int m;
for (m = 0; m < gpio->nr_ports; ++m)
if (gpio->ports[m].is_registered)
gpiochip_remove(&gpio->ports[m].gc);
for (m = 0; m < gpio->nr_ports; ++m) {
struct dwapb_gpio_port *port = &gpio->ports[m];
if (!port->is_registered)
continue;
acpi_gpiochip_free_interrupts(&port->gc);
gpiochip_remove(&port->gc);
}
}
static struct dwapb_platform_data *
dwapb_gpio_get_pdata(struct device *dev)
static void dwapb_get_irq(struct device *dev, struct fwnode_handle *fwnode,
struct dwapb_port_property *pp)
{
struct device_node *np = NULL;
int irq = -ENXIO, j;
if (fwnode_property_read_bool(fwnode, "interrupt-controller"))
np = to_of_node(fwnode);
for (j = 0; j < pp->ngpio; j++) {
if (np)
irq = of_irq_get(np, j);
else if (has_acpi_companion(dev))
irq = platform_get_irq_optional(to_platform_device(dev), j);
if (irq > 0)
pp->irq[j] = irq;
}
}
static struct dwapb_platform_data *dwapb_gpio_get_pdata(struct device *dev)
{
struct fwnode_handle *fwnode;
struct dwapb_platform_data *pdata;
struct dwapb_port_property *pp;
int nports;
int i, j;
int i;
nports = device_get_child_node_count(dev);
if (nports == 0)
@ -580,8 +583,6 @@ dwapb_gpio_get_pdata(struct device *dev)
i = 0;
device_for_each_child_node(dev, fwnode) {
struct device_node *np = NULL;
pp = &pdata->properties[i++];
pp->fwnode = fwnode;
@ -593,8 +594,7 @@ dwapb_gpio_get_pdata(struct device *dev)
return ERR_PTR(-EINVAL);
}
if (fwnode_property_read_u32(fwnode, "snps,nr-gpios",
&pp->ngpio)) {
if (fwnode_property_read_u32(fwnode, "snps,nr-gpios", &pp->ngpio)) {
dev_info(dev,
"failed to get number of gpios for port%d\n",
i);
@ -608,28 +608,8 @@ dwapb_gpio_get_pdata(struct device *dev)
* Only port A can provide interrupts in all configurations of
* the IP.
*/
if (pp->idx != 0)
continue;
if (dev->of_node && fwnode_property_read_bool(fwnode,
"interrupt-controller")) {
np = to_of_node(fwnode);
}
for (j = 0; j < pp->ngpio; j++) {
pp->irq[j] = -ENXIO;
if (np)
pp->irq[j] = of_irq_get(np, j);
else if (has_acpi_companion(dev))
pp->irq[j] = platform_get_irq(to_platform_device(dev), j);
if (pp->irq[j] >= 0)
pp->has_irq = true;
}
if (!pp->has_irq)
dev_warn(dev, "no irq for port%d\n", pp->idx);
if (pp->idx == 0)
dwapb_get_irq(dev, fwnode, pp);
}
return pdata;
@ -689,29 +669,24 @@ static int dwapb_gpio_probe(struct platform_device *pdev)
if (IS_ERR(gpio->regs))
return PTR_ERR(gpio->regs);
/* Optional bus clock */
gpio->clk = devm_clk_get(&pdev->dev, "bus");
if (!IS_ERR(gpio->clk)) {
err = clk_prepare_enable(gpio->clk);
if (err) {
dev_info(&pdev->dev, "Cannot enable clock\n");
return err;
}
/* Optional bus and debounce clocks */
gpio->clks[0].id = "bus";
gpio->clks[1].id = "db";
err = devm_clk_bulk_get_optional(&pdev->dev, DWAPB_NR_CLOCKS,
gpio->clks);
if (err) {
dev_err(&pdev->dev, "Cannot get APB/Debounce clocks\n");
return err;
}
gpio->flags = 0;
if (dev->of_node) {
gpio->flags = (uintptr_t)of_device_get_match_data(dev);
} else if (has_acpi_companion(dev)) {
const struct acpi_device_id *acpi_id;
acpi_id = acpi_match_device(dwapb_acpi_match, dev);
if (acpi_id) {
if (acpi_id->driver_data)
gpio->flags = acpi_id->driver_data;
}
err = clk_bulk_prepare_enable(DWAPB_NR_CLOCKS, gpio->clks);
if (err) {
dev_err(&pdev->dev, "Cannot enable APB/Debounce clocks\n");
return err;
}
gpio->flags = (uintptr_t)device_get_match_data(dev);
for (i = 0; i < gpio->nr_ports; i++) {
err = dwapb_gpio_add_port(gpio, &pdata->properties[i], i);
if (err)
@ -724,7 +699,7 @@ static int dwapb_gpio_probe(struct platform_device *pdev)
out_unregister:
dwapb_gpio_unregister(gpio);
dwapb_irq_teardown(gpio);
clk_disable_unprepare(gpio->clk);
clk_bulk_disable_unprepare(DWAPB_NR_CLOCKS, gpio->clks);
return err;
}
@ -736,7 +711,7 @@ static int dwapb_gpio_remove(struct platform_device *pdev)
dwapb_gpio_unregister(gpio);
dwapb_irq_teardown(gpio);
reset_control_assert(gpio->rst);
clk_disable_unprepare(gpio->clk);
clk_bulk_disable_unprepare(DWAPB_NR_CLOCKS, gpio->clks);
return 0;
}
@ -755,8 +730,6 @@ static int dwapb_gpio_suspend(struct device *dev)
unsigned int idx = gpio->ports[i].idx;
struct dwapb_context *ctx = gpio->ports[i].ctx;
BUG_ON(!ctx);
offset = GPIO_SWPORTA_DDR + idx * GPIO_SWPORT_DDR_STRIDE;
ctx->dir = dwapb_read(gpio, offset);
@ -775,13 +748,12 @@ static int dwapb_gpio_suspend(struct device *dev)
ctx->int_deb = dwapb_read(gpio, GPIO_PORTA_DEBOUNCE);
/* Mask out interrupts */
dwapb_write(gpio, GPIO_INTMASK,
0xffffffff & ~ctx->wake_en);
dwapb_write(gpio, GPIO_INTMASK, ~ctx->wake_en);
}
}
spin_unlock_irqrestore(&gc->bgpio_lock, flags);
clk_disable_unprepare(gpio->clk);
clk_bulk_disable_unprepare(DWAPB_NR_CLOCKS, gpio->clks);
return 0;
}
@ -791,10 +763,13 @@ static int dwapb_gpio_resume(struct device *dev)
struct dwapb_gpio *gpio = dev_get_drvdata(dev);
struct gpio_chip *gc = &gpio->ports[0].gc;
unsigned long flags;
int i;
int i, err;
if (!IS_ERR(gpio->clk))
clk_prepare_enable(gpio->clk);
err = clk_bulk_prepare_enable(DWAPB_NR_CLOCKS, gpio->clks);
if (err) {
dev_err(gpio->dev, "Cannot reenable APB/Debounce clocks\n");
return err;
}
spin_lock_irqsave(&gc->bgpio_lock, flags);
for (i = 0; i < gpio->nr_ports; i++) {
@ -802,8 +777,6 @@ static int dwapb_gpio_resume(struct device *dev)
unsigned int idx = gpio->ports[i].idx;
struct dwapb_context *ctx = gpio->ports[i].ctx;
BUG_ON(!ctx);
offset = GPIO_SWPORTA_DR + idx * GPIO_SWPORT_DR_STRIDE;
dwapb_write(gpio, offset, ctx->data);
@ -836,10 +809,10 @@ static SIMPLE_DEV_PM_OPS(dwapb_gpio_pm_ops, dwapb_gpio_suspend,
static struct platform_driver dwapb_gpio_driver = {
.driver = {
.name = "gpio-dwapb",
.name = DWAPB_DRIVER_NAME,
.pm = &dwapb_gpio_pm_ops,
.of_match_table = of_match_ptr(dwapb_of_match),
.acpi_match_table = ACPI_PTR(dwapb_acpi_match),
.of_match_table = dwapb_of_match,
.acpi_match_table = dwapb_acpi_match,
},
.probe = dwapb_gpio_probe,
.remove = dwapb_gpio_remove,
@ -850,3 +823,4 @@ module_platform_driver(dwapb_gpio_driver);
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Jamie Iles");
MODULE_DESCRIPTION("Synopsys DesignWare APB GPIO driver");
MODULE_ALIAS("platform:" DWAPB_DRIVER_NAME);

29
drivers/gpio/gpio-f7188x.c
View File

@ -36,9 +36,19 @@
#define SIO_F71889A_ID 0x1005 /* F71889A chipset ID */
#define SIO_F81866_ID 0x1010 /* F81866 chipset ID */
#define SIO_F81804_ID 0x1502 /* F81804 chipset ID, same for f81966 */
#define SIO_F81865_ID 0x0704 /* F81865 chipset ID */
enum chips { f71869, f71869a, f71882fg, f71889a, f71889f, f81866, f81804 };
enum chips {
f71869,
f71869a,
f71882fg,
f71889a,
f71889f,
f81866,
f81804,
f81865,
};
static const char * const f7188x_names[] = {
"f71869",
@ -48,6 +58,7 @@ static const char * const f7188x_names[] = {
"f71889f",
"f81866",
"f81804",
"f81865",
};
struct f7188x_sio {
@ -233,6 +244,15 @@ static struct f7188x_gpio_bank f81804_gpio_bank[] = {
F7188X_GPIO_BANK(90, 8, 0x98),
};
static struct f7188x_gpio_bank f81865_gpio_bank[] = {
F7188X_GPIO_BANK(0, 8, 0xF0),
F7188X_GPIO_BANK(10, 8, 0xE0),
F7188X_GPIO_BANK(20, 8, 0xD0),
F7188X_GPIO_BANK(30, 8, 0xC0),
F7188X_GPIO_BANK(40, 8, 0xB0),
F7188X_GPIO_BANK(50, 8, 0xA0),
F7188X_GPIO_BANK(60, 5, 0x90),
};
static int f7188x_gpio_get_direction(struct gpio_chip *chip, unsigned offset)
{
@ -425,6 +445,10 @@ static int f7188x_gpio_probe(struct platform_device *pdev)
data->nr_bank = ARRAY_SIZE(f81804_gpio_bank);
data->bank = f81804_gpio_bank;
break;
case f81865:
data->nr_bank = ARRAY_SIZE(f81865_gpio_bank);
data->bank = f81865_gpio_bank;
break;
default:
return -ENODEV;
}
@ -490,6 +514,9 @@ static int __init f7188x_find(int addr, struct f7188x_sio *sio)
case SIO_F81804_ID:
sio->type = f81804;
break;
case SIO_F81865_ID:
sio->type = f81865;
break;
default:
pr_info(DRVNAME ": Unsupported Fintek device 0x%04x\n", devid);
goto err;

2
drivers/gpio/gpio-ftgpio010.c
View File

@ -193,7 +193,7 @@ static int ftgpio_gpio_set_config(struct gpio_chip *gc, unsigned int offset,
if (val == deb_div) {
/*
* The debounce timer happens to already be set to the
* desireable value, what a coincidence! We can just enable
* desirable value, what a coincidence! We can just enable
* debounce on this GPIO line and return. This happens more
* often than you think, for example when all GPIO keys
* on a system are requesting the same debounce interval.

2
drivers/gpio/gpio-ich.c
View File

@ -89,7 +89,7 @@ static struct {
struct device *dev;
struct gpio_chip chip;
struct resource *gpio_base; /* GPIO IO base */
struct resource *pm_base; /* Power Mangagment IO base */
struct resource *pm_base; /* Power Management IO base */
struct ichx_desc *desc; /* Pointer to chipset-specific description */
u32 orig_gpio_ctrl; /* Orig CTRL value, used to restore on exit */
u8 use_gpio; /* Which GPIO groups are usable */

12
drivers/gpio/gpio-max730x.c
View File

@ -47,7 +47,7 @@
static int max7301_direction_input(struct gpio_chip *chip, unsigned offset)
{
struct max7301 *ts = gpiochip_get_data(chip);
struct max7301 *ts = container_of(chip, struct max7301, chip);
u8 *config;
u8 offset_bits, pin_config;
int ret;
@ -89,7 +89,7 @@ static int __max7301_set(struct max7301 *ts, unsigned offset, int value)
static int max7301_direction_output(struct gpio_chip *chip, unsigned offset,
int value)
{
struct max7301 *ts = gpiochip_get_data(chip);
struct max7301 *ts = container_of(chip, struct max7301, chip);
u8 *config;
u8 offset_bits;
int ret;
@ -189,10 +189,6 @@ int __max730x_probe(struct max7301 *ts)
ts->chip.parent = dev;
ts->chip.owner = THIS_MODULE;
ret = gpiochip_add_data(&ts->chip, ts);
if (ret)
goto exit_destroy;
/*
* initialize pullups according to platform data and cache the
* register values for later use.
@ -214,7 +210,9 @@ int __max730x_probe(struct max7301 *ts)
}
}
return ret;
ret = gpiochip_add_data(&ts->chip, ts);
if (!ret)
return ret;
exit_destroy:
mutex_destroy(&ts->lock);

28
drivers/gpio/gpio-mb86s7x.c
View File

@ -145,7 +145,9 @@ static int mb86s70_gpio_to_irq(struct gpio_chip *gc, unsigned int offset)
for (index = 0;; index++) {
irq = platform_get_irq(to_platform_device(gc->parent), index);
if (irq <= 0)
if (irq < 0)
return irq;
if (irq == 0)
break;
if (irq_get_irq_data(irq)->hwirq == offset)
return irq;
@ -168,15 +170,13 @@ static int mb86s70_gpio_probe(struct platform_device *pdev)
if (IS_ERR(gchip->base))
return PTR_ERR(gchip->base);
if (!has_acpi_companion(&pdev->dev)) {
gchip->clk = devm_clk_get(&pdev->dev, NULL);
if (IS_ERR(gchip->clk))
return PTR_ERR(gchip->clk);
gchip->clk = devm_clk_get_optional(&pdev->dev, NULL);
if (IS_ERR(gchip->clk))
return PTR_ERR(gchip->clk);
ret = clk_prepare_enable(gchip->clk);
if (ret)
return ret;
}
ret = clk_prepare_enable(gchip->clk);
if (ret)
return ret;
spin_lock_init(&gchip->lock);
@ -186,15 +186,13 @@ static int mb86s70_gpio_probe(struct platform_device *pdev)
gchip->gc.free = mb86s70_gpio_free;
gchip->gc.get = mb86s70_gpio_get;
gchip->gc.set = mb86s70_gpio_set;
gchip->gc.to_irq = mb86s70_gpio_to_irq;
gchip->gc.label = dev_name(&pdev->dev);
gchip->gc.ngpio = 32;
gchip->gc.owner = THIS_MODULE;
gchip->gc.parent = &pdev->dev;
gchip->gc.base = -1;
if (has_acpi_companion(&pdev->dev))
gchip->gc.to_irq = mb86s70_gpio_to_irq;
ret = gpiochip_add_data(&gchip->gc, gchip);
if (ret) {
dev_err(&pdev->dev, "couldn't register gpio driver\n");
@ -202,8 +200,7 @@ static int mb86s70_gpio_probe(struct platform_device *pdev)
return ret;
}
if (has_acpi_companion(&pdev->dev))
acpi_gpiochip_request_interrupts(&gchip->gc);
acpi_gpiochip_request_interrupts(&gchip->gc);
return 0;
}
@ -212,8 +209,7 @@ static int mb86s70_gpio_remove(struct platform_device *pdev)
{
struct mb86s70_gpio_chip *gchip = platform_get_drvdata(pdev);
if (has_acpi_companion(&pdev->dev))
acpi_gpiochip_free_interrupts(&gchip->gc);
acpi_gpiochip_free_interrupts(&gchip->gc);
gpiochip_remove(&gchip->gc);
clk_disable_unprepare(gchip->clk);

10
drivers/gpio/gpio-merrifield.c
View File

@ -443,8 +443,8 @@ static int mrfld_gpio_probe(struct pci_dev *pdev, const struct pci_device_id *id
base = pcim_iomap_table(pdev)[1];
irq_base = readl(base);
gpio_base = readl(sizeof(u32) + base);
irq_base = readl(base + 0 * sizeof(u32));
gpio_base = readl(base + 1 * sizeof(u32));
/* Release the IO mapping, since we already get the info from BAR1 */
pcim_iounmap_regions(pdev, BIT(1));
@ -473,6 +473,10 @@ static int mrfld_gpio_probe(struct pci_dev *pdev, const struct pci_device_id *id
raw_spin_lock_init(&priv->lock);
retval = pci_alloc_irq_vectors(pdev, 1, 1, PCI_IRQ_ALL_TYPES);
if (retval < 0)
return retval;
girq = &priv->chip.irq;
girq->chip = &mrfld_irqchip;
girq->init_hw = mrfld_irq_init_hw;
@ -482,7 +486,7 @@ static int mrfld_gpio_probe(struct pci_dev *pdev, const struct pci_device_id *id
sizeof(*girq->parents), GFP_KERNEL);
if (!girq->parents)
return -ENOMEM;
girq->parents[0] = pdev->irq;
girq->parents[0] = pci_irq_vector(pdev, 0);
girq->first = irq_base;
girq->default_type = IRQ_TYPE_NONE;
girq->handler = handle_bad_irq;

5
drivers/gpio/gpio-mlxbf2.c
View File

@ -14,7 +14,6 @@
#include <linux/resource.h>
#include <linux/spinlock.h>
#include <linux/types.h>
#include <linux/version.h>
/*
* There are 3 YU GPIO blocks:
@ -110,8 +109,8 @@ static int mlxbf2_gpio_get_lock_res(struct platform_device *pdev)
}
yu_arm_gpio_lock_param.io = devm_ioremap(dev, res->start, size);
if (IS_ERR(yu_arm_gpio_lock_param.io))
ret = PTR_ERR(yu_arm_gpio_lock_param.io);
if (!yu_arm_gpio_lock_param.io)
ret = -ENOMEM;
exit:
mutex_unlock(yu_arm_gpio_lock_param.lock);

2
drivers/gpio/gpio-mm-lantiq.c
View File

@ -36,7 +36,7 @@ struct ltq_mm {
* @chip: Pointer to our private data structure.
*
* Write the shadow value to the EBU to set the gpios. We need to set the
* global EBU lock to make sure that PCI/MTD dont break.
* global EBU lock to make sure that PCI/MTD don't break.
*/
static void ltq_mm_apply(struct ltq_mm *chip)
{

96
drivers/gpio/gpio-pca953x.c
View File

@ -306,37 +306,39 @@ static const struct regmap_config pca953x_i2c_regmap = {
.writeable_reg = pca953x_writeable_register,
.volatile_reg = pca953x_volatile_register,
.disable_locking = true,
.cache_type = REGCACHE_RBTREE,
/* REVISIT: should be 0x7f but some 24 bit chips use REG_ADDR_AI */
.max_register = 0xff,
.max_register = 0x7f,
};
static u8 pca953x_recalc_addr(struct pca953x_chip *chip, int reg, int off,
bool write, bool addrinc)
static const struct regmap_config pca953x_ai_i2c_regmap = {
.reg_bits = 8,
.val_bits = 8,
.read_flag_mask = REG_ADDR_AI,
.write_flag_mask = REG_ADDR_AI,
.readable_reg = pca953x_readable_register,
.writeable_reg = pca953x_writeable_register,
.volatile_reg = pca953x_volatile_register,
.cache_type = REGCACHE_RBTREE,
.max_register = 0x7f,
};
static u8 pca953x_recalc_addr(struct pca953x_chip *chip, int reg, int off)
{
int bank_shift = pca953x_bank_shift(chip);
int addr = (reg & PCAL_GPIO_MASK) << bank_shift;
int pinctrl = (reg & PCAL_PINCTRL_MASK) << 1;
u8 regaddr = pinctrl | addr | (off / BANK_SZ);
/* Single byte read doesn't need AI bit set. */
if (!addrinc)
return regaddr;
/* Chips with 24 and more GPIOs always support Auto Increment */
if (write && NBANK(chip) > 2)
regaddr |= REG_ADDR_AI;
/* PCA9575 needs address-increment on multi-byte writes */
if (PCA_CHIP_TYPE(chip->driver_data) == PCA957X_TYPE)
regaddr |= REG_ADDR_AI;
return regaddr;
}
static int pca953x_write_regs(struct pca953x_chip *chip, int reg, unsigned long *val)
{
u8 regaddr = pca953x_recalc_addr(chip, reg, 0, true, true);
u8 regaddr = pca953x_recalc_addr(chip, reg, 0);
u8 value[MAX_BANK];
int i, ret;
@ -354,7 +356,7 @@ static int pca953x_write_regs(struct pca953x_chip *chip, int reg, unsigned long
static int pca953x_read_regs(struct pca953x_chip *chip, int reg, unsigned long *val)
{
u8 regaddr = pca953x_recalc_addr(chip, reg, 0, false, true);
u8 regaddr = pca953x_recalc_addr(chip, reg, 0);
u8 value[MAX_BANK];
int i, ret;
@ -373,8 +375,7 @@ static int pca953x_read_regs(struct pca953x_chip *chip, int reg, unsigned long *
static int pca953x_gpio_direction_input(struct gpio_chip *gc, unsigned off)
{
struct pca953x_chip *chip = gpiochip_get_data(gc);
u8 dirreg = pca953x_recalc_addr(chip, chip->regs->direction, off,
true, false);
u8 dirreg = pca953x_recalc_addr(chip, chip->regs->direction, off);
u8 bit = BIT(off % BANK_SZ);
int ret;
@ -388,10 +389,8 @@ static int pca953x_gpio_direction_output(struct gpio_chip *gc,
unsigned off, int val)
{
struct pca953x_chip *chip = gpiochip_get_data(gc);
u8 dirreg = pca953x_recalc_addr(chip, chip->regs->direction, off,
true, false);
u8 outreg = pca953x_recalc_addr(chip, chip->regs->output, off,
true, false);
u8 dirreg = pca953x_recalc_addr(chip, chip->regs->direction, off);
u8 outreg = pca953x_recalc_addr(chip, chip->regs->output, off);
u8 bit = BIT(off % BANK_SZ);
int ret;
@ -411,8 +410,7 @@ static int pca953x_gpio_direction_output(struct gpio_chip *gc,
static int pca953x_gpio_get_value(struct gpio_chip *gc, unsigned off)
{
struct pca953x_chip *chip = gpiochip_get_data(gc);
u8 inreg = pca953x_recalc_addr(chip, chip->regs->input, off,
true, false);
u8 inreg = pca953x_recalc_addr(chip, chip->regs->input, off);
u8 bit = BIT(off % BANK_SZ);
u32 reg_val;
int ret;
@ -436,8 +434,7 @@ static int pca953x_gpio_get_value(struct gpio_chip *gc, unsigned off)
static void pca953x_gpio_set_value(struct gpio_chip *gc, unsigned off, int val)
{
struct pca953x_chip *chip = gpiochip_get_data(gc);
u8 outreg = pca953x_recalc_addr(chip, chip->regs->output, off,
true, false);
u8 outreg = pca953x_recalc_addr(chip, chip->regs->output, off);
u8 bit = BIT(off % BANK_SZ);
mutex_lock(&chip->i2c_lock);
@ -448,8 +445,7 @@ static void pca953x_gpio_set_value(struct gpio_chip *gc, unsigned off, int val)
static int pca953x_gpio_get_direction(struct gpio_chip *gc, unsigned off)
{
struct pca953x_chip *chip = gpiochip_get_data(gc);
u8 dirreg = pca953x_recalc_addr(chip, chip->regs->direction, off,
true, false);
u8 dirreg = pca953x_recalc_addr(chip, chip->regs->direction, off);
u8 bit = BIT(off % BANK_SZ);
u32 reg_val;
int ret;
@ -466,6 +462,23 @@ static int pca953x_gpio_get_direction(struct gpio_chip *gc, unsigned off)
return GPIO_LINE_DIRECTION_OUT;
}
static int pca953x_gpio_get_multiple(struct gpio_chip *gc,
unsigned long *mask, unsigned long *bits)
{
struct pca953x_chip *chip = gpiochip_get_data(gc);
DECLARE_BITMAP(reg_val, MAX_LINE);
int ret;
mutex_lock(&chip->i2c_lock);
ret = pca953x_read_regs(chip, chip->regs->input, reg_val);
mutex_unlock(&chip->i2c_lock);
if (ret)
return ret;
bitmap_replace(bits, bits, reg_val, mask, gc->ngpio);
return 0;
}
static void pca953x_gpio_set_multiple(struct gpio_chip *gc,
unsigned long *mask, unsigned long *bits)
{
@ -489,10 +502,8 @@ static int pca953x_gpio_set_pull_up_down(struct pca953x_chip *chip,
unsigned int offset,
unsigned long config)
{
u8 pull_en_reg = pca953x_recalc_addr(chip, PCAL953X_PULL_EN, offset,
true, false);
u8 pull_sel_reg = pca953x_recalc_addr(chip, PCAL953X_PULL_SEL, offset,
true, false);
u8 pull_en_reg = pca953x_recalc_addr(chip, PCAL953X_PULL_EN, offset);
u8 pull_sel_reg = pca953x_recalc_addr(chip, PCAL953X_PULL_SEL, offset);
u8 bit = BIT(offset % BANK_SZ);
int ret;
@ -551,6 +562,7 @@ static void pca953x_setup_gpio(struct pca953x_chip *chip, int gpios)
gc->get = pca953x_gpio_get_value;
gc->set = pca953x_gpio_set_value;
gc->get_direction = pca953x_gpio_get_direction;
gc->get_multiple = pca953x_gpio_get_multiple;
gc->set_multiple = pca953x_gpio_set_multiple;
gc->set_config = pca953x_gpio_set_config;
gc->can_sleep = true;
@ -863,6 +875,7 @@ static int pca953x_probe(struct i2c_client *client,
int ret;
u32 invert = 0;
struct regulator *reg;
const struct regmap_config *regmap_config;
chip = devm_kzalloc(&client->dev, sizeof(*chip), GFP_KERNEL);
if (chip == NULL)
@ -925,7 +938,17 @@ static int pca953x_probe(struct i2c_client *client,
i2c_set_clientdata(client, chip);
chip->regmap = devm_regmap_init_i2c(client, &pca953x_i2c_regmap);
pca953x_setup_gpio(chip, chip->driver_data & PCA_GPIO_MASK);
if (NBANK(chip) > 2 || PCA_CHIP_TYPE(chip->driver_data) == PCA957X_TYPE) {
dev_info(&client->dev, "using AI\n");
regmap_config = &pca953x_ai_i2c_regmap;
} else {
dev_info(&client->dev, "using no AI\n");
regmap_config = &pca953x_i2c_regmap;
}
chip->regmap = devm_regmap_init_i2c(client, regmap_config);
if (IS_ERR(chip->regmap)) {
ret = PTR_ERR(chip->regmap);
goto err_exit;
@ -956,7 +979,6 @@ static int pca953x_probe(struct i2c_client *client,
/* initialize cached registers from their original values.
* we can't share this chip with another i2c master.
*/
pca953x_setup_gpio(chip, chip->driver_data & PCA_GPIO_MASK);
if (PCA_CHIP_TYPE(chip->driver_data) == PCA953X_TYPE) {
chip->regs = &pca953x_regs;
@ -1154,7 +1176,7 @@ static struct i2c_driver pca953x_driver = {
.name = "pca953x",
.pm = &pca953x_pm_ops,
.of_match_table = pca953x_dt_ids,
.acpi_match_table = ACPI_PTR(pca953x_acpi_ids),
.acpi_match_table = pca953x_acpi_ids,
},
.probe = pca953x_probe,
.remove = pca953x_remove,

73
drivers/gpio/gpio-pch.c
View File

@ -2,6 +2,7 @@
/*
* Copyright (C) 2011 LAPIS Semiconductor Co., Ltd.
*/
#include <linux/bits.h>
#include <linux/gpio/driver.h>
#include <linux/interrupt.h>
#include <linux/irq.h>
@ -11,11 +12,11 @@
#include <linux/slab.h>
#define PCH_EDGE_FALLING 0
#define PCH_EDGE_RISING BIT(0)
#define PCH_LEVEL_L BIT(1)
#define PCH_LEVEL_H (BIT(0) | BIT(1))
#define PCH_EDGE_BOTH BIT(2)
#define PCH_IM_MASK (BIT(0) | BIT(1) | BIT(2))
#define PCH_EDGE_RISING 1
#define PCH_LEVEL_L 2
#define PCH_LEVEL_H 3
#define PCH_EDGE_BOTH 4
#define PCH_IM_MASK GENMASK(2, 0)
#define PCH_IRQ_BASE 24
@ -103,9 +104,9 @@ static void pch_gpio_set(struct gpio_chip *gpio, unsigned nr, int val)
spin_lock_irqsave(&chip->spinlock, flags);
reg_val = ioread32(&chip->reg->po);
if (val)
reg_val |= (1 << nr);
reg_val |= BIT(nr);
else
reg_val &= ~(1 << nr);
reg_val &= ~BIT(nr);
iowrite32(reg_val, &chip->reg->po);
spin_unlock_irqrestore(&chip->spinlock, flags);
@ -115,7 +116,7 @@ static int pch_gpio_get(struct gpio_chip *gpio, unsigned nr)
{
struct pch_gpio *chip = gpiochip_get_data(gpio);
return (ioread32(&chip->reg->pi) >> nr) & 1;
return !!(ioread32(&chip->reg->pi) & BIT(nr));
}
static int pch_gpio_direction_output(struct gpio_chip *gpio, unsigned nr,
@ -130,13 +131,14 @@ static int pch_gpio_direction_output(struct gpio_chip *gpio, unsigned nr,
reg_val = ioread32(&chip->reg->po);
if (val)
reg_val |= (1 << nr);
reg_val |= BIT(nr);
else
reg_val &= ~(1 << nr);
reg_val &= ~BIT(nr);
iowrite32(reg_val, &chip->reg->po);
pm = ioread32(&chip->reg->pm) & ((1 << gpio_pins[chip->ioh]) - 1);
pm |= (1 << nr);
pm = ioread32(&chip->reg->pm);
pm &= BIT(gpio_pins[chip->ioh]) - 1;
pm |= BIT(nr);
iowrite32(pm, &chip->reg->pm);
spin_unlock_irqrestore(&chip->spinlock, flags);
@ -151,8 +153,9 @@ static int pch_gpio_direction_input(struct gpio_chip *gpio, unsigned nr)
unsigned long flags;
spin_lock_irqsave(&chip->spinlock, flags);
pm = ioread32(&chip->reg->pm) & ((1 << gpio_pins[chip->ioh]) - 1);
pm &= ~(1 << nr);
pm = ioread32(&chip->reg->pm);
pm &= BIT(gpio_pins[chip->ioh]) - 1;
pm &= ~BIT(nr);
iowrite32(pm, &chip->reg->pm);
spin_unlock_irqrestore(&chip->spinlock, flags);
@ -226,17 +229,15 @@ static int pch_irq_type(struct irq_data *d, unsigned int type)
int ch, irq = d->irq;
ch = irq - chip->irq_base;
if (irq <= chip->irq_base + 7) {
if (irq < chip->irq_base + 8) {
im_reg = &chip->reg->im0;
im_pos = ch;
im_pos = ch - 0;
} else {
im_reg = &chip->reg->im1;
im_pos = ch - 8;
}
dev_dbg(chip->dev, "irq=%d type=%d ch=%d pos=%d\n", irq, type, ch, im_pos);
spin_lock_irqsave(&chip->spinlock, flags);
switch (type) {
case IRQ_TYPE_EDGE_RISING:
val = PCH_EDGE_RISING;
@ -254,20 +255,21 @@ static int pch_irq_type(struct irq_data *d, unsigned int type)
val = PCH_LEVEL_L;
break;
default:
goto unlock;
return 0;
}
spin_lock_irqsave(&chip->spinlock, flags);
/* Set interrupt mode */
im = ioread32(im_reg) & ~(PCH_IM_MASK << (im_pos * 4));
iowrite32(im | (val << (im_pos * 4)), im_reg);
/* And the handler */
if (type & (IRQ_TYPE_LEVEL_LOW | IRQ_TYPE_LEVEL_HIGH))
if (type & IRQ_TYPE_LEVEL_MASK)
irq_set_handler_locked(d, handle_level_irq);
else if (type & (IRQ_TYPE_EDGE_FALLING | IRQ_TYPE_EDGE_RISING))
else if (type & IRQ_TYPE_EDGE_BOTH)
irq_set_handler_locked(d, handle_edge_irq);
unlock:
spin_unlock_irqrestore(&chip->spinlock, flags);
return 0;
}
@ -277,7 +279,7 @@ static void pch_irq_unmask(struct irq_data *d)
struct irq_chip_generic *gc = irq_data_get_irq_chip_data(d);
struct pch_gpio *chip = gc->private;
iowrite32(1 << (d->irq - chip->irq_base), &chip->reg->imaskclr);
iowrite32(BIT(d->irq - chip->irq_base), &chip->reg->imaskclr);
}
static void pch_irq_mask(struct irq_data *d)
@ -285,7 +287,7 @@ static void pch_irq_mask(struct irq_data *d)
struct irq_chip_generic *gc = irq_data_get_irq_chip_data(d);
struct pch_gpio *chip = gc->private;
iowrite32(1 << (d->irq - chip->irq_base), &chip->reg->imask);
iowrite32(BIT(d->irq - chip->irq_base), &chip->reg->imask);
}
static void pch_irq_ack(struct irq_data *d)
@ -293,21 +295,22 @@ static void pch_irq_ack(struct irq_data *d)
struct irq_chip_generic *gc = irq_data_get_irq_chip_data(d);
struct pch_gpio *chip = gc->private;
iowrite32(1 << (d->irq - chip->irq_base), &chip->reg->iclr);
iowrite32(BIT(d->irq - chip->irq_base), &chip->reg->iclr);
}
static irqreturn_t pch_gpio_handler(int irq, void *dev_id)
{
struct pch_gpio *chip = dev_id;
unsigned long reg_val = ioread32(&chip->reg->istatus);
int i, ret = IRQ_NONE;
int i;
for_each_set_bit(i, &reg_val, gpio_pins[chip->ioh]) {
dev_dbg(chip->dev, "[%d]:irq=%d status=0x%lx\n", i, irq, reg_val);
dev_dbg(chip->dev, "irq=%d status=0x%lx\n", irq, reg_val);
reg_val &= BIT(gpio_pins[chip->ioh]) - 1;
for_each_set_bit(i, &reg_val, gpio_pins[chip->ioh])
generic_handle_irq(chip->irq_base + i);
ret = IRQ_HANDLED;
}
return ret;
return IRQ_RETVAL(reg_val);
}
static int pch_gpio_alloc_generic_chip(struct pch_gpio *chip,
@ -344,7 +347,6 @@ static int pch_gpio_probe(struct pci_dev *pdev,
s32 ret;
struct pch_gpio *chip;
int irq_base;
u32 msk;
chip = devm_kzalloc(&pdev->dev, sizeof(*chip), GFP_KERNEL);
if (chip == NULL)
@ -357,7 +359,7 @@ static int pch_gpio_probe(struct pci_dev *pdev,
return ret;
}
ret = pcim_iomap_regions(pdev, 1 << 1, KBUILD_MODNAME);
ret = pcim_iomap_regions(pdev, BIT(1), KBUILD_MODNAME);
if (ret) {
dev_err(&pdev->dev, "pci_request_regions FAILED-%d", ret);
return ret;
@ -393,9 +395,8 @@ static int pch_gpio_probe(struct pci_dev *pdev,
chip->irq_base = irq_base;
/* Mask all interrupts, but enable them */
msk = (1 << gpio_pins[chip->ioh]) - 1;
iowrite32(msk, &chip->reg->imask);
iowrite32(msk, &chip->reg->ien);
iowrite32(BIT(gpio_pins[chip->ioh]) - 1, &chip->reg->imask);
iowrite32(BIT(gpio_pins[chip->ioh]) - 1, &chip->reg->ien);
ret = devm_request_irq(&pdev->dev, pdev->irq, pch_gpio_handler,
IRQF_SHARED, KBUILD_MODNAME, chip);

9
drivers/gpio/gpio-pl061.c
View File

@ -16,6 +16,7 @@
#include <linux/interrupt.h>
#include <linux/irq.h>
#include <linux/irqchip/chained_irq.h>
#include <linux/module.h>
#include <linux/bitops.h>
#include <linux/gpio/driver.h>
#include <linux/device.h>
@ -408,6 +409,7 @@ static const struct amba_id pl061_ids[] = {
},
{ 0, 0 },
};
MODULE_DEVICE_TABLE(amba, pl061_ids);
static struct amba_driver pl061_gpio_driver = {
.drv = {
@ -419,9 +421,6 @@ static struct amba_driver pl061_gpio_driver = {
.id_table = pl061_ids,
.probe = pl061_probe,
};
module_amba_driver(pl061_gpio_driver);
static int __init pl061_gpio_init(void)
{
return amba_driver_register(&pl061_gpio_driver);
}
device_initcall(pl061_gpio_init);
MODULE_LICENSE("GPL v2");

4
drivers/gpio/gpio-rcar.c
View File

@ -250,8 +250,10 @@ static int gpio_rcar_request(struct gpio_chip *chip, unsigned offset)
int error;
error = pm_runtime_get_sync(p->dev);
if (error < 0)
if (error < 0) {
pm_runtime_put(p->dev);
return error;
}
error = pinctrl_gpio_request(chip->base + offset);
if (error)

349
drivers/gpio/gpio-regmap.c Normal file
View File

@ -0,0 +1,349 @@
// SPDX-License-Identifier: GPL-2.0-only
/*
* regmap based generic GPIO driver
*
* Copyright 2020 Michael Walle <michael@walle.cc>
*/
#include <linux/gpio/driver.h>
#include <linux/gpio/regmap.h>
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/regmap.h>
struct gpio_regmap {
struct device *parent;
struct regmap *regmap;
struct gpio_chip gpio_chip;
int reg_stride;
int ngpio_per_reg;
unsigned int reg_dat_base;
unsigned int reg_set_base;
unsigned int reg_clr_base;
unsigned int reg_dir_in_base;
unsigned int reg_dir_out_base;
int (*reg_mask_xlate)(struct gpio_regmap *gpio, unsigned int base,
unsigned int offset, unsigned int *reg,
unsigned int *mask);
void *driver_data;
};
static unsigned int gpio_regmap_addr(unsigned int addr)
{
if (addr == GPIO_REGMAP_ADDR_ZERO)
return 0;
return addr;
}
static int gpio_regmap_simple_xlate(struct gpio_regmap *gpio,
unsigned int base, unsigned int offset,
unsigned int *reg, unsigned int *mask)
{
unsigned int line = offset % gpio->ngpio_per_reg;
unsigned int stride = offset / gpio->ngpio_per_reg;
*reg = base + stride * gpio->reg_stride;
*mask = BIT(line);
return 0;
}
static int gpio_regmap_get(struct gpio_chip *chip, unsigned int offset)
{
struct gpio_regmap *gpio = gpiochip_get_data(chip);
unsigned int base, val, reg, mask;
int ret;
/* we might not have an output register if we are input only */
if (gpio->reg_dat_base)
base = gpio_regmap_addr(gpio->reg_dat_base);
else
base = gpio_regmap_addr(gpio->reg_set_base);
ret = gpio->reg_mask_xlate(gpio, base, offset, &reg, &mask);
if (ret)
return ret;
ret = regmap_read(gpio->regmap, reg, &val);
if (ret)
return ret;
return !!(val & mask);
}
static void gpio_regmap_set(struct gpio_chip *chip, unsigned int offset,
int val)
{
struct gpio_regmap *gpio = gpiochip_get_data(chip);
unsigned int base = gpio_regmap_addr(gpio->reg_set_base);
unsigned int reg, mask;
gpio->reg_mask_xlate(gpio, base, offset, &reg, &mask);
if (val)
regmap_update_bits(gpio->regmap, reg, mask, mask);
else
regmap_update_bits(gpio->regmap, reg, mask, 0);
}
static void gpio_regmap_set_with_clear(struct gpio_chip *chip,
unsigned int offset, int val)
{
struct gpio_regmap *gpio = gpiochip_get_data(chip);
unsigned int base, reg, mask;
if (val)
base = gpio_regmap_addr(gpio->reg_set_base);
else
base = gpio_regmap_addr(gpio->reg_clr_base);
gpio->reg_mask_xlate(gpio, base, offset, &reg, &mask);
regmap_write(gpio->regmap, reg, mask);
}
static int gpio_regmap_get_direction(struct gpio_chip *chip,
unsigned int offset)
{
struct gpio_regmap *gpio = gpiochip_get_data(chip);
unsigned int base, val, reg, mask;
int invert, ret;
if (gpio->reg_dir_out_base) {
base = gpio_regmap_addr(gpio->reg_dir_out_base);
invert = 0;
} else if (gpio->reg_dir_in_base) {
base = gpio_regmap_addr(gpio->reg_dir_in_base);
invert = 1;
} else {
return -EOPNOTSUPP;
}
ret = gpio->reg_mask_xlate(gpio, base, offset, &reg, &mask);
if (ret)
return ret;
ret = regmap_read(gpio->regmap, reg, &val);
if (ret)
return ret;
if (!!(val & mask) ^ invert)
return GPIO_LINE_DIRECTION_OUT;
else
return GPIO_LINE_DIRECTION_IN;
}
static int gpio_regmap_set_direction(struct gpio_chip *chip,
unsigned int offset, bool output)
{
struct gpio_regmap *gpio = gpiochip_get_data(chip);
unsigned int base, val, reg, mask;
int invert, ret;
if (gpio->reg_dir_out_base) {
base = gpio_regmap_addr(gpio->reg_dir_out_base);
invert = 0;
} else if (gpio->reg_dir_in_base) {
base = gpio_regmap_addr(gpio->reg_dir_in_base);
invert = 1;
} else {
return -EOPNOTSUPP;
}
ret = gpio->reg_mask_xlate(gpio, base, offset, &reg, &mask);
if (ret)
return ret;
if (invert)
val = output ? 0 : mask;
else
val = output ? mask : 0;
return regmap_update_bits(gpio->regmap, reg, mask, val);
}
static int gpio_regmap_direction_input(struct gpio_chip *chip,
unsigned int offset)
{
return gpio_regmap_set_direction(chip, offset, false);
}
static int gpio_regmap_direction_output(struct gpio_chip *chip,
unsigned int offset, int value)
{
gpio_regmap_set(chip, offset, value);
return gpio_regmap_set_direction(chip, offset, true);
}
void gpio_regmap_set_drvdata(struct gpio_regmap *gpio, void *data)
{
gpio->driver_data = data;
}
EXPORT_SYMBOL_GPL(gpio_regmap_set_drvdata);
void *gpio_regmap_get_drvdata(struct gpio_regmap *gpio)
{
return gpio->driver_data;
}
EXPORT_SYMBOL_GPL(gpio_regmap_get_drvdata);
/**
* gpio_regmap_register() - Register a generic regmap GPIO controller
* @config: configuration for gpio_regmap
*
* Return: A pointer to the registered gpio_regmap or ERR_PTR error value.
*/
struct gpio_regmap *gpio_regmap_register(const struct gpio_regmap_config *config)
{
struct gpio_regmap *gpio;
struct gpio_chip *chip;
int ret;
if (!config->parent)
return ERR_PTR(-EINVAL);
if (!config->ngpio)
return ERR_PTR(-EINVAL);
/* we need at least one */
if (!config->reg_dat_base && !config->reg_set_base)
return ERR_PTR(-EINVAL);
/* if we have a direction register we need both input and output */
if ((config->reg_dir_out_base || config->reg_dir_in_base) &&
(!config->reg_dat_base || !config->reg_set_base))
return ERR_PTR(-EINVAL);
/* we don't support having both registers simultaneously for now */
if (config->reg_dir_out_base && config->reg_dir_in_base)
return ERR_PTR(-EINVAL);
gpio = kzalloc(sizeof(*gpio), GFP_KERNEL);
if (!gpio)
return ERR_PTR(-ENOMEM);
gpio->parent = config->parent;
gpio->regmap = config->regmap;
gpio->ngpio_per_reg = config->ngpio_per_reg;
gpio->reg_stride = config->reg_stride;
gpio->reg_mask_xlate = config->reg_mask_xlate;
gpio->reg_dat_base = config->reg_dat_base;
gpio->reg_set_base = config->reg_set_base;
gpio->reg_clr_base = config->reg_clr_base;
gpio->reg_dir_in_base = config->reg_dir_in_base;
gpio->reg_dir_out_base = config->reg_dir_out_base;
/* if not set, assume there is only one register */
if (!gpio->ngpio_per_reg)
gpio->ngpio_per_reg = config->ngpio;
/* if not set, assume they are consecutive */
if (!gpio->reg_stride)
gpio->reg_stride = 1;
if (!gpio->reg_mask_xlate)
gpio->reg_mask_xlate = gpio_regmap_simple_xlate;
chip = &gpio->gpio_chip;
chip->parent = config->parent;
chip->base = -1;
chip->ngpio = config->ngpio;
chip->names = config->names;
chip->label = config->label ?: dev_name(config->parent);
/*
* If our regmap is fast_io we should probably set can_sleep to false.
* Right now, the regmap doesn't save this property, nor is there any
* access function for it.
* The only regmap type which uses fast_io is regmap-mmio. For now,
* assume a safe default of true here.
*/
chip->can_sleep = true;
chip->get = gpio_regmap_get;
if (gpio->reg_set_base && gpio->reg_clr_base)
chip->set = gpio_regmap_set_with_clear;
else if (gpio->reg_set_base)
chip->set = gpio_regmap_set;
if (gpio->reg_dir_in_base || gpio->reg_dir_out_base) {
chip->get_direction = gpio_regmap_get_direction;
chip->direction_input = gpio_regmap_direction_input;
chip->direction_output = gpio_regmap_direction_output;
}
ret = gpiochip_add_data(chip, gpio);
if (ret < 0)
goto err_free_gpio;
if (config->irq_domain) {
ret = gpiochip_irqchip_add_domain(chip, config->irq_domain);
if (ret)
goto err_remove_gpiochip;
}
return gpio;
err_remove_gpiochip:
gpiochip_remove(chip);
err_free_gpio:
kfree(gpio);
return ERR_PTR(ret);
}
EXPORT_SYMBOL_GPL(gpio_regmap_register);
/**
* gpio_regmap_unregister() - Unregister a generic regmap GPIO controller
* @gpio: gpio_regmap device to unregister
*/
void gpio_regmap_unregister(struct gpio_regmap *gpio)
{
gpiochip_remove(&gpio->gpio_chip);
kfree(gpio);
}
EXPORT_SYMBOL_GPL(gpio_regmap_unregister);
static void devm_gpio_regmap_unregister(struct device *dev, void *res)
{
gpio_regmap_unregister(*(struct gpio_regmap **)res);
}
/**
* devm_gpio_regmap_register() - resource managed gpio_regmap_register()
* @dev: device that is registering this GPIO device
* @config: configuration for gpio_regmap
*
* Managed gpio_regmap_register(). For generic regmap GPIO device registered by
* this function, gpio_regmap_unregister() is automatically called on driver
* detach. See gpio_regmap_register() for more information.
*
* Return: A pointer to the registered gpio_regmap or ERR_PTR error value.
*/
struct gpio_regmap *devm_gpio_regmap_register(struct device *dev,
const struct gpio_regmap_config *config)
{
struct gpio_regmap **ptr, *gpio;
ptr = devres_alloc(devm_gpio_regmap_unregister, sizeof(*ptr),
GFP_KERNEL);
if (!ptr)
return ERR_PTR(-ENOMEM);
gpio = gpio_regmap_register(config);
if (!IS_ERR(gpio)) {
*ptr = gpio;
devres_add(dev, ptr);
} else {
devres_free(ptr);
}
return gpio;
}
EXPORT_SYMBOL_GPL(devm_gpio_regmap_register);
MODULE_AUTHOR("Michael Walle <michael@walle.cc>");
MODULE_DESCRIPTION("GPIO generic regmap driver core");
MODULE_LICENSE("GPL");

1
drivers/gpio/gpio-tegra186.c
View File

@ -894,6 +894,7 @@ static const struct of_device_id tegra186_gpio_of_match[] = {
/* sentinel */
}
};
MODULE_DEVICE_TABLE(of, tegra186_gpio_of_match);
static struct platform_driver tegra186_gpio_driver = {
.driver = {

14
drivers/gpio/gpio-xgene-sb.c
View File

@ -10,8 +10,8 @@
#include <linux/module.h>
#include <linux/io.h>
#include <linux/of.h>
#include <linux/platform_device.h>
#include <linux/of_gpio.h>
#include <linux/gpio/driver.h>
#include <linux/acpi.h>
@ -122,7 +122,7 @@ static int xgene_gpio_sb_to_irq(struct gpio_chip *gc, u32 gpio)
fwspec.fwnode = gc->parent->fwnode;
fwspec.param_count = 2;
fwspec.param[0] = GPIO_TO_HWIRQ(priv, gpio);
fwspec.param[1] = IRQ_TYPE_NONE;
fwspec.param[1] = IRQ_TYPE_EDGE_RISING;
return irq_create_fwspec_mapping(&fwspec);
}
@ -290,10 +290,8 @@ static int xgene_gpio_sb_probe(struct platform_device *pdev)
dev_info(&pdev->dev, "X-Gene GPIO Standby driver registered\n");
if (priv->nirq > 0) {
/* Register interrupt handlers for gpio signaled acpi events */
acpi_gpiochip_request_interrupts(&priv->gc);
}
/* Register interrupt handlers for GPIO signaled ACPI Events */
acpi_gpiochip_request_interrupts(&priv->gc);
return ret;
}
@ -302,9 +300,7 @@ static int xgene_gpio_sb_remove(struct platform_device *pdev)
{
struct xgene_gpio_sb *priv = platform_get_drvdata(pdev);
if (priv->nirq > 0) {
acpi_gpiochip_free_interrupts(&priv->gc);
}
acpi_gpiochip_free_interrupts(&priv->gc);
irq_domain_remove(priv->irq_domain);

6
drivers/gpio/gpiolib-acpi.c
View File

@ -1353,7 +1353,7 @@ int acpi_gpio_count(struct device *dev, const char *con_id)
}
/* Run deferred acpi_gpiochip_request_irqs() */
static int acpi_gpio_handle_deferred_request_irqs(void)
static int __init acpi_gpio_handle_deferred_request_irqs(void)
{
struct acpi_gpio_chip *acpi_gpio, *tmp;
@ -1371,7 +1371,7 @@ static int acpi_gpio_handle_deferred_request_irqs(void)
/* We must use _sync so that this runs after the first deferred_probe run */
late_initcall_sync(acpi_gpio_handle_deferred_request_irqs);
static const struct dmi_system_id gpiolib_acpi_quirks[] = {
static const struct dmi_system_id gpiolib_acpi_quirks[] __initconst = {
{
/*
* The Minix Neo Z83-4 has a micro-USB-B id-pin handler for
@ -1455,7 +1455,7 @@ static const struct dmi_system_id gpiolib_acpi_quirks[] = {
{} /* Terminating entry */
};
static int acpi_gpio_setup_params(void)
static int __init acpi_gpio_setup_params(void)
{
const struct acpi_gpiolib_dmi_quirk *quirk = NULL;
const struct dmi_system_id *id;

5
drivers/gpio/gpiolib-devprop.c
View File

@ -37,8 +37,11 @@ void devprop_gpiochip_set_names(struct gpio_chip *chip,
if (count < 0)
return;
if (count > gdev->ngpio)
if (count > gdev->ngpio) {
dev_warn(&gdev->dev, "gpio-line-names is length %d but should be at most length %d",
count, gdev->ngpio);
count = gdev->ngpio;
}
names = kcalloc(count, sizeof(*names), GFP_KERNEL);
if (!names)

10
drivers/gpio/gpiolib-of.c
View File

@ -344,6 +344,12 @@ struct gpio_desc *gpiod_get_from_of_node(struct device_node *node,
if (transitory)
lflags |= GPIO_TRANSITORY;
if (flags & OF_GPIO_PULL_UP)
lflags |= GPIO_PULL_UP;
if (flags & OF_GPIO_PULL_DOWN)
lflags |= GPIO_PULL_DOWN;
ret = gpiod_configure_flags(desc, propname, lflags, dflags);
if (ret < 0) {
gpiod_put(desc);
@ -585,6 +591,10 @@ static struct gpio_desc *of_parse_own_gpio(struct device_node *np,
*lflags |= GPIO_ACTIVE_LOW;
if (xlate_flags & OF_GPIO_TRANSITORY)
*lflags |= GPIO_TRANSITORY;
if (xlate_flags & OF_GPIO_PULL_UP)
*lflags |= GPIO_PULL_UP;
if (xlate_flags & OF_GPIO_PULL_DOWN)
*lflags |= GPIO_PULL_DOWN;
if (of_property_read_bool(np, "input"))
*dflags |= GPIOD_IN;

167
drivers/gpio/gpiolib.c
View File

@ -296,6 +296,9 @@ static int gpiodev_add_to_list(struct gpio_device *gdev)
/*
* Convert a GPIO name to its descriptor
* Note that there is no guarantee that GPIO names are globally unique!
* Hence this function will return, if it exists, a reference to the first GPIO
* line found that matches the given name.
*/
static struct gpio_desc *gpio_name_to_desc(const char * const name)
{
@ -329,10 +332,12 @@ static struct gpio_desc *gpio_name_to_desc(const char * const name)
}
/*
* Takes the names from gc->names and checks if they are all unique. If they
* are, they are assigned to their gpio descriptors.
* Take the names from gc->names and assign them to their GPIO descriptors.
* Warn if a name is already used for a GPIO line on a different GPIO chip.
*
* Warning if one of the names is already used for a different GPIO.
* Note that:
* 1. Non-unique names are still accepted,
* 2. Name collisions within the same GPIO chip are not reported.
*/
static int gpiochip_set_desc_names(struct gpio_chip *gc)
{
@ -1267,8 +1272,7 @@ static long gpio_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
if (copy_to_user(ip, &chipinfo, sizeof(chipinfo)))
return -EFAULT;
return 0;
} else if (cmd == GPIO_GET_LINEINFO_IOCTL ||
cmd == GPIO_GET_LINEINFO_WATCH_IOCTL) {
} else if (cmd == GPIO_GET_LINEINFO_IOCTL) {
struct gpioline_info lineinfo;
if (copy_from_user(&lineinfo, ip, sizeof(lineinfo)))
@ -1280,8 +1284,28 @@ static long gpio_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
hwgpio = gpio_chip_hwgpio(desc);
if (cmd == GPIO_GET_LINEINFO_WATCH_IOCTL &&
test_bit(hwgpio, priv->watched_lines))
gpio_desc_to_lineinfo(desc, &lineinfo);
if (copy_to_user(ip, &lineinfo, sizeof(lineinfo)))
return -EFAULT;
return 0;
} else if (cmd == GPIO_GET_LINEHANDLE_IOCTL) {
return linehandle_create(gdev, ip);
} else if (cmd == GPIO_GET_LINEEVENT_IOCTL) {
return lineevent_create(gdev, ip);
} else if (cmd == GPIO_GET_LINEINFO_WATCH_IOCTL) {
struct gpioline_info lineinfo;
if (copy_from_user(&lineinfo, ip, sizeof(lineinfo)))
return -EFAULT;
desc = gpiochip_get_desc(gc, lineinfo.line_offset);
if (IS_ERR(desc))
return PTR_ERR(desc);
hwgpio = gpio_chip_hwgpio(desc);
if (test_bit(hwgpio, priv->watched_lines))
return -EBUSY;
gpio_desc_to_lineinfo(desc, &lineinfo);
@ -1289,14 +1313,8 @@ static long gpio_ioctl(struct file *filp, unsigned int cmd, unsigned long arg)
if (copy_to_user(ip, &lineinfo, sizeof(lineinfo)))
return -EFAULT;
if (cmd == GPIO_GET_LINEINFO_WATCH_IOCTL)
set_bit(hwgpio, priv->watched_lines);
set_bit(hwgpio, priv->watched_lines);
return 0;
} else if (cmd == GPIO_GET_LINEHANDLE_IOCTL) {
return linehandle_create(gdev, ip);
} else if (cmd == GPIO_GET_LINEEVENT_IOCTL) {
return lineevent_create(gdev, ip);
} else if (cmd == GPIO_GET_LINEINFO_UNWATCH_IOCTL) {
if (copy_from_user(&offset, ip, sizeof(offset)))
return -EFAULT;
@ -1538,9 +1556,8 @@ static int gpiochip_setup_dev(struct gpio_device *gdev)
/* From this point, the .release() function cleans up gpio_device */
gdev->dev.release = gpiodevice_release;
pr_debug("%s: registered GPIOs %d to %d on device: %s (%s)\n",
__func__, gdev->base, gdev->base + gdev->ngpio - 1,
dev_name(&gdev->dev), gdev->chip->label ? : "generic");
dev_dbg(&gdev->dev, "registered GPIOs %d to %d on %s\n", gdev->base,
gdev->base + gdev->ngpio - 1, gdev->chip->label ? : "generic");
return 0;
@ -1556,8 +1573,8 @@ static void gpiochip_machine_hog(struct gpio_chip *gc, struct gpiod_hog *hog)
desc = gpiochip_get_desc(gc, hog->chip_hwnum);
if (IS_ERR(desc)) {
pr_err("%s: unable to get GPIO desc: %ld\n",
__func__, PTR_ERR(desc));
chip_err(gc, "%s: unable to get GPIO desc: %ld\n", __func__,
PTR_ERR(desc));
return;
}
@ -1566,8 +1583,8 @@ static void gpiochip_machine_hog(struct gpio_chip *gc, struct gpiod_hog *hog)
rv = gpiod_hog(desc, hog->line_name, hog->lflags, hog->dflags);
if (rv)
pr_err("%s: unable to hog GPIO line (%s:%u): %d\n",
__func__, gc->label, hog->chip_hwnum, rv);
gpiod_err(desc, "%s: unable to hog GPIO line (%s:%u): %d\n",
__func__, gc->label, hog->chip_hwnum, rv);
}
static void machine_gpiochip_add(struct gpio_chip *gc)
@ -1592,8 +1609,8 @@ static void gpiochip_setup_devs(void)
list_for_each_entry(gdev, &gpio_devices, list) {
ret = gpiochip_setup_dev(gdev);
if (ret)
pr_err("%s: Failed to initialize gpio device (%d)\n",
dev_name(&gdev->dev), ret);
dev_err(&gdev->dev,
"Failed to initialize gpio device (%d)\n", ret);
}
}
@ -2461,32 +2478,37 @@ static void gpiochip_irq_relres(struct irq_data *d)
gpiochip_relres_irq(gc, d->hwirq);
}
static void gpiochip_irq_mask(struct irq_data *d)
{
struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
if (gc->irq.irq_mask)
gc->irq.irq_mask(d);
gpiochip_disable_irq(gc, d->hwirq);
}
static void gpiochip_irq_unmask(struct irq_data *d)
{
struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
gpiochip_enable_irq(gc, d->hwirq);
if (gc->irq.irq_unmask)
gc->irq.irq_unmask(d);
}
static void gpiochip_irq_enable(struct irq_data *d)
{
struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
gpiochip_enable_irq(gc, d->hwirq);
if (gc->irq.irq_enable)
gc->irq.irq_enable(d);
else
gc->irq.chip->irq_unmask(d);
gc->irq.irq_enable(d);
}
static void gpiochip_irq_disable(struct irq_data *d)
{
struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
/*
* Since we override .irq_disable() we need to mimic the
* behaviour of __irq_disable() in irq/chip.c.
* First call .irq_disable() if it exists, else mimic the
* behaviour of mask_irq() which calls .irq_mask() if
* it exists.
*/
if (gc->irq.irq_disable)
gc->irq.irq_disable(d);
else if (gc->irq.chip->irq_mask)
gc->irq.chip->irq_mask(d);
gc->irq.irq_disable(d);
gpiochip_disable_irq(gc, d->hwirq);
}
@ -2511,10 +2533,22 @@ static void gpiochip_set_irq_hooks(struct gpio_chip *gc)
"detected irqchip that is shared with multiple gpiochips: please fix the driver.\n");
return;
}
gc->irq.irq_enable = irqchip->irq_enable;
gc->irq.irq_disable = irqchip->irq_disable;
irqchip->irq_enable = gpiochip_irq_enable;
irqchip->irq_disable = gpiochip_irq_disable;
if (irqchip->irq_disable) {
gc->irq.irq_disable = irqchip->irq_disable;
irqchip->irq_disable = gpiochip_irq_disable;
} else {
gc->irq.irq_mask = irqchip->irq_mask;
irqchip->irq_mask = gpiochip_irq_mask;
}
if (irqchip->irq_enable) {
gc->irq.irq_enable = irqchip->irq_enable;
irqchip->irq_enable = gpiochip_irq_enable;
} else {
gc->irq.irq_unmask = irqchip->irq_unmask;
irqchip->irq_unmask = gpiochip_irq_unmask;
}
}
/**
@ -2702,7 +2736,7 @@ int gpiochip_irqchip_add_key(struct gpio_chip *gc,
return -EINVAL;
if (!gc->parent) {
pr_err("missing gpiochip .dev parent pointer\n");
chip_err(gc, "missing gpiochip .dev parent pointer\n");
return -EINVAL;
}
gc->irq.threaded = threaded;
@ -2752,6 +2786,26 @@ int gpiochip_irqchip_add_key(struct gpio_chip *gc,
}
EXPORT_SYMBOL_GPL(gpiochip_irqchip_add_key);
/**
* gpiochip_irqchip_add_domain() - adds an irqdomain to a gpiochip
* @gc: the gpiochip to add the irqchip to
* @domain: the irqdomain to add to the gpiochip
*
* This function adds an IRQ domain to the gpiochip.
*/
int gpiochip_irqchip_add_domain(struct gpio_chip *gc,
struct irq_domain *domain)
{
if (!domain)
return -EINVAL;
gc->to_irq = gpiochip_to_irq;
gc->irq.domain = domain;
return 0;
}
EXPORT_SYMBOL_GPL(gpiochip_irqchip_add_domain);
#else /* CONFIG_GPIOLIB_IRQCHIP */
static inline int gpiochip_add_irqchip(struct gpio_chip *gc,
@ -4653,7 +4707,7 @@ static struct gpio_desc *gpiod_find(struct device *dev, const char *con_id,
if (!table)
return desc;
for (p = &table->table[0]; p->chip_label; p++) {
for (p = &table->table[0]; p->key; p++) {
struct gpio_chip *gc;
/* idx must always match exactly */
@ -4664,18 +4718,30 @@ static struct gpio_desc *gpiod_find(struct device *dev, const char *con_id,
if (p->con_id && (!con_id || strcmp(p->con_id, con_id)))
continue;
gc = find_chip_by_name(p->chip_label);
if (p->chip_hwnum == U16_MAX) {
desc = gpio_name_to_desc(p->key);
if (desc) {
*flags = p->flags;
return desc;
}
dev_warn(dev, "cannot find GPIO line %s, deferring\n",
p->key);
return ERR_PTR(-EPROBE_DEFER);
}
gc = find_chip_by_name(p->key);
if (!gc) {
/*
* As the lookup table indicates a chip with
* p->chip_label should exist, assume it may
* p->key should exist, assume it may
* still appear later and let the interested
* consumer be probed again or let the Deferred
* Probe infrastructure handle the error.
*/
dev_warn(dev, "cannot find GPIO chip %s, deferring\n",
p->chip_label);
p->key);
return ERR_PTR(-EPROBE_DEFER);
}
@ -4706,7 +4772,7 @@ static int platform_gpio_count(struct device *dev, const char *con_id)
if (!table)
return -ENOENT;
for (p = &table->table[0]; p->chip_label; p++) {
for (p = &table->table[0]; p->key; p++) {
if ((con_id && p->con_id && !strcmp(con_id, p->con_id)) ||
(!con_id && !p->con_id))
count++;
@ -4877,7 +4943,7 @@ int gpiod_configure_flags(struct gpio_desc *desc, const char *con_id,
/* No particular flag request, return here... */
if (!(dflags & GPIOD_FLAGS_BIT_DIR_SET)) {
pr_debug("no flags found for %s\n", con_id);
gpiod_dbg(desc, "no flags found for %s\n", con_id);
return 0;
}
@ -5108,8 +5174,7 @@ int gpiod_hog(struct gpio_desc *desc, const char *name,
/* Mark GPIO as hogged so it can be identified and removed later */
set_bit(FLAG_IS_HOGGED, &desc->flags);
pr_info("GPIO line %d (%s) hogged as %s%s\n",
desc_to_gpio(desc), name,
gpiod_info(desc, "hogged as %s%s\n",
(dflags & GPIOD_FLAGS_BIT_DIR_OUT) ? "output" : "input",
(dflags & GPIOD_FLAGS_BIT_DIR_OUT) ?
(dflags & GPIOD_FLAGS_BIT_DIR_VAL) ? "/high" : "/low" : "");

27
drivers/gpio/gpiolib.h
View File

@ -81,8 +81,7 @@ struct gpio_array {
unsigned long invert_mask[];
};
struct gpio_desc *gpiochip_get_desc(struct gpio_chip *chip,
unsigned int hwnum);
struct gpio_desc *gpiochip_get_desc(struct gpio_chip *gc, unsigned int hwnum);
int gpiod_get_array_value_complex(bool raw, bool can_sleep,
unsigned int array_size,
struct gpio_desc **desc_array,
@ -163,18 +162,18 @@ static inline int gpio_chip_hwgpio(const struct gpio_desc *desc)
/* With chip prefix */
#define chip_emerg(chip, fmt, ...) \
dev_emerg(&chip->gpiodev->dev, "(%s): " fmt, chip->label, ##__VA_ARGS__)
#define chip_crit(chip, fmt, ...) \
dev_crit(&chip->gpiodev->dev, "(%s): " fmt, chip->label, ##__VA_ARGS__)
#define chip_err(chip, fmt, ...) \
dev_err(&chip->gpiodev->dev, "(%s): " fmt, chip->label, ##__VA_ARGS__)
#define chip_warn(chip, fmt, ...) \
dev_warn(&chip->gpiodev->dev, "(%s): " fmt, chip->label, ##__VA_ARGS__)
#define chip_info(chip, fmt, ...) \
dev_info(&chip->gpiodev->dev, "(%s): " fmt, chip->label, ##__VA_ARGS__)
#define chip_dbg(chip, fmt, ...) \
dev_dbg(&chip->gpiodev->dev, "(%s): " fmt, chip->label, ##__VA_ARGS__)
#define chip_emerg(gc, fmt, ...) \
dev_emerg(&gc->gpiodev->dev, "(%s): " fmt, gc->label, ##__VA_ARGS__)
#define chip_crit(gc, fmt, ...) \
dev_crit(&gc->gpiodev->dev, "(%s): " fmt, gc->label, ##__VA_ARGS__)
#define chip_err(gc, fmt, ...) \
dev_err(&gc->gpiodev->dev, "(%s): " fmt, gc->label, ##__VA_ARGS__)
#define chip_warn(gc, fmt, ...) \
dev_warn(&gc->gpiodev->dev, "(%s): " fmt, gc->label, ##__VA_ARGS__)
#define chip_info(gc, fmt, ...) \
dev_info(&gc->gpiodev->dev, "(%s): " fmt, gc->label, ##__VA_ARGS__)
#define chip_dbg(gc, fmt, ...) \
dev_dbg(&gc->gpiodev->dev, "(%s): " fmt, gc->label, ##__VA_ARGS__)
#ifdef CONFIG_GPIO_SYSFS

6
drivers/i2c/busses/i2c-i801.c
View File

@ -1439,9 +1439,9 @@ static int i801_add_mux(struct i801_priv *priv)
return -ENOMEM;
lookup->dev_id = "i2c-mux-gpio";
for (i = 0; i < mux_config->n_gpios; i++) {
lookup->table[i].chip_label = mux_config->gpio_chip;
lookup->table[i].chip_hwnum = mux_config->gpios[i];
lookup->table[i].con_id = "mux";
lookup->table[i] = (struct gpiod_lookup)
GPIO_LOOKUP(mux_config->gpio_chip,
mux_config->gpios[i], "mux", 0);
}
gpiod_add_lookup_table(lookup);
priv->lookup = lookup;

1
drivers/infiniband/Kconfig
View File

@ -107,6 +107,7 @@ source "drivers/infiniband/ulp/srpt/Kconfig"
source "drivers/infiniband/ulp/iser/Kconfig"
source "drivers/infiniband/ulp/isert/Kconfig"
source "drivers/infiniband/ulp/rtrs/Kconfig"
source "drivers/infiniband/ulp/opa_vnic/Kconfig"

9
drivers/infiniband/core/Makefile
View File

@ -8,11 +8,11 @@ obj-$(CONFIG_INFINIBAND_USER_MAD) += ib_umad.o
obj-$(CONFIG_INFINIBAND_USER_ACCESS) += ib_uverbs.o $(user_access-y)
ib_core-y := packer.o ud_header.o verbs.o cq.o rw.o sysfs.o \
device.o fmr_pool.o cache.o netlink.o \
device.o cache.o netlink.o \
roce_gid_mgmt.o mr_pool.o addr.o sa_query.o \
multicast.o mad.o smi.o agent.o mad_rmpp.o \
nldev.o restrack.o counters.o ib_core_uverbs.o \
trace.o
trace.o lag.o
ib_core-$(CONFIG_SECURITY_INFINIBAND) += security.o
ib_core-$(CONFIG_CGROUP_RDMA) += cgroup.o
@ -36,6 +36,9 @@ ib_uverbs-y := uverbs_main.o uverbs_cmd.o uverbs_marshall.o \
uverbs_std_types_flow_action.o uverbs_std_types_dm.o \
uverbs_std_types_mr.o uverbs_std_types_counters.o \
uverbs_uapi.o uverbs_std_types_device.o \
uverbs_std_types_async_fd.o
uverbs_std_types_async_fd.o \
uverbs_std_types_srq.o \
uverbs_std_types_wq.o \
uverbs_std_types_qp.o
ib_uverbs-$(CONFIG_INFINIBAND_USER_MEM) += umem.o
ib_uverbs-$(CONFIG_INFINIBAND_ON_DEMAND_PAGING) += umem_odp.o

4
drivers/infiniband/core/addr.c
View File

@ -371,6 +371,8 @@ static int fetch_ha(const struct dst_entry *dst, struct rdma_dev_addr *dev_addr,
(const void *)&dst_in6->sin6_addr;
sa_family_t family = dst_in->sa_family;
might_sleep();
/* If we have a gateway in IB mode then it must be an IB network */
if (has_gateway(dst, family) && dev_addr->network == RDMA_NETWORK_IB)
return ib_nl_fetch_ha(dev_addr, daddr, seq, family);
@ -727,6 +729,8 @@ int roce_resolve_route_from_path(struct sa_path_rec *rec,
struct rdma_dev_addr dev_addr = {};
int ret;
might_sleep();
if (rec->roce.route_resolved)
return 0;

306
drivers/infiniband/core/cm.c
View File

@ -66,6 +66,8 @@ static const char * const ibcm_rej_reason_strs[] = {
[IB_CM_REJ_INVALID_CLASS_VERSION] = "invalid class version",
[IB_CM_REJ_INVALID_FLOW_LABEL] = "invalid flow label",
[IB_CM_REJ_INVALID_ALT_FLOW_LABEL] = "invalid alt flow label",
[IB_CM_REJ_VENDOR_OPTION_NOT_SUPPORTED] =
"vendor option is not supported",
};
const char *__attribute_const__ ibcm_reject_msg(int reason)
@ -81,8 +83,11 @@ const char *__attribute_const__ ibcm_reject_msg(int reason)
EXPORT_SYMBOL(ibcm_reject_msg);
struct cm_id_private;
static void cm_add_one(struct ib_device *device);
struct cm_work;
static int cm_add_one(struct ib_device *device);
static void cm_remove_one(struct ib_device *device, void *client_data);
static void cm_process_work(struct cm_id_private *cm_id_priv,
struct cm_work *work);
static int cm_send_sidr_rep_locked(struct cm_id_private *cm_id_priv,
struct ib_cm_sidr_rep_param *param);
static int cm_send_dreq_locked(struct cm_id_private *cm_id_priv,
@ -287,6 +292,8 @@ struct cm_id_private {
struct list_head work_list;
atomic_t work_count;
struct rdma_ucm_ece ece;
};
static void cm_work_handler(struct work_struct *work);
@ -474,24 +481,19 @@ static int cm_init_av_for_response(struct cm_port *port, struct ib_wc *wc,
grh, &av->ah_attr);
}
static int add_cm_id_to_port_list(struct cm_id_private *cm_id_priv,
struct cm_av *av,
struct cm_port *port)
static void add_cm_id_to_port_list(struct cm_id_private *cm_id_priv,
struct cm_av *av, struct cm_port *port)
{
unsigned long flags;
int ret = 0;
spin_lock_irqsave(&cm.lock, flags);
if (&cm_id_priv->av == av)
list_add_tail(&cm_id_priv->prim_list, &port->cm_priv_prim_list);
else if (&cm_id_priv->alt_av == av)
list_add_tail(&cm_id_priv->altr_list, &port->cm_priv_altr_list);
else
ret = -EINVAL;
WARN_ON(true);
spin_unlock_irqrestore(&cm.lock, flags);
return ret;
}
static struct cm_port *
@ -572,12 +574,7 @@ static int cm_init_av_by_path(struct sa_path_rec *path,
return ret;
av->timeout = path->packet_life_time + 1;
ret = add_cm_id_to_port_list(cm_id_priv, av, port);
if (ret) {
rdma_destroy_ah_attr(&new_ah_attr);
return ret;
}
add_cm_id_to_port_list(cm_id_priv, av, port);
rdma_move_ah_attr(&av->ah_attr, &new_ah_attr);
return 0;
}
@ -587,11 +584,6 @@ static u32 cm_local_id(__be32 local_id)
return (__force u32) (local_id ^ cm.random_id_operand);
}
static void cm_free_id(__be32 local_id)
{
xa_erase_irq(&cm.local_id_table, cm_local_id(local_id));
}
static struct cm_id_private *cm_acquire_id(__be32 local_id, __be32 remote_id)
{
struct cm_id_private *cm_id_priv;
@ -698,9 +690,10 @@ static struct cm_id_private * cm_find_listen(struct ib_device *device,
cm_id_priv = rb_entry(node, struct cm_id_private, service_node);
if ((cm_id_priv->id.service_mask & service_id) ==
cm_id_priv->id.service_id &&
(cm_id_priv->id.device == device))
(cm_id_priv->id.device == device)) {
refcount_inc(&cm_id_priv->refcount);
return cm_id_priv;
}
if (device < cm_id_priv->id.device)
node = node->rb_left;
else if (device > cm_id_priv->id.device)
@ -745,12 +738,14 @@ static struct cm_timewait_info * cm_insert_remote_id(struct cm_timewait_info
return NULL;
}
static struct cm_timewait_info * cm_find_remote_id(__be64 remote_ca_guid,
__be32 remote_id)
static struct cm_id_private *cm_find_remote_id(__be64 remote_ca_guid,
__be32 remote_id)
{
struct rb_node *node = cm.remote_id_table.rb_node;
struct cm_timewait_info *timewait_info;
struct cm_id_private *res = NULL;
spin_lock_irq(&cm.lock);
while (node) {
timewait_info = rb_entry(node, struct cm_timewait_info,
remote_id_node);
@ -762,10 +757,14 @@ static struct cm_timewait_info * cm_find_remote_id(__be64 remote_ca_guid,
node = node->rb_left;
else if (be64_gt(remote_ca_guid, timewait_info->remote_ca_guid))
node = node->rb_right;
else
return timewait_info;
else {
res = cm_acquire_id(timewait_info->work.local_id,
timewait_info->work.remote_id);
break;
}
}
return NULL;
spin_unlock_irq(&cm.lock);
return res;
}
static struct cm_timewait_info * cm_insert_remote_qpn(struct cm_timewait_info
@ -917,6 +916,35 @@ static void cm_free_work(struct cm_work *work)
kfree(work);
}
static void cm_queue_work_unlock(struct cm_id_private *cm_id_priv,
struct cm_work *work)
{
bool immediate;
/*
* To deliver the event to the user callback we have the drop the
* spinlock, however, we need to ensure that the user callback is single
* threaded and receives events in the temporal order. If there are
* already events being processed then thread new events onto a list,
* the thread currently processing will pick them up.
*/
immediate = atomic_inc_and_test(&cm_id_priv->work_count);
if (!immediate) {
list_add_tail(&work->list, &cm_id_priv->work_list);
/*
* This routine always consumes incoming reference. Once queued
* to the work_list then a reference is held by the thread
* currently running cm_process_work() and this reference is not
* needed.
*/
cm_deref_id(cm_id_priv);
}
spin_unlock_irq(&cm_id_priv->lock);
if (immediate)
cm_process_work(cm_id_priv, work);
}
static inline int cm_convert_to_ms(int iba_time)
{
/* approximate conversion to ms from 4.096us x 2^iba_time */
@ -942,8 +970,10 @@ static u8 cm_ack_timeout(u8 ca_ack_delay, u8 packet_life_time)
return min(31, ack_timeout);
}
static void cm_cleanup_timewait(struct cm_timewait_info *timewait_info)
static void cm_remove_remote(struct cm_id_private *cm_id_priv)
{
struct cm_timewait_info *timewait_info = cm_id_priv->timewait_info;
if (timewait_info->inserted_remote_id) {
rb_erase(&timewait_info->remote_id_node, &cm.remote_id_table);
timewait_info->inserted_remote_id = 0;
@ -982,7 +1012,7 @@ static void cm_enter_timewait(struct cm_id_private *cm_id_priv)
return;
spin_lock_irqsave(&cm.lock, flags);
cm_cleanup_timewait(cm_id_priv->timewait_info);
cm_remove_remote(cm_id_priv);
list_add_tail(&cm_id_priv->timewait_info->list, &cm.timewait_list);
spin_unlock_irqrestore(&cm.lock, flags);
@ -1001,6 +1031,11 @@ static void cm_enter_timewait(struct cm_id_private *cm_id_priv)
msecs_to_jiffies(wait_time));
spin_unlock_irqrestore(&cm.lock, flags);
/*
* The timewait_info is converted into a work and gets freed during
* cm_free_work() in cm_timewait_handler().
*/
BUILD_BUG_ON(offsetof(struct cm_timewait_info, work) != 0);
cm_id_priv->timewait_info = NULL;
}
@ -1013,7 +1048,7 @@ static void cm_reset_to_idle(struct cm_id_private *cm_id_priv)
cm_id_priv->id.state = IB_CM_IDLE;
if (cm_id_priv->timewait_info) {
spin_lock_irqsave(&cm.lock, flags);
cm_cleanup_timewait(cm_id_priv->timewait_info);
cm_remove_remote(cm_id_priv);
spin_unlock_irqrestore(&cm.lock, flags);
kfree(cm_id_priv->timewait_info);
cm_id_priv->timewait_info = NULL;
@ -1076,7 +1111,9 @@ static void cm_destroy_id(struct ib_cm_id *cm_id, int err)
case IB_CM_REP_SENT:
case IB_CM_MRA_REP_RCVD:
ib_cancel_mad(cm_id_priv->av.port->mad_agent, cm_id_priv->msg);
/* Fall through */
cm_send_rej_locked(cm_id_priv, IB_CM_REJ_CONSUMER_DEFINED, NULL,
0, NULL, 0);
goto retest;
case IB_CM_MRA_REQ_SENT:
case IB_CM_REP_RCVD:
case IB_CM_MRA_REP_SENT:
@ -1101,7 +1138,7 @@ static void cm_destroy_id(struct ib_cm_id *cm_id, int err)
case IB_CM_TIMEWAIT:
/*
* The cm_acquire_id in cm_timewait_handler will stop working
* once we do cm_free_id() below, so just move to idle here for
* once we do xa_erase below, so just move to idle here for
* consistency.
*/
cm_id->state = IB_CM_IDLE;
@ -1114,7 +1151,7 @@ static void cm_destroy_id(struct ib_cm_id *cm_id, int err)
spin_lock(&cm.lock);
/* Required for cleanup paths related cm_req_handler() */
if (cm_id_priv->timewait_info) {
cm_cleanup_timewait(cm_id_priv->timewait_info);
cm_remove_remote(cm_id_priv);
kfree(cm_id_priv->timewait_info);
cm_id_priv->timewait_info = NULL;
}
@ -1131,7 +1168,7 @@ static void cm_destroy_id(struct ib_cm_id *cm_id, int err)
spin_unlock(&cm.lock);
spin_unlock_irq(&cm_id_priv->lock);
cm_free_id(cm_id->local_id);
xa_erase_irq(&cm.local_id_table, cm_local_id(cm_id->local_id));
cm_deref_id(cm_id_priv);
wait_for_completion(&cm_id_priv->comp);
while ((work = cm_dequeue_work(cm_id_priv)) != NULL)
@ -1287,6 +1324,13 @@ static void cm_format_mad_hdr(struct ib_mad_hdr *hdr,
hdr->tid = tid;
}
static void cm_format_mad_ece_hdr(struct ib_mad_hdr *hdr, __be16 attr_id,
__be64 tid, u32 attr_mod)
{
cm_format_mad_hdr(hdr, attr_id, tid);
hdr->attr_mod = cpu_to_be32(attr_mod);
}
static void cm_format_req(struct cm_req_msg *req_msg,
struct cm_id_private *cm_id_priv,
struct ib_cm_req_param *param)
@ -1299,8 +1343,8 @@ static void cm_format_req(struct cm_req_msg *req_msg,
pri_ext = opa_is_extended_lid(pri_path->opa.dlid,
pri_path->opa.slid);
cm_format_mad_hdr(&req_msg->hdr, CM_REQ_ATTR_ID,
cm_form_tid(cm_id_priv));
cm_format_mad_ece_hdr(&req_msg->hdr, CM_REQ_ATTR_ID,
cm_form_tid(cm_id_priv), param->ece.attr_mod);
IBA_SET(CM_REQ_LOCAL_COMM_ID, req_msg,
be32_to_cpu(cm_id_priv->id.local_id));
@ -1423,6 +1467,7 @@ static void cm_format_req(struct cm_req_msg *req_msg,
cm_ack_timeout(cm_id_priv->av.port->cm_dev->ack_delay,
alt_path->packet_life_time));
}
IBA_SET(CM_REQ_VENDOR_ID, req_msg, param->ece.vendor_id);
if (param->private_data && param->private_data_len)
IBA_SET_MEM(CM_REQ_PRIVATE_DATA, req_msg, param->private_data,
@ -1779,6 +1824,9 @@ static void cm_format_req_event(struct cm_work *work,
param->rnr_retry_count = IBA_GET(CM_REQ_RNR_RETRY_COUNT, req_msg);
param->srq = IBA_GET(CM_REQ_SRQ, req_msg);
param->ppath_sgid_attr = cm_id_priv->av.ah_attr.grh.sgid_attr;
param->ece.vendor_id = IBA_GET(CM_REQ_VENDOR_ID, req_msg);
param->ece.attr_mod = be32_to_cpu(req_msg->hdr.attr_mod);
work->cm_event.private_data =
IBA_GET_MEM_PTR(CM_REQ_PRIVATE_DATA, req_msg);
}
@ -1927,7 +1975,6 @@ static struct cm_id_private * cm_match_req(struct cm_work *work,
struct cm_id_private *listen_cm_id_priv, *cur_cm_id_priv;
struct cm_timewait_info *timewait_info;
struct cm_req_msg *req_msg;
struct ib_cm_id *cm_id;
req_msg = (struct cm_req_msg *)work->mad_recv_wc->recv_buf.mad;
@ -1948,7 +1995,7 @@ static struct cm_id_private * cm_match_req(struct cm_work *work,
/* Check for stale connections. */
timewait_info = cm_insert_remote_qpn(cm_id_priv->timewait_info);
if (timewait_info) {
cm_cleanup_timewait(cm_id_priv->timewait_info);
cm_remove_remote(cm_id_priv);
cur_cm_id_priv = cm_acquire_id(timewait_info->work.local_id,
timewait_info->work.remote_id);
@ -1957,8 +2004,7 @@ static struct cm_id_private * cm_match_req(struct cm_work *work,
IB_CM_REJ_STALE_CONN, CM_MSG_RESPONSE_REQ,
NULL, 0);
if (cur_cm_id_priv) {
cm_id = &cur_cm_id_priv->id;
ib_send_cm_dreq(cm_id, NULL, 0);
ib_send_cm_dreq(&cur_cm_id_priv->id, NULL, 0);
cm_deref_id(cur_cm_id_priv);
}
return NULL;
@ -1969,14 +2015,13 @@ static struct cm_id_private * cm_match_req(struct cm_work *work,
cm_id_priv->id.device,
cpu_to_be64(IBA_GET(CM_REQ_SERVICE_ID, req_msg)));
if (!listen_cm_id_priv) {
cm_cleanup_timewait(cm_id_priv->timewait_info);
cm_remove_remote(cm_id_priv);
spin_unlock_irq(&cm.lock);
cm_issue_rej(work->port, work->mad_recv_wc,
IB_CM_REJ_INVALID_SERVICE_ID, CM_MSG_RESPONSE_REQ,
NULL, 0);
return NULL;
}
refcount_inc(&listen_cm_id_priv->refcount);
spin_unlock_irq(&cm.lock);
return listen_cm_id_priv;
}
@ -2153,9 +2198,7 @@ static int cm_req_handler(struct cm_work *work)
/* Refcount belongs to the event, pairs with cm_process_work() */
refcount_inc(&cm_id_priv->refcount);
atomic_inc(&cm_id_priv->work_count);
spin_unlock_irq(&cm_id_priv->lock);
cm_process_work(cm_id_priv, work);
cm_queue_work_unlock(cm_id_priv, work);
/*
* Since this ID was just created and was not made visible to other MAD
* handlers until the cm_finalize_id() above we know that the
@ -2176,7 +2219,8 @@ static void cm_format_rep(struct cm_rep_msg *rep_msg,
struct cm_id_private *cm_id_priv,
struct ib_cm_rep_param *param)
{
cm_format_mad_hdr(&rep_msg->hdr, CM_REP_ATTR_ID, cm_id_priv->tid);
cm_format_mad_ece_hdr(&rep_msg->hdr, CM_REP_ATTR_ID, cm_id_priv->tid,
param->ece.attr_mod);
IBA_SET(CM_REP_LOCAL_COMM_ID, rep_msg,
be32_to_cpu(cm_id_priv->id.local_id));
IBA_SET(CM_REP_REMOTE_COMM_ID, rep_msg,
@ -2203,6 +2247,10 @@ static void cm_format_rep(struct cm_rep_msg *rep_msg,
IBA_SET(CM_REP_LOCAL_EE_CONTEXT_NUMBER, rep_msg, param->qp_num);
}
IBA_SET(CM_REP_VENDOR_ID_L, rep_msg, param->ece.vendor_id);
IBA_SET(CM_REP_VENDOR_ID_M, rep_msg, param->ece.vendor_id >> 8);
IBA_SET(CM_REP_VENDOR_ID_H, rep_msg, param->ece.vendor_id >> 16);
if (param->private_data && param->private_data_len)
IBA_SET_MEM(CM_REP_PRIVATE_DATA, rep_msg, param->private_data,
param->private_data_len);
@ -2350,6 +2398,11 @@ static void cm_format_rep_event(struct cm_work *work, enum ib_qp_type qp_type)
param->flow_control = IBA_GET(CM_REP_END_TO_END_FLOW_CONTROL, rep_msg);
param->rnr_retry_count = IBA_GET(CM_REP_RNR_RETRY_COUNT, rep_msg);
param->srq = IBA_GET(CM_REP_SRQ, rep_msg);
param->ece.vendor_id = IBA_GET(CM_REP_VENDOR_ID_H, rep_msg) << 16;
param->ece.vendor_id |= IBA_GET(CM_REP_VENDOR_ID_M, rep_msg) << 8;
param->ece.vendor_id |= IBA_GET(CM_REP_VENDOR_ID_L, rep_msg);
param->ece.attr_mod = be32_to_cpu(rep_msg->hdr.attr_mod);
work->cm_event.private_data =
IBA_GET_MEM_PTR(CM_REP_PRIVATE_DATA, rep_msg);
}
@ -2404,7 +2457,6 @@ static int cm_rep_handler(struct cm_work *work)
struct cm_rep_msg *rep_msg;
int ret;
struct cm_id_private *cur_cm_id_priv;
struct ib_cm_id *cm_id;
struct cm_timewait_info *timewait_info;
rep_msg = (struct cm_rep_msg *)work->mad_recv_wc->recv_buf.mad;
@ -2454,9 +2506,7 @@ static int cm_rep_handler(struct cm_work *work)
/* Check for a stale connection. */
timewait_info = cm_insert_remote_qpn(cm_id_priv->timewait_info);
if (timewait_info) {
rb_erase(&cm_id_priv->timewait_info->remote_id_node,
&cm.remote_id_table);
cm_id_priv->timewait_info->inserted_remote_id = 0;
cm_remove_remote(cm_id_priv);
cur_cm_id_priv = cm_acquire_id(timewait_info->work.local_id,
timewait_info->work.remote_id);
@ -2472,8 +2522,7 @@ static int cm_rep_handler(struct cm_work *work)
IBA_GET(CM_REP_REMOTE_COMM_ID, rep_msg));
if (cur_cm_id_priv) {
cm_id = &cur_cm_id_priv->id;
ib_send_cm_dreq(cm_id, NULL, 0);
ib_send_cm_dreq(&cur_cm_id_priv->id, NULL, 0);
cm_deref_id(cur_cm_id_priv);
}
@ -2501,15 +2550,7 @@ static int cm_rep_handler(struct cm_work *work)
cm_id_priv->alt_av.timeout - 1);
ib_cancel_mad(cm_id_priv->av.port->mad_agent, cm_id_priv->msg);
ret = atomic_inc_and_test(&cm_id_priv->work_count);
if (!ret)
list_add_tail(&work->list, &cm_id_priv->work_list);
spin_unlock_irq(&cm_id_priv->lock);
if (ret)
cm_process_work(cm_id_priv, work);
else
cm_deref_id(cm_id_priv);
cm_queue_work_unlock(cm_id_priv, work);
return 0;
error:
@ -2520,7 +2561,6 @@ static int cm_rep_handler(struct cm_work *work)
static int cm_establish_handler(struct cm_work *work)
{
struct cm_id_private *cm_id_priv;
int ret;
/* See comment in cm_establish about lookup. */
cm_id_priv = cm_acquire_id(work->local_id, work->remote_id);
@ -2534,15 +2574,7 @@ static int cm_establish_handler(struct cm_work *work)
}
ib_cancel_mad(cm_id_priv->av.port->mad_agent, cm_id_priv->msg);
ret = atomic_inc_and_test(&cm_id_priv->work_count);
if (!ret)
list_add_tail(&work->list, &cm_id_priv->work_list);
spin_unlock_irq(&cm_id_priv->lock);
if (ret)
cm_process_work(cm_id_priv, work);
else
cm_deref_id(cm_id_priv);
cm_queue_work_unlock(cm_id_priv, work);
return 0;
out:
cm_deref_id(cm_id_priv);
@ -2553,7 +2585,6 @@ static int cm_rtu_handler(struct cm_work *work)
{
struct cm_id_private *cm_id_priv;
struct cm_rtu_msg *rtu_msg;
int ret;
rtu_msg = (struct cm_rtu_msg *)work->mad_recv_wc->recv_buf.mad;
cm_id_priv = cm_acquire_id(
@ -2576,15 +2607,7 @@ static int cm_rtu_handler(struct cm_work *work)
cm_id_priv->id.state = IB_CM_ESTABLISHED;
ib_cancel_mad(cm_id_priv->av.port->mad_agent, cm_id_priv->msg);
ret = atomic_inc_and_test(&cm_id_priv->work_count);
if (!ret)
list_add_tail(&work->list, &cm_id_priv->work_list);
spin_unlock_irq(&cm_id_priv->lock);
if (ret)
cm_process_work(cm_id_priv, work);
else
cm_deref_id(cm_id_priv);
cm_queue_work_unlock(cm_id_priv, work);
return 0;
out:
cm_deref_id(cm_id_priv);
@ -2777,7 +2800,6 @@ static int cm_dreq_handler(struct cm_work *work)
struct cm_id_private *cm_id_priv;
struct cm_dreq_msg *dreq_msg;
struct ib_mad_send_buf *msg = NULL;
int ret;
dreq_msg = (struct cm_dreq_msg *)work->mad_recv_wc->recv_buf.mad;
cm_id_priv = cm_acquire_id(
@ -2842,15 +2864,7 @@ static int cm_dreq_handler(struct cm_work *work)
}
cm_id_priv->id.state = IB_CM_DREQ_RCVD;
cm_id_priv->tid = dreq_msg->hdr.tid;
ret = atomic_inc_and_test(&cm_id_priv->work_count);
if (!ret)
list_add_tail(&work->list, &cm_id_priv->work_list);
spin_unlock_irq(&cm_id_priv->lock);
if (ret)
cm_process_work(cm_id_priv, work);
else
cm_deref_id(cm_id_priv);
cm_queue_work_unlock(cm_id_priv, work);
return 0;
unlock: spin_unlock_irq(&cm_id_priv->lock);
@ -2862,7 +2876,6 @@ static int cm_drep_handler(struct cm_work *work)
{
struct cm_id_private *cm_id_priv;
struct cm_drep_msg *drep_msg;
int ret;
drep_msg = (struct cm_drep_msg *)work->mad_recv_wc->recv_buf.mad;
cm_id_priv = cm_acquire_id(
@ -2883,15 +2896,7 @@ static int cm_drep_handler(struct cm_work *work)
cm_enter_timewait(cm_id_priv);
ib_cancel_mad(cm_id_priv->av.port->mad_agent, cm_id_priv->msg);
ret = atomic_inc_and_test(&cm_id_priv->work_count);
if (!ret)
list_add_tail(&work->list, &cm_id_priv->work_list);
spin_unlock_irq(&cm_id_priv->lock);
if (ret)
cm_process_work(cm_id_priv, work);
else
cm_deref_id(cm_id_priv);
cm_queue_work_unlock(cm_id_priv, work);
return 0;
out:
cm_deref_id(cm_id_priv);
@ -2987,24 +2992,15 @@ static void cm_format_rej_event(struct cm_work *work)
static struct cm_id_private * cm_acquire_rejected_id(struct cm_rej_msg *rej_msg)
{
struct cm_timewait_info *timewait_info;
struct cm_id_private *cm_id_priv;
__be32 remote_id;
remote_id = cpu_to_be32(IBA_GET(CM_REJ_LOCAL_COMM_ID, rej_msg));
if (IBA_GET(CM_REJ_REASON, rej_msg) == IB_CM_REJ_TIMEOUT) {
spin_lock_irq(&cm.lock);
timewait_info = cm_find_remote_id(
cm_id_priv = cm_find_remote_id(
*((__be64 *)IBA_GET_MEM_PTR(CM_REJ_ARI, rej_msg)),
remote_id);
if (!timewait_info) {
spin_unlock_irq(&cm.lock);
return NULL;
}
cm_id_priv =
cm_acquire_id(timewait_info->work.local_id, remote_id);
spin_unlock_irq(&cm.lock);
} else if (IBA_GET(CM_REJ_MESSAGE_REJECTED, rej_msg) ==
CM_MSG_RESPONSE_REQ)
cm_id_priv = cm_acquire_id(
@ -3022,7 +3018,6 @@ static int cm_rej_handler(struct cm_work *work)
{
struct cm_id_private *cm_id_priv;
struct cm_rej_msg *rej_msg;
int ret;
rej_msg = (struct cm_rej_msg *)work->mad_recv_wc->recv_buf.mad;
cm_id_priv = cm_acquire_rejected_id(rej_msg);
@ -3068,19 +3063,10 @@ static int cm_rej_handler(struct cm_work *work)
__func__, be32_to_cpu(cm_id_priv->id.local_id),
cm_id_priv->id.state);
spin_unlock_irq(&cm_id_priv->lock);
ret = -EINVAL;
goto out;
}
ret = atomic_inc_and_test(&cm_id_priv->work_count);
if (!ret)
list_add_tail(&work->list, &cm_id_priv->work_list);
spin_unlock_irq(&cm_id_priv->lock);
if (ret)
cm_process_work(cm_id_priv, work);
else
cm_deref_id(cm_id_priv);
cm_queue_work_unlock(cm_id_priv, work);
return 0;
out:
cm_deref_id(cm_id_priv);
@ -3190,7 +3176,7 @@ static int cm_mra_handler(struct cm_work *work)
{
struct cm_id_private *cm_id_priv;
struct cm_mra_msg *mra_msg;
int timeout, ret;
int timeout;
mra_msg = (struct cm_mra_msg *)work->mad_recv_wc->recv_buf.mad;
cm_id_priv = cm_acquire_mraed_id(mra_msg);
@ -3250,15 +3236,7 @@ static int cm_mra_handler(struct cm_work *work)
cm_id_priv->msg->context[1] = (void *) (unsigned long)
cm_id_priv->id.state;
ret = atomic_inc_and_test(&cm_id_priv->work_count);
if (!ret)
list_add_tail(&work->list, &cm_id_priv->work_list);
spin_unlock_irq(&cm_id_priv->lock);
if (ret)
cm_process_work(cm_id_priv, work);
else
cm_deref_id(cm_id_priv);
cm_queue_work_unlock(cm_id_priv, work);
return 0;
out:
spin_unlock_irq(&cm_id_priv->lock);
@ -3393,15 +3371,7 @@ static int cm_lap_handler(struct cm_work *work)
cm_id_priv->id.lap_state = IB_CM_LAP_RCVD;
cm_id_priv->tid = lap_msg->hdr.tid;
ret = atomic_inc_and_test(&cm_id_priv->work_count);
if (!ret)
list_add_tail(&work->list, &cm_id_priv->work_list);
spin_unlock_irq(&cm_id_priv->lock);
if (ret)
cm_process_work(cm_id_priv, work);
else
cm_deref_id(cm_id_priv);
cm_queue_work_unlock(cm_id_priv, work);
return 0;
unlock: spin_unlock_irq(&cm_id_priv->lock);
@ -3413,7 +3383,6 @@ static int cm_apr_handler(struct cm_work *work)
{
struct cm_id_private *cm_id_priv;
struct cm_apr_msg *apr_msg;
int ret;
/* Currently Alternate path messages are not supported for
* RoCE link layer.
@ -3448,16 +3417,7 @@ static int cm_apr_handler(struct cm_work *work)
cm_id_priv->id.lap_state = IB_CM_LAP_IDLE;
ib_cancel_mad(cm_id_priv->av.port->mad_agent, cm_id_priv->msg);
cm_id_priv->msg = NULL;
ret = atomic_inc_and_test(&cm_id_priv->work_count);
if (!ret)
list_add_tail(&work->list, &cm_id_priv->work_list);
spin_unlock_irq(&cm_id_priv->lock);
if (ret)
cm_process_work(cm_id_priv, work);
else
cm_deref_id(cm_id_priv);
cm_queue_work_unlock(cm_id_priv, work);
return 0;
out:
cm_deref_id(cm_id_priv);
@ -3468,7 +3428,6 @@ static int cm_timewait_handler(struct cm_work *work)
{
struct cm_timewait_info *timewait_info;
struct cm_id_private *cm_id_priv;
int ret;
timewait_info = container_of(work, struct cm_timewait_info, work);
spin_lock_irq(&cm.lock);
@ -3487,15 +3446,7 @@ static int cm_timewait_handler(struct cm_work *work)
goto out;
}
cm_id_priv->id.state = IB_CM_IDLE;
ret = atomic_inc_and_test(&cm_id_priv->work_count);
if (!ret)
list_add_tail(&work->list, &cm_id_priv->work_list);
spin_unlock_irq(&cm_id_priv->lock);
if (ret)
cm_process_work(cm_id_priv, work);
else
cm_deref_id(cm_id_priv);
cm_queue_work_unlock(cm_id_priv, work);
return 0;
out:
cm_deref_id(cm_id_priv);
@ -3642,7 +3593,6 @@ static int cm_sidr_req_handler(struct cm_work *work)
.status = IB_SIDR_UNSUPPORTED });
goto out; /* No match. */
}
refcount_inc(&listen_cm_id_priv->refcount);
spin_unlock_irq(&cm.lock);
cm_id_priv->id.cm_handler = listen_cm_id_priv->id.cm_handler;
@ -3674,8 +3624,8 @@ static void cm_format_sidr_rep(struct cm_sidr_rep_msg *sidr_rep_msg,
struct cm_id_private *cm_id_priv,
struct ib_cm_sidr_rep_param *param)
{
cm_format_mad_hdr(&sidr_rep_msg->hdr, CM_SIDR_REP_ATTR_ID,
cm_id_priv->tid);
cm_format_mad_ece_hdr(&sidr_rep_msg->hdr, CM_SIDR_REP_ATTR_ID,
cm_id_priv->tid, param->ece.attr_mod);
IBA_SET(CM_SIDR_REP_REQUESTID, sidr_rep_msg,
be32_to_cpu(cm_id_priv->id.remote_id));
IBA_SET(CM_SIDR_REP_STATUS, sidr_rep_msg, param->status);
@ -3683,6 +3633,10 @@ static void cm_format_sidr_rep(struct cm_sidr_rep_msg *sidr_rep_msg,
IBA_SET(CM_SIDR_REP_SERVICEID, sidr_rep_msg,
be64_to_cpu(cm_id_priv->id.service_id));
IBA_SET(CM_SIDR_REP_Q_KEY, sidr_rep_msg, param->qkey);
IBA_SET(CM_SIDR_REP_VENDOR_ID_L, sidr_rep_msg,
param->ece.vendor_id & 0xFF);
IBA_SET(CM_SIDR_REP_VENDOR_ID_H, sidr_rep_msg,
(param->ece.vendor_id >> 8) & 0xFF);
if (param->info && param->info_length)
IBA_SET_MEM(CM_SIDR_REP_ADDITIONAL_INFORMATION, sidr_rep_msg,
@ -4384,7 +4338,7 @@ static void cm_remove_port_fs(struct cm_port *port)
}
static void cm_add_one(struct ib_device *ib_device)
static int cm_add_one(struct ib_device *ib_device)
{
struct cm_device *cm_dev;
struct cm_port *port;
@ -4403,7 +4357,7 @@ static void cm_add_one(struct ib_device *ib_device)
cm_dev = kzalloc(struct_size(cm_dev, port, ib_device->phys_port_cnt),
GFP_KERNEL);
if (!cm_dev)
return;
return -ENOMEM;
cm_dev->ib_device = ib_device;
cm_dev->ack_delay = ib_device->attrs.local_ca_ack_delay;
@ -4415,8 +4369,10 @@ static void cm_add_one(struct ib_device *ib_device)
continue;
port = kzalloc(sizeof *port, GFP_KERNEL);
if (!port)
if (!port) {
ret = -ENOMEM;
goto error1;
}
cm_dev->port[i-1] = port;
port->cm_dev = cm_dev;
@ -4437,8 +4393,10 @@ static void cm_add_one(struct ib_device *ib_device)
cm_recv_handler,
port,
0);
if (IS_ERR(port->mad_agent))
if (IS_ERR(port->mad_agent)) {
ret = PTR_ERR(port->mad_agent);
goto error2;
}
ret = ib_modify_port(ib_device, i, 0, &port_modify);
if (ret)
@ -4447,15 +4405,17 @@ static void cm_add_one(struct ib_device *ib_device)
count++;
}
if (!count)
if (!count) {
ret = -EOPNOTSUPP;
goto free;
}
ib_set_client_data(ib_device, &cm_client, cm_dev);
write_lock_irqsave(&cm.device_lock, flags);
list_add_tail(&cm_dev->list, &cm.device_list);
write_unlock_irqrestore(&cm.device_lock, flags);
return;
return 0;
error3:
ib_unregister_mad_agent(port->mad_agent);
@ -4477,6 +4437,7 @@ static void cm_add_one(struct ib_device *ib_device)
}
free:
kfree(cm_dev);
return ret;
}
static void cm_remove_one(struct ib_device *ib_device, void *client_data)
@ -4491,9 +4452,6 @@ static void cm_remove_one(struct ib_device *ib_device, void *client_data)
unsigned long flags;
int i;
if (!cm_dev)
return;
write_lock_irqsave(&cm.device_lock, flags);
list_del(&cm_dev->list);
write_unlock_irqrestore(&cm.device_lock, flags);

114
drivers/infiniband/core/cma.c
View File

@ -91,7 +91,13 @@ const char *__attribute_const__ rdma_reject_msg(struct rdma_cm_id *id,
}
EXPORT_SYMBOL(rdma_reject_msg);
bool rdma_is_consumer_reject(struct rdma_cm_id *id, int reason)
/**
* rdma_is_consumer_reject - return true if the consumer rejected the connect
* request.
* @id: Communication identifier that received the REJECT event.
* @reason: Value returned in the REJECT event status field.
*/
static bool rdma_is_consumer_reject(struct rdma_cm_id *id, int reason)
{
if (rdma_ib_or_roce(id->device, id->port_num))
return reason == IB_CM_REJ_CONSUMER_DEFINED;
@ -102,7 +108,6 @@ bool rdma_is_consumer_reject(struct rdma_cm_id *id, int reason)
WARN_ON_ONCE(1);
return false;
}
EXPORT_SYMBOL(rdma_is_consumer_reject);
const void *rdma_consumer_reject_data(struct rdma_cm_id *id,
struct rdma_cm_event *ev, u8 *data_len)
@ -148,7 +153,7 @@ struct rdma_cm_id *rdma_res_to_id(struct rdma_restrack_entry *res)
}
EXPORT_SYMBOL(rdma_res_to_id);
static void cma_add_one(struct ib_device *device);
static int cma_add_one(struct ib_device *device);
static void cma_remove_one(struct ib_device *device, void *client_data);
static struct ib_client cma_client = {
@ -479,6 +484,7 @@ static void _cma_attach_to_dev(struct rdma_id_private *id_priv,
rdma_restrack_kadd(&id_priv->res);
else
rdma_restrack_uadd(&id_priv->res);
trace_cm_id_attach(id_priv, cma_dev->device);
}
static void cma_attach_to_dev(struct rdma_id_private *id_priv,
@ -883,7 +889,6 @@ struct rdma_cm_id *__rdma_create_id(struct net *net,
id_priv->id.route.addr.dev_addr.net = get_net(net);
id_priv->seq_num &= 0x00ffffff;
trace_cm_id_create(id_priv);
return &id_priv->id;
}
EXPORT_SYMBOL(__rdma_create_id);
@ -1906,6 +1911,9 @@ static void cma_set_rep_event_data(struct rdma_cm_event *event,
event->param.conn.rnr_retry_count = rep_data->rnr_retry_count;
event->param.conn.srq = rep_data->srq;
event->param.conn.qp_num = rep_data->remote_qpn;
event->ece.vendor_id = rep_data->ece.vendor_id;
event->ece.attr_mod = rep_data->ece.attr_mod;
}
static int cma_cm_event_handler(struct rdma_id_private *id_priv,
@ -2124,6 +2132,9 @@ static void cma_set_req_event_data(struct rdma_cm_event *event,
event->param.conn.rnr_retry_count = req_data->rnr_retry_count;
event->param.conn.srq = req_data->srq;
event->param.conn.qp_num = req_data->remote_qpn;
event->ece.vendor_id = req_data->ece.vendor_id;
event->ece.attr_mod = req_data->ece.attr_mod;
}
static int cma_ib_check_req_qp_type(const struct rdma_cm_id *id,
@ -2904,6 +2915,24 @@ static int iboe_tos_to_sl(struct net_device *ndev, int tos)
return 0;
}
static __be32 cma_get_roce_udp_flow_label(struct rdma_id_private *id_priv)
{
struct sockaddr_in6 *addr6;
u16 dport, sport;
u32 hash, fl;
addr6 = (struct sockaddr_in6 *)cma_src_addr(id_priv);
fl = be32_to_cpu(addr6->sin6_flowinfo) & IB_GRH_FLOWLABEL_MASK;
if ((cma_family(id_priv) != AF_INET6) || !fl) {
dport = be16_to_cpu(cma_port(cma_dst_addr(id_priv)));
sport = be16_to_cpu(cma_port(cma_src_addr(id_priv)));
hash = (u32)sport * 31 + dport;
fl = hash & IB_GRH_FLOWLABEL_MASK;
}
return cpu_to_be32(fl);
}
static int cma_resolve_iboe_route(struct rdma_id_private *id_priv)
{
struct rdma_route *route = &id_priv->id.route;
@ -2970,6 +2999,11 @@ static int cma_resolve_iboe_route(struct rdma_id_private *id_priv)
goto err2;
}
if (rdma_protocol_roce_udp_encap(id_priv->id.device,
id_priv->id.port_num))
route->path_rec->flow_label =
cma_get_roce_udp_flow_label(id_priv);
cma_init_resolve_route_work(work, id_priv);
queue_work(cma_wq, &work->work);
@ -3919,6 +3953,8 @@ static int cma_connect_ib(struct rdma_id_private *id_priv,
req.local_cm_response_timeout = CMA_CM_RESPONSE_TIMEOUT;
req.max_cm_retries = CMA_MAX_CM_RETRIES;
req.srq = id_priv->srq ? 1 : 0;
req.ece.vendor_id = id_priv->ece.vendor_id;
req.ece.attr_mod = id_priv->ece.attr_mod;
trace_cm_send_req(id_priv);
ret = ib_send_cm_req(id_priv->cm_id.ib, &req);
@ -4008,6 +4044,27 @@ int rdma_connect(struct rdma_cm_id *id, struct rdma_conn_param *conn_param)
}
EXPORT_SYMBOL(rdma_connect);
/**
* rdma_connect_ece - Initiate an active connection request with ECE data.
* @id: Connection identifier to connect.
* @conn_param: Connection information used for connected QPs.
* @ece: ECE parameters
*
* See rdma_connect() explanation.
*/
int rdma_connect_ece(struct rdma_cm_id *id, struct rdma_conn_param *conn_param,
struct rdma_ucm_ece *ece)
{
struct rdma_id_private *id_priv =
container_of(id, struct rdma_id_private, id);
id_priv->ece.vendor_id = ece->vendor_id;
id_priv->ece.attr_mod = ece->attr_mod;
return rdma_connect(id, conn_param);
}
EXPORT_SYMBOL(rdma_connect_ece);
static int cma_accept_ib(struct rdma_id_private *id_priv,
struct rdma_conn_param *conn_param)
{
@ -4033,6 +4090,8 @@ static int cma_accept_ib(struct rdma_id_private *id_priv,
rep.flow_control = conn_param->flow_control;
rep.rnr_retry_count = min_t(u8, 7, conn_param->rnr_retry_count);
rep.srq = id_priv->srq ? 1 : 0;
rep.ece.vendor_id = id_priv->ece.vendor_id;
rep.ece.attr_mod = id_priv->ece.attr_mod;
trace_cm_send_rep(id_priv);
ret = ib_send_cm_rep(id_priv->cm_id.ib, &rep);
@ -4080,7 +4139,11 @@ static int cma_send_sidr_rep(struct rdma_id_private *id_priv,
return ret;
rep.qp_num = id_priv->qp_num;
rep.qkey = id_priv->qkey;
rep.ece.vendor_id = id_priv->ece.vendor_id;
rep.ece.attr_mod = id_priv->ece.attr_mod;
}
rep.private_data = private_data;
rep.private_data_len = private_data_len;
@ -4133,11 +4196,24 @@ int __rdma_accept(struct rdma_cm_id *id, struct rdma_conn_param *conn_param,
return 0;
reject:
cma_modify_qp_err(id_priv);
rdma_reject(id, NULL, 0);
rdma_reject(id, NULL, 0, IB_CM_REJ_CONSUMER_DEFINED);
return ret;
}
EXPORT_SYMBOL(__rdma_accept);
int __rdma_accept_ece(struct rdma_cm_id *id, struct rdma_conn_param *conn_param,
const char *caller, struct rdma_ucm_ece *ece)
{
struct rdma_id_private *id_priv =
container_of(id, struct rdma_id_private, id);
id_priv->ece.vendor_id = ece->vendor_id;
id_priv->ece.attr_mod = ece->attr_mod;
return __rdma_accept(id, conn_param, caller);
}
EXPORT_SYMBOL(__rdma_accept_ece);
int rdma_notify(struct rdma_cm_id *id, enum ib_event_type event)
{
struct rdma_id_private *id_priv;
@ -4160,7 +4236,7 @@ int rdma_notify(struct rdma_cm_id *id, enum ib_event_type event)
EXPORT_SYMBOL(rdma_notify);
int rdma_reject(struct rdma_cm_id *id, const void *private_data,
u8 private_data_len)
u8 private_data_len, u8 reason)
{
struct rdma_id_private *id_priv;
int ret;
@ -4175,9 +4251,8 @@ int rdma_reject(struct rdma_cm_id *id, const void *private_data,
private_data, private_data_len);
} else {
trace_cm_send_rej(id_priv);
ret = ib_send_cm_rej(id_priv->cm_id.ib,
IB_CM_REJ_CONSUMER_DEFINED, NULL,
0, private_data, private_data_len);
ret = ib_send_cm_rej(id_priv->cm_id.ib, reason, NULL, 0,
private_data, private_data_len);
}
} else if (rdma_cap_iw_cm(id->device, id->port_num)) {
ret = iw_cm_reject(id_priv->cm_id.iw,
@ -4633,29 +4708,34 @@ static struct notifier_block cma_nb = {
.notifier_call = cma_netdev_callback
};
static void cma_add_one(struct ib_device *device)
static int cma_add_one(struct ib_device *device)
{
struct cma_device *cma_dev;
struct rdma_id_private *id_priv;
unsigned int i;
unsigned long supported_gids = 0;
int ret;
cma_dev = kmalloc(sizeof *cma_dev, GFP_KERNEL);
if (!cma_dev)
return;
return -ENOMEM;
cma_dev->device = device;
cma_dev->default_gid_type = kcalloc(device->phys_port_cnt,
sizeof(*cma_dev->default_gid_type),
GFP_KERNEL);
if (!cma_dev->default_gid_type)
if (!cma_dev->default_gid_type) {
ret = -ENOMEM;
goto free_cma_dev;
}
cma_dev->default_roce_tos = kcalloc(device->phys_port_cnt,
sizeof(*cma_dev->default_roce_tos),
GFP_KERNEL);
if (!cma_dev->default_roce_tos)
if (!cma_dev->default_roce_tos) {
ret = -ENOMEM;
goto free_gid_type;
}
rdma_for_each_port (device, i) {
supported_gids = roce_gid_type_mask_support(device, i);
@ -4681,15 +4761,14 @@ static void cma_add_one(struct ib_device *device)
mutex_unlock(&lock);
trace_cm_add_one(device);
return;
return 0;
free_gid_type:
kfree(cma_dev->default_gid_type);
free_cma_dev:
kfree(cma_dev);
return;
return ret;
}
static int cma_remove_id_dev(struct rdma_id_private *id_priv)
@ -4751,9 +4830,6 @@ static void cma_remove_one(struct ib_device *device, void *client_data)
trace_cm_remove_one(device);
if (!cma_dev)
return;
mutex_lock(&lock);
list_del(&cma_dev->list);
mutex_unlock(&lock);

13
drivers/infiniband/core/cma_configfs.c
View File

@ -322,8 +322,21 @@ static struct config_group *make_cma_dev(struct config_group *group,
return ERR_PTR(err);
}
static void drop_cma_dev(struct config_group *cgroup, struct config_item *item)
{
struct config_group *group =
container_of(item, struct config_group, cg_item);
struct cma_dev_group *cma_dev_group =
container_of(group, struct cma_dev_group, device_group);
configfs_remove_default_groups(&cma_dev_group->ports_group);
configfs_remove_default_groups(&cma_dev_group->device_group);
config_item_put(item);
}
static struct configfs_group_operations cma_subsys_group_ops = {
.make_group = make_cma_dev,
.drop_item = drop_cma_dev,
};
static const struct config_item_type cma_subsys_type = {

1
drivers/infiniband/core/cma_priv.h
View File

@ -95,6 +95,7 @@ struct rdma_id_private {
* Internal to RDMA/core, don't use in the drivers
*/
struct rdma_restrack_entry res;
struct rdma_ucm_ece ece;
};
#if IS_ENABLED(CONFIG_INFINIBAND_ADDR_TRANS_CONFIGFS)

20
drivers/infiniband/core/cma_trace.h
View File

@ -103,23 +103,33 @@ DEFINE_CMA_FSM_EVENT(sent_drep);
DEFINE_CMA_FSM_EVENT(sent_dreq);
DEFINE_CMA_FSM_EVENT(id_destroy);
TRACE_EVENT(cm_id_create,
TRACE_EVENT(cm_id_attach,
TP_PROTO(
const struct rdma_id_private *id_priv
const struct rdma_id_private *id_priv,
const struct ib_device *device
),
TP_ARGS(id_priv),
TP_ARGS(id_priv, device),
TP_STRUCT__entry(
__field(u32, cm_id)
__array(unsigned char, srcaddr, sizeof(struct sockaddr_in6))
__array(unsigned char, dstaddr, sizeof(struct sockaddr_in6))
__string(devname, device->name)
),
TP_fast_assign(
__entry->cm_id = id_priv->res.id;
memcpy(__entry->srcaddr, &id_priv->id.route.addr.src_addr,
sizeof(struct sockaddr_in6));
memcpy(__entry->dstaddr, &id_priv->id.route.addr.dst_addr,
sizeof(struct sockaddr_in6));
__assign_str(devname, device->name);
),
TP_printk("cm.id=%u",
__entry->cm_id
TP_printk("cm.id=%u src=%pISpc dst=%pISpc device=%s",
__entry->cm_id, __entry->srcaddr, __entry->dstaddr,
__get_str(devname)
)
);

3
drivers/infiniband/core/core_priv.h
View File

@ -414,4 +414,7 @@ void rdma_umap_priv_init(struct rdma_umap_priv *priv,
struct vm_area_struct *vma,
struct rdma_user_mmap_entry *entry);
void ib_cq_pool_init(struct ib_device *dev);
void ib_cq_pool_destroy(struct ib_device *dev);
#endif /* _CORE_PRIV_H */

173
drivers/infiniband/core/cq.c
View File

@ -7,7 +7,11 @@
#include <linux/slab.h>
#include <rdma/ib_verbs.h>
#include "core_priv.h"
#include <trace/events/rdma_core.h>
/* Max size for shared CQ, may require tuning */
#define IB_MAX_SHARED_CQ_SZ 4096U
/* # of WCs to poll for with a single call to ib_poll_cq */
#define IB_POLL_BATCH 16
@ -218,6 +222,7 @@ struct ib_cq *__ib_alloc_cq_user(struct ib_device *dev, void *private,
cq->cq_context = private;
cq->poll_ctx = poll_ctx;
atomic_set(&cq->usecnt, 0);
cq->comp_vector = comp_vector;
cq->wc = kmalloc_array(IB_POLL_BATCH, sizeof(*cq->wc), GFP_KERNEL);
if (!cq->wc)
@ -309,6 +314,8 @@ void ib_free_cq_user(struct ib_cq *cq, struct ib_udata *udata)
{
if (WARN_ON_ONCE(atomic_read(&cq->usecnt)))
return;
if (WARN_ON_ONCE(cq->cqe_used))
return;
switch (cq->poll_ctx) {
case IB_POLL_DIRECT:
@ -334,3 +341,169 @@ void ib_free_cq_user(struct ib_cq *cq, struct ib_udata *udata)
kfree(cq);
}
EXPORT_SYMBOL(ib_free_cq_user);
void ib_cq_pool_init(struct ib_device *dev)
{
unsigned int i;
spin_lock_init(&dev->cq_pools_lock);
for (i = 0; i < ARRAY_SIZE(dev->cq_pools); i++)
INIT_LIST_HEAD(&dev->cq_pools[i]);
}
void ib_cq_pool_destroy(struct ib_device *dev)
{
struct ib_cq *cq, *n;
unsigned int i;
for (i = 0; i < ARRAY_SIZE(dev->cq_pools); i++) {
list_for_each_entry_safe(cq, n, &dev->cq_pools[i],
pool_entry) {
WARN_ON(cq->cqe_used);
cq->shared = false;
ib_free_cq(cq);
}
}
}
static int ib_alloc_cqs(struct ib_device *dev, unsigned int nr_cqes,
enum ib_poll_context poll_ctx)
{
LIST_HEAD(tmp_list);
unsigned int nr_cqs, i;
struct ib_cq *cq;
int ret;
if (poll_ctx > IB_POLL_LAST_POOL_TYPE) {
WARN_ON_ONCE(poll_ctx > IB_POLL_LAST_POOL_TYPE);
return -EINVAL;
}
/*
* Allocate at least as many CQEs as requested, and otherwise
* a reasonable batch size so that we can share CQs between
* multiple users instead of allocating a larger number of CQs.
*/
nr_cqes = min_t(unsigned int, dev->attrs.max_cqe,
max(nr_cqes, IB_MAX_SHARED_CQ_SZ));
nr_cqs = min_t(unsigned int, dev->num_comp_vectors, num_online_cpus());
for (i = 0; i < nr_cqs; i++) {
cq = ib_alloc_cq(dev, NULL, nr_cqes, i, poll_ctx);
if (IS_ERR(cq)) {
ret = PTR_ERR(cq);
goto out_free_cqs;
}
cq->shared = true;
list_add_tail(&cq->pool_entry, &tmp_list);
}
spin_lock_irq(&dev->cq_pools_lock);
list_splice(&tmp_list, &dev->cq_pools[poll_ctx]);
spin_unlock_irq(&dev->cq_pools_lock);
return 0;
out_free_cqs:
list_for_each_entry(cq, &tmp_list, pool_entry) {
cq->shared = false;
ib_free_cq(cq);
}
return ret;
}
/**
* ib_cq_pool_get() - Find the least used completion queue that matches
* a given cpu hint (or least used for wild card affinity) and fits
* nr_cqe.
* @dev: rdma device
* @nr_cqe: number of needed cqe entries
* @comp_vector_hint: completion vector hint (-1) for the driver to assign
* a comp vector based on internal counter
* @poll_ctx: cq polling context
*
* Finds a cq that satisfies @comp_vector_hint and @nr_cqe requirements and
* claim entries in it for us. In case there is no available cq, allocate
* a new cq with the requirements and add it to the device pool.
* IB_POLL_DIRECT cannot be used for shared cqs so it is not a valid value
* for @poll_ctx.
*/
struct ib_cq *ib_cq_pool_get(struct ib_device *dev, unsigned int nr_cqe,
int comp_vector_hint,
enum ib_poll_context poll_ctx)
{
static unsigned int default_comp_vector;
unsigned int vector, num_comp_vectors;
struct ib_cq *cq, *found = NULL;
int ret;
if (poll_ctx > IB_POLL_LAST_POOL_TYPE) {
WARN_ON_ONCE(poll_ctx > IB_POLL_LAST_POOL_TYPE);
return ERR_PTR(-EINVAL);
}
num_comp_vectors =
min_t(unsigned int, dev->num_comp_vectors, num_online_cpus());
/* Project the affinty to the device completion vector range */
if (comp_vector_hint < 0) {
comp_vector_hint =
(READ_ONCE(default_comp_vector) + 1) % num_comp_vectors;
WRITE_ONCE(default_comp_vector, comp_vector_hint);
}
vector = comp_vector_hint % num_comp_vectors;
/*
* Find the least used CQ with correct affinity and
* enough free CQ entries
*/
while (!found) {
spin_lock_irq(&dev->cq_pools_lock);
list_for_each_entry(cq, &dev->cq_pools[poll_ctx],
pool_entry) {
/*
* Check to see if we have found a CQ with the
* correct completion vector
*/
if (vector != cq->comp_vector)
continue;
if (cq->cqe_used + nr_cqe > cq->cqe)
continue;
found = cq;
break;
}
if (found) {
found->cqe_used += nr_cqe;
spin_unlock_irq(&dev->cq_pools_lock);
return found;
}
spin_unlock_irq(&dev->cq_pools_lock);
/*
* Didn't find a match or ran out of CQs in the device
* pool, allocate a new array of CQs.
*/
ret = ib_alloc_cqs(dev, nr_cqe, poll_ctx);
if (ret)
return ERR_PTR(ret);
}
return found;
}
EXPORT_SYMBOL(ib_cq_pool_get);
/**
* ib_cq_pool_put - Return a CQ taken from a shared pool.
* @cq: The CQ to return.
* @nr_cqe: The max number of cqes that the user had requested.
*/
void ib_cq_pool_put(struct ib_cq *cq, unsigned int nr_cqe)
{
if (WARN_ON_ONCE(nr_cqe > cq->cqe_used))
return;
spin_lock_irq(&cq->device->cq_pools_lock);
cq->cqe_used -= nr_cqe;
spin_unlock_irq(&cq->device->cq_pools_lock);
}
EXPORT_SYMBOL(ib_cq_pool_put);

22
drivers/infiniband/core/device.c
View File

@ -677,8 +677,20 @@ static int add_client_context(struct ib_device *device,
if (ret)
goto out;
downgrade_write(&device->client_data_rwsem);
if (client->add)
client->add(device);
if (client->add) {
if (client->add(device)) {
/*
* If a client fails to add then the error code is
* ignored, but we won't call any more ops on this
* client.
*/
xa_erase(&device->client_data, client->client_id);
up_read(&device->client_data_rwsem);
ib_device_put(device);
ib_client_put(client);
return 0;
}
}
/* Readers shall not see a client until add has been completed */
xa_set_mark(&device->client_data, client->client_id,
@ -1381,6 +1393,7 @@ int ib_register_device(struct ib_device *device, const char *name)
goto dev_cleanup;
}
ib_cq_pool_init(device);
ret = enable_device_and_get(device);
dev_set_uevent_suppress(&device->dev, false);
/* Mark for userspace that device is ready */
@ -1435,6 +1448,7 @@ static void __ib_unregister_device(struct ib_device *ib_dev)
goto out;
disable_device(ib_dev);
ib_cq_pool_destroy(ib_dev);
/* Expedite removing unregistered pointers from the hash table */
free_netdevs(ib_dev);
@ -2557,7 +2571,6 @@ void ib_set_device_ops(struct ib_device *dev, const struct ib_device_ops *ops)
SET_DEVICE_OP(dev_ops, add_gid);
SET_DEVICE_OP(dev_ops, advise_mr);
SET_DEVICE_OP(dev_ops, alloc_dm);
SET_DEVICE_OP(dev_ops, alloc_fmr);
SET_DEVICE_OP(dev_ops, alloc_hw_stats);
SET_DEVICE_OP(dev_ops, alloc_mr);
SET_DEVICE_OP(dev_ops, alloc_mr_integrity);
@ -2584,7 +2597,6 @@ void ib_set_device_ops(struct ib_device *dev, const struct ib_device_ops *ops)
SET_DEVICE_OP(dev_ops, create_wq);
SET_DEVICE_OP(dev_ops, dealloc_dm);
SET_DEVICE_OP(dev_ops, dealloc_driver);
SET_DEVICE_OP(dev_ops, dealloc_fmr);
SET_DEVICE_OP(dev_ops, dealloc_mw);
SET_DEVICE_OP(dev_ops, dealloc_pd);
SET_DEVICE_OP(dev_ops, dealloc_ucontext);
@ -2628,7 +2640,6 @@ void ib_set_device_ops(struct ib_device *dev, const struct ib_device_ops *ops)
SET_DEVICE_OP(dev_ops, iw_rem_ref);
SET_DEVICE_OP(dev_ops, map_mr_sg);
SET_DEVICE_OP(dev_ops, map_mr_sg_pi);
SET_DEVICE_OP(dev_ops, map_phys_fmr);
SET_DEVICE_OP(dev_ops, mmap);
SET_DEVICE_OP(dev_ops, mmap_free);
SET_DEVICE_OP(dev_ops, modify_ah);
@ -2662,7 +2673,6 @@ void ib_set_device_ops(struct ib_device *dev, const struct ib_device_ops *ops)
SET_DEVICE_OP(dev_ops, resize_cq);
SET_DEVICE_OP(dev_ops, set_vf_guid);
SET_DEVICE_OP(dev_ops, set_vf_link_state);
SET_DEVICE_OP(dev_ops, unmap_fmr);
SET_OBJ_SIZE(dev_ops, ib_ah);
SET_OBJ_SIZE(dev_ops, ib_cq);

494
drivers/infiniband/core/fmr_pool.c
View File

@ -1,494 +0,0 @@
/*
* Copyright (c) 2004 Topspin Communications. All rights reserved.
* Copyright (c) 2005 Sun Microsystems, Inc. All rights reserved.
*
* This software is available to you under a choice of one of two
* licenses. You may choose to be licensed under the terms of the GNU
* General Public License (GPL) Version 2, available from the file
* COPYING in the main directory of this source tree, or the
* OpenIB.org BSD license below:
*
* Redistribution and use in source and binary forms, with or
* without modification, are permitted provided that the following
* conditions are met:
*
* - Redistributions of source code must retain the above
* copyright notice, this list of conditions and the following
* disclaimer.
*
* - Redistributions in binary form must reproduce the above
* copyright notice, this list of conditions and the following
* disclaimer in the documentation and/or other materials
* provided with the distribution.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
* MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
* BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
* ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
#include <linux/errno.h>
#include <linux/spinlock.h>
#include <linux/export.h>
#include <linux/slab.h>
#include <linux/jhash.h>
#include <linux/kthread.h>
#include <rdma/ib_fmr_pool.h>
#include "core_priv.h"
#define PFX "fmr_pool: "
enum {
IB_FMR_MAX_REMAPS = 32,
IB_FMR_HASH_BITS = 8,
IB_FMR_HASH_SIZE = 1 << IB_FMR_HASH_BITS,
IB_FMR_HASH_MASK = IB_FMR_HASH_SIZE - 1
};
/*
* If an FMR is not in use, then the list member will point to either
* its pool's free_list (if the FMR can be mapped again; that is,
* remap_count < pool->max_remaps) or its pool's dirty_list (if the
* FMR needs to be unmapped before being remapped). In either of
* these cases it is a bug if the ref_count is not 0. In other words,
* if ref_count is > 0, then the list member must not be linked into
* either free_list or dirty_list.
*
* The cache_node member is used to link the FMR into a cache bucket
* (if caching is enabled). This is independent of the reference
* count of the FMR. When a valid FMR is released, its ref_count is
* decremented, and if ref_count reaches 0, the FMR is placed in
* either free_list or dirty_list as appropriate. However, it is not
* removed from the cache and may be "revived" if a call to
* ib_fmr_register_physical() occurs before the FMR is remapped. In
* this case we just increment the ref_count and remove the FMR from
* free_list/dirty_list.
*
* Before we remap an FMR from free_list, we remove it from the cache
* (to prevent another user from obtaining a stale FMR). When an FMR
* is released, we add it to the tail of the free list, so that our
* cache eviction policy is "least recently used."
*
* All manipulation of ref_count, list and cache_node is protected by
* pool_lock to maintain consistency.
*/
struct ib_fmr_pool {
spinlock_t pool_lock;
int pool_size;
int max_pages;
int max_remaps;
int dirty_watermark;
int dirty_len;
struct list_head free_list;
struct list_head dirty_list;
struct hlist_head *cache_bucket;
void (*flush_function)(struct ib_fmr_pool *pool,
void * arg);
void *flush_arg;
struct kthread_worker *worker;
struct kthread_work work;
atomic_t req_ser;
atomic_t flush_ser;
wait_queue_head_t force_wait;
};
static inline u32 ib_fmr_hash(u64 first_page)
{
return jhash_2words((u32) first_page, (u32) (first_page >> 32), 0) &
(IB_FMR_HASH_SIZE - 1);
}
/* Caller must hold pool_lock */
static inline struct ib_pool_fmr *ib_fmr_cache_lookup(struct ib_fmr_pool *pool,
u64 *page_list,
int page_list_len,
u64 io_virtual_address)
{
struct hlist_head *bucket;
struct ib_pool_fmr *fmr;
if (!pool->cache_bucket)
return NULL;
bucket = pool->cache_bucket + ib_fmr_hash(*page_list);
hlist_for_each_entry(fmr, bucket, cache_node)
if (io_virtual_address == fmr->io_virtual_address &&
page_list_len == fmr->page_list_len &&
!memcmp(page_list, fmr->page_list,
page_list_len * sizeof *page_list))
return fmr;
return NULL;
}
static void ib_fmr_batch_release(struct ib_fmr_pool *pool)
{
int ret;
struct ib_pool_fmr *fmr;
LIST_HEAD(unmap_list);
LIST_HEAD(fmr_list);
spin_lock_irq(&pool->pool_lock);
list_for_each_entry(fmr, &pool->dirty_list, list) {
hlist_del_init(&fmr->cache_node);
fmr->remap_count = 0;
list_add_tail(&fmr->fmr->list, &fmr_list);
}
list_splice_init(&pool->dirty_list, &unmap_list);
pool->dirty_len = 0;
spin_unlock_irq(&pool->pool_lock);
if (list_empty(&unmap_list)) {
return;
}
ret = ib_unmap_fmr(&fmr_list);
if (ret)
pr_warn(PFX "ib_unmap_fmr returned %d\n", ret);
spin_lock_irq(&pool->pool_lock);
list_splice(&unmap_list, &pool->free_list);
spin_unlock_irq(&pool->pool_lock);
}
static void ib_fmr_cleanup_func(struct kthread_work *work)
{
struct ib_fmr_pool *pool = container_of(work, struct ib_fmr_pool, work);
ib_fmr_batch_release(pool);
atomic_inc(&pool->flush_ser);
wake_up_interruptible(&pool->force_wait);
if (pool->flush_function)
pool->flush_function(pool, pool->flush_arg);
if (atomic_read(&pool->flush_ser) - atomic_read(&pool->req_ser) < 0)
kthread_queue_work(pool->worker, &pool->work);
}
/**
* ib_create_fmr_pool - Create an FMR pool
* @pd:Protection domain for FMRs
* @params:FMR pool parameters
*
* Create a pool of FMRs. Return value is pointer to new pool or
* error code if creation failed.
*/
struct ib_fmr_pool *ib_create_fmr_pool(struct ib_pd *pd,
struct ib_fmr_pool_param *params)
{
struct ib_device *device;
struct ib_fmr_pool *pool;
int i;
int ret;
int max_remaps;
if (!params)
return ERR_PTR(-EINVAL);
device = pd->device;
if (!device->ops.alloc_fmr || !device->ops.dealloc_fmr ||
!device->ops.map_phys_fmr || !device->ops.unmap_fmr) {
dev_info(&device->dev, "Device does not support FMRs\n");
return ERR_PTR(-ENOSYS);
}
if (!device->attrs.max_map_per_fmr)
max_remaps = IB_FMR_MAX_REMAPS;
else
max_remaps = device->attrs.max_map_per_fmr;
pool = kmalloc(sizeof *pool, GFP_KERNEL);
if (!pool)
return ERR_PTR(-ENOMEM);
pool->cache_bucket = NULL;
pool->flush_function = params->flush_function;
pool->flush_arg = params->flush_arg;
INIT_LIST_HEAD(&pool->free_list);
INIT_LIST_HEAD(&pool->dirty_list);
if (params->cache) {
pool->cache_bucket =
kmalloc_array(IB_FMR_HASH_SIZE,
sizeof(*pool->cache_bucket),
GFP_KERNEL);
if (!pool->cache_bucket) {
ret = -ENOMEM;
goto out_free_pool;
}
for (i = 0; i < IB_FMR_HASH_SIZE; ++i)
INIT_HLIST_HEAD(pool->cache_bucket + i);
}
pool->pool_size = 0;
pool->max_pages = params->max_pages_per_fmr;
pool->max_remaps = max_remaps;
pool->dirty_watermark = params->dirty_watermark;
pool->dirty_len = 0;
spin_lock_init(&pool->pool_lock);
atomic_set(&pool->req_ser, 0);
atomic_set(&pool->flush_ser, 0);
init_waitqueue_head(&pool->force_wait);
pool->worker =
kthread_create_worker(0, "ib_fmr(%s)", dev_name(&device->dev));
if (IS_ERR(pool->worker)) {
pr_warn(PFX "couldn't start cleanup kthread worker\n");
ret = PTR_ERR(pool->worker);
goto out_free_pool;
}
kthread_init_work(&pool->work, ib_fmr_cleanup_func);
{
struct ib_pool_fmr *fmr;
struct ib_fmr_attr fmr_attr = {
.max_pages = params->max_pages_per_fmr,
.max_maps = pool->max_remaps,
.page_shift = params->page_shift
};
int bytes_per_fmr = sizeof *fmr;
if (pool->cache_bucket)
bytes_per_fmr += params->max_pages_per_fmr * sizeof (u64);
for (i = 0; i < params->pool_size; ++i) {
fmr = kmalloc(bytes_per_fmr, GFP_KERNEL);
if (!fmr)
goto out_fail;
fmr->pool = pool;
fmr->remap_count = 0;
fmr->ref_count = 0;
INIT_HLIST_NODE(&fmr->cache_node);
fmr->fmr = ib_alloc_fmr(pd, params->access, &fmr_attr);
if (IS_ERR(fmr->fmr)) {
pr_warn(PFX "fmr_create failed for FMR %d\n",
i);
kfree(fmr);
goto out_fail;
}
list_add_tail(&fmr->list, &pool->free_list);
++pool->pool_size;
}
}
return pool;
out_free_pool:
kfree(pool->cache_bucket);
kfree(pool);
return ERR_PTR(ret);
out_fail:
ib_destroy_fmr_pool(pool);
return ERR_PTR(-ENOMEM);
}
EXPORT_SYMBOL(ib_create_fmr_pool);
/**
* ib_destroy_fmr_pool - Free FMR pool
* @pool:FMR pool to free
*
* Destroy an FMR pool and free all associated resources.
*/
void ib_destroy_fmr_pool(struct ib_fmr_pool *pool)
{
struct ib_pool_fmr *fmr;
struct ib_pool_fmr *tmp;
LIST_HEAD(fmr_list);
int i;
kthread_destroy_worker(pool->worker);
ib_fmr_batch_release(pool);
i = 0;
list_for_each_entry_safe(fmr, tmp, &pool->free_list, list) {
if (fmr->remap_count) {
INIT_LIST_HEAD(&fmr_list);
list_add_tail(&fmr->fmr->list, &fmr_list);
ib_unmap_fmr(&fmr_list);
}
ib_dealloc_fmr(fmr->fmr);
list_del(&fmr->list);
kfree(fmr);
++i;
}
if (i < pool->pool_size)
pr_warn(PFX "pool still has %d regions registered\n",
pool->pool_size - i);
kfree(pool->cache_bucket);
kfree(pool);
}
EXPORT_SYMBOL(ib_destroy_fmr_pool);
/**
* ib_flush_fmr_pool - Invalidate all unmapped FMRs
* @pool:FMR pool to flush
*
* Ensure that all unmapped FMRs are fully invalidated.
*/
int ib_flush_fmr_pool(struct ib_fmr_pool *pool)
{
int serial;
struct ib_pool_fmr *fmr, *next;
/*
* The free_list holds FMRs that may have been used
* but have not been remapped enough times to be dirty.
* Put them on the dirty list now so that the cleanup
* thread will reap them too.
*/
spin_lock_irq(&pool->pool_lock);
list_for_each_entry_safe(fmr, next, &pool->free_list, list) {
if (fmr->remap_count > 0)
list_move(&fmr->list, &pool->dirty_list);
}
spin_unlock_irq(&pool->pool_lock);
serial = atomic_inc_return(&pool->req_ser);
kthread_queue_work(pool->worker, &pool->work);
if (wait_event_interruptible(pool->force_wait,
atomic_read(&pool->flush_ser) - serial >= 0))
return -EINTR;
return 0;
}
EXPORT_SYMBOL(ib_flush_fmr_pool);
/**
* ib_fmr_pool_map_phys - Map an FMR from an FMR pool.
* @pool_handle: FMR pool to allocate FMR from
* @page_list: List of pages to map
* @list_len: Number of pages in @page_list
* @io_virtual_address: I/O virtual address for new FMR
*/
struct ib_pool_fmr *ib_fmr_pool_map_phys(struct ib_fmr_pool *pool_handle,
u64 *page_list,
int list_len,
u64 io_virtual_address)
{
struct ib_fmr_pool *pool = pool_handle;
struct ib_pool_fmr *fmr;
unsigned long flags;
int result;
if (list_len < 1 || list_len > pool->max_pages)
return ERR_PTR(-EINVAL);
spin_lock_irqsave(&pool->pool_lock, flags);
fmr = ib_fmr_cache_lookup(pool,
page_list,
list_len,
io_virtual_address);
if (fmr) {
/* found in cache */
++fmr->ref_count;
if (fmr->ref_count == 1) {
list_del(&fmr->list);
}
spin_unlock_irqrestore(&pool->pool_lock, flags);
return fmr;
}
if (list_empty(&pool->free_list)) {
spin_unlock_irqrestore(&pool->pool_lock, flags);
return ERR_PTR(-EAGAIN);
}
fmr = list_entry(pool->free_list.next, struct ib_pool_fmr, list);
list_del(&fmr->list);
hlist_del_init(&fmr->cache_node);
spin_unlock_irqrestore(&pool->pool_lock, flags);
result = ib_map_phys_fmr(fmr->fmr, page_list, list_len,
io_virtual_address);
if (result) {
spin_lock_irqsave(&pool->pool_lock, flags);
list_add(&fmr->list, &pool->free_list);
spin_unlock_irqrestore(&pool->pool_lock, flags);
pr_warn(PFX "fmr_map returns %d\n", result);
return ERR_PTR(result);
}
++fmr->remap_count;
fmr->ref_count = 1;
if (pool->cache_bucket) {
fmr->io_virtual_address = io_virtual_address;
fmr->page_list_len = list_len;
memcpy(fmr->page_list, page_list, list_len * sizeof(*page_list));
spin_lock_irqsave(&pool->pool_lock, flags);
hlist_add_head(&fmr->cache_node,
pool->cache_bucket + ib_fmr_hash(fmr->page_list[0]));
spin_unlock_irqrestore(&pool->pool_lock, flags);
}
return fmr;
}
EXPORT_SYMBOL(ib_fmr_pool_map_phys);
/**
* ib_fmr_pool_unmap - Unmap FMR
* @fmr:FMR to unmap
*
* Unmap an FMR. The FMR mapping may remain valid until the FMR is
* reused (or until ib_flush_fmr_pool() is called).
*/
void ib_fmr_pool_unmap(struct ib_pool_fmr *fmr)
{
struct ib_fmr_pool *pool;
unsigned long flags;
pool = fmr->pool;
spin_lock_irqsave(&pool->pool_lock, flags);
--fmr->ref_count;
if (!fmr->ref_count) {
if (fmr->remap_count < pool->max_remaps) {
list_add_tail(&fmr->list, &pool->free_list);
} else {
list_add_tail(&fmr->list, &pool->dirty_list);
if (++pool->dirty_len >= pool->dirty_watermark) {
atomic_inc(&pool->req_ser);
kthread_queue_work(pool->worker, &pool->work);
}
}
}
spin_unlock_irqrestore(&pool->pool_lock, flags);
}
EXPORT_SYMBOL(ib_fmr_pool_unmap);

138
drivers/infiniband/core/lag.c Normal file
View File

@ -0,0 +1,138 @@
// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
/*
* Copyright (c) 2020 Mellanox Technologies. All rights reserved.
*/
#include <rdma/ib_verbs.h>
#include <rdma/ib_cache.h>
#include <rdma/lag.h>
static struct sk_buff *rdma_build_skb(struct ib_device *device,
struct net_device *netdev,
struct rdma_ah_attr *ah_attr,
gfp_t flags)
{
struct ipv6hdr *ip6h;
struct sk_buff *skb;
struct ethhdr *eth;
struct iphdr *iph;
struct udphdr *uh;
u8 smac[ETH_ALEN];
bool is_ipv4;
int hdr_len;
is_ipv4 = ipv6_addr_v4mapped((struct in6_addr *)ah_attr->grh.dgid.raw);
hdr_len = ETH_HLEN + sizeof(struct udphdr) + LL_RESERVED_SPACE(netdev);
hdr_len += is_ipv4 ? sizeof(struct iphdr) : sizeof(struct ipv6hdr);
skb = alloc_skb(hdr_len, flags);
if (!skb)
return NULL;
skb->dev = netdev;
skb_reserve(skb, hdr_len);
skb_push(skb, sizeof(struct udphdr));
skb_reset_transport_header(skb);
uh = udp_hdr(skb);
uh->source =
htons(rdma_flow_label_to_udp_sport(ah_attr->grh.flow_label));
uh->dest = htons(ROCE_V2_UDP_DPORT);
uh->len = htons(sizeof(struct udphdr));
if (is_ipv4) {
skb_push(skb, sizeof(struct iphdr));
skb_reset_network_header(skb);
iph = ip_hdr(skb);
iph->frag_off = 0;
iph->version = 4;
iph->protocol = IPPROTO_UDP;
iph->ihl = 0x5;
iph->tot_len = htons(sizeof(struct udphdr) + sizeof(struct
iphdr));
memcpy(&iph->saddr, ah_attr->grh.sgid_attr->gid.raw + 12,
sizeof(struct in_addr));
memcpy(&iph->daddr, ah_attr->grh.dgid.raw + 12,
sizeof(struct in_addr));
} else {
skb_push(skb, sizeof(struct ipv6hdr));
skb_reset_network_header(skb);
ip6h = ipv6_hdr(skb);
ip6h->version = 6;
ip6h->nexthdr = IPPROTO_UDP;
memcpy(&ip6h->flow_lbl, &ah_attr->grh.flow_label,
sizeof(*ip6h->flow_lbl));
memcpy(&ip6h->saddr, ah_attr->grh.sgid_attr->gid.raw,
sizeof(struct in6_addr));
memcpy(&ip6h->daddr, ah_attr->grh.dgid.raw,
sizeof(struct in6_addr));
}
skb_push(skb, sizeof(struct ethhdr));
skb_reset_mac_header(skb);
eth = eth_hdr(skb);
skb->protocol = eth->h_proto = htons(is_ipv4 ? ETH_P_IP : ETH_P_IPV6);
rdma_read_gid_l2_fields(ah_attr->grh.sgid_attr, NULL, smac);
memcpy(eth->h_source, smac, ETH_ALEN);
memcpy(eth->h_dest, ah_attr->roce.dmac, ETH_ALEN);
return skb;
}
static struct net_device *rdma_get_xmit_slave_udp(struct ib_device *device,
struct net_device *master,
struct rdma_ah_attr *ah_attr,
gfp_t flags)
{
struct net_device *slave;
struct sk_buff *skb;
skb = rdma_build_skb(device, master, ah_attr, flags);
if (!skb)
return ERR_PTR(-ENOMEM);
rcu_read_lock();
slave = netdev_get_xmit_slave(master, skb,
!!(device->lag_flags &
RDMA_LAG_FLAGS_HASH_ALL_SLAVES));
if (slave)
dev_hold(slave);
rcu_read_unlock();
kfree_skb(skb);
return slave;
}
void rdma_lag_put_ah_roce_slave(struct net_device *xmit_slave)
{
if (xmit_slave)
dev_put(xmit_slave);
}
struct net_device *rdma_lag_get_ah_roce_slave(struct ib_device *device,
struct rdma_ah_attr *ah_attr,
gfp_t flags)
{
struct net_device *slave = NULL;
struct net_device *master;
if (!(ah_attr->type == RDMA_AH_ATTR_TYPE_ROCE &&
ah_attr->grh.sgid_attr->gid_type == IB_GID_TYPE_ROCE_UDP_ENCAP &&
ah_attr->grh.flow_label))
return NULL;
rcu_read_lock();
master = rdma_read_gid_attr_ndev_rcu(ah_attr->grh.sgid_attr);
if (IS_ERR(master)) {
rcu_read_unlock();
return master;
}
dev_hold(master);
rcu_read_unlock();
if (!netif_is_bond_master(master))
goto put;
slave = rdma_get_xmit_slave_udp(device, master, ah_attr, flags);
put:
dev_put(master);
return slave;
}

253
drivers/infiniband/core/mad.c
View File

@ -85,7 +85,6 @@ MODULE_PARM_DESC(send_queue_size, "Size of send queue in number of work requests
module_param_named(recv_queue_size, mad_recvq_size, int, 0444);
MODULE_PARM_DESC(recv_queue_size, "Size of receive queue in number of work requests");
/* Client ID 0 is used for snoop-only clients */
static DEFINE_XARRAY_ALLOC1(ib_mad_clients);
static u32 ib_mad_client_next;
static struct list_head ib_mad_port_list;
@ -483,141 +482,12 @@ struct ib_mad_agent *ib_register_mad_agent(struct ib_device *device,
}
EXPORT_SYMBOL(ib_register_mad_agent);
static inline int is_snooping_sends(int mad_snoop_flags)
{
return (mad_snoop_flags &
(/*IB_MAD_SNOOP_POSTED_SENDS |
IB_MAD_SNOOP_RMPP_SENDS |*/
IB_MAD_SNOOP_SEND_COMPLETIONS /*|
IB_MAD_SNOOP_RMPP_SEND_COMPLETIONS*/));
}
static inline int is_snooping_recvs(int mad_snoop_flags)
{
return (mad_snoop_flags &
(IB_MAD_SNOOP_RECVS /*|
IB_MAD_SNOOP_RMPP_RECVS*/));
}
static int register_snoop_agent(struct ib_mad_qp_info *qp_info,
struct ib_mad_snoop_private *mad_snoop_priv)
{
struct ib_mad_snoop_private **new_snoop_table;
unsigned long flags;
int i;
spin_lock_irqsave(&qp_info->snoop_lock, flags);
/* Check for empty slot in array. */
for (i = 0; i < qp_info->snoop_table_size; i++)
if (!qp_info->snoop_table[i])
break;
if (i == qp_info->snoop_table_size) {
/* Grow table. */
new_snoop_table = krealloc(qp_info->snoop_table,
sizeof mad_snoop_priv *
(qp_info->snoop_table_size + 1),
GFP_ATOMIC);
if (!new_snoop_table) {
i = -ENOMEM;
goto out;
}
qp_info->snoop_table = new_snoop_table;
qp_info->snoop_table_size++;
}
qp_info->snoop_table[i] = mad_snoop_priv;
atomic_inc(&qp_info->snoop_count);
out:
spin_unlock_irqrestore(&qp_info->snoop_lock, flags);
return i;
}
struct ib_mad_agent *ib_register_mad_snoop(struct ib_device *device,
u8 port_num,
enum ib_qp_type qp_type,
int mad_snoop_flags,
ib_mad_snoop_handler snoop_handler,
ib_mad_recv_handler recv_handler,
void *context)
{
struct ib_mad_port_private *port_priv;
struct ib_mad_agent *ret;
struct ib_mad_snoop_private *mad_snoop_priv;
int qpn;
int err;
/* Validate parameters */
if ((is_snooping_sends(mad_snoop_flags) && !snoop_handler) ||
(is_snooping_recvs(mad_snoop_flags) && !recv_handler)) {
ret = ERR_PTR(-EINVAL);
goto error1;
}
qpn = get_spl_qp_index(qp_type);
if (qpn == -1) {
ret = ERR_PTR(-EINVAL);
goto error1;
}
port_priv = ib_get_mad_port(device, port_num);
if (!port_priv) {
ret = ERR_PTR(-ENODEV);
goto error1;
}
/* Allocate structures */
mad_snoop_priv = kzalloc(sizeof *mad_snoop_priv, GFP_KERNEL);
if (!mad_snoop_priv) {
ret = ERR_PTR(-ENOMEM);
goto error1;
}
/* Now, fill in the various structures */
mad_snoop_priv->qp_info = &port_priv->qp_info[qpn];
mad_snoop_priv->agent.device = device;
mad_snoop_priv->agent.recv_handler = recv_handler;
mad_snoop_priv->agent.snoop_handler = snoop_handler;
mad_snoop_priv->agent.context = context;
mad_snoop_priv->agent.qp = port_priv->qp_info[qpn].qp;
mad_snoop_priv->agent.port_num = port_num;
mad_snoop_priv->mad_snoop_flags = mad_snoop_flags;
init_completion(&mad_snoop_priv->comp);
err = ib_mad_agent_security_setup(&mad_snoop_priv->agent, qp_type);
if (err) {
ret = ERR_PTR(err);
goto error2;
}
mad_snoop_priv->snoop_index = register_snoop_agent(
&port_priv->qp_info[qpn],
mad_snoop_priv);
if (mad_snoop_priv->snoop_index < 0) {
ret = ERR_PTR(mad_snoop_priv->snoop_index);
goto error3;
}
atomic_set(&mad_snoop_priv->refcount, 1);
return &mad_snoop_priv->agent;
error3:
ib_mad_agent_security_cleanup(&mad_snoop_priv->agent);
error2:
kfree(mad_snoop_priv);
error1:
return ret;
}
EXPORT_SYMBOL(ib_register_mad_snoop);
static inline void deref_mad_agent(struct ib_mad_agent_private *mad_agent_priv)
{
if (atomic_dec_and_test(&mad_agent_priv->refcount))
complete(&mad_agent_priv->comp);
}
static inline void deref_snoop_agent(struct ib_mad_snoop_private *mad_snoop_priv)
{
if (atomic_dec_and_test(&mad_snoop_priv->refcount))
complete(&mad_snoop_priv->comp);
}
static void unregister_mad_agent(struct ib_mad_agent_private *mad_agent_priv)
{
struct ib_mad_port_private *port_priv;
@ -650,25 +520,6 @@ static void unregister_mad_agent(struct ib_mad_agent_private *mad_agent_priv)
kfree_rcu(mad_agent_priv, rcu);
}
static void unregister_mad_snoop(struct ib_mad_snoop_private *mad_snoop_priv)
{
struct ib_mad_qp_info *qp_info;
unsigned long flags;
qp_info = mad_snoop_priv->qp_info;
spin_lock_irqsave(&qp_info->snoop_lock, flags);
qp_info->snoop_table[mad_snoop_priv->snoop_index] = NULL;
atomic_dec(&qp_info->snoop_count);
spin_unlock_irqrestore(&qp_info->snoop_lock, flags);
deref_snoop_agent(mad_snoop_priv);
wait_for_completion(&mad_snoop_priv->comp);
ib_mad_agent_security_cleanup(&mad_snoop_priv->agent);
kfree(mad_snoop_priv);
}
/*
* ib_unregister_mad_agent - Unregisters a client from using MAD services
*
@ -677,20 +528,11 @@ static void unregister_mad_snoop(struct ib_mad_snoop_private *mad_snoop_priv)
void ib_unregister_mad_agent(struct ib_mad_agent *mad_agent)
{
struct ib_mad_agent_private *mad_agent_priv;
struct ib_mad_snoop_private *mad_snoop_priv;
/* If the TID is zero, the agent can only snoop. */
if (mad_agent->hi_tid) {
mad_agent_priv = container_of(mad_agent,
struct ib_mad_agent_private,
agent);
unregister_mad_agent(mad_agent_priv);
} else {
mad_snoop_priv = container_of(mad_agent,
struct ib_mad_snoop_private,
agent);
unregister_mad_snoop(mad_snoop_priv);
}
mad_agent_priv = container_of(mad_agent,
struct ib_mad_agent_private,
agent);
unregister_mad_agent(mad_agent_priv);
}
EXPORT_SYMBOL(ib_unregister_mad_agent);
@ -706,57 +548,6 @@ static void dequeue_mad(struct ib_mad_list_head *mad_list)
spin_unlock_irqrestore(&mad_queue->lock, flags);
}
static void snoop_send(struct ib_mad_qp_info *qp_info,
struct ib_mad_send_buf *send_buf,
struct ib_mad_send_wc *mad_send_wc,
int mad_snoop_flags)
{
struct ib_mad_snoop_private *mad_snoop_priv;
unsigned long flags;
int i;
spin_lock_irqsave(&qp_info->snoop_lock, flags);
for (i = 0; i < qp_info->snoop_table_size; i++) {
mad_snoop_priv = qp_info->snoop_table[i];
if (!mad_snoop_priv ||
!(mad_snoop_priv->mad_snoop_flags & mad_snoop_flags))
continue;
atomic_inc(&mad_snoop_priv->refcount);
spin_unlock_irqrestore(&qp_info->snoop_lock, flags);
mad_snoop_priv->agent.snoop_handler(&mad_snoop_priv->agent,
send_buf, mad_send_wc);
deref_snoop_agent(mad_snoop_priv);
spin_lock_irqsave(&qp_info->snoop_lock, flags);
}
spin_unlock_irqrestore(&qp_info->snoop_lock, flags);
}
static void snoop_recv(struct ib_mad_qp_info *qp_info,
struct ib_mad_recv_wc *mad_recv_wc,
int mad_snoop_flags)
{
struct ib_mad_snoop_private *mad_snoop_priv;
unsigned long flags;
int i;
spin_lock_irqsave(&qp_info->snoop_lock, flags);
for (i = 0; i < qp_info->snoop_table_size; i++) {
mad_snoop_priv = qp_info->snoop_table[i];
if (!mad_snoop_priv ||
!(mad_snoop_priv->mad_snoop_flags & mad_snoop_flags))
continue;
atomic_inc(&mad_snoop_priv->refcount);
spin_unlock_irqrestore(&qp_info->snoop_lock, flags);
mad_snoop_priv->agent.recv_handler(&mad_snoop_priv->agent, NULL,
mad_recv_wc);
deref_snoop_agent(mad_snoop_priv);
spin_lock_irqsave(&qp_info->snoop_lock, flags);
}
spin_unlock_irqrestore(&qp_info->snoop_lock, flags);
}
static void build_smp_wc(struct ib_qp *qp, struct ib_cqe *cqe, u16 slid,
u16 pkey_index, u8 port_num, struct ib_wc *wc)
{
@ -2289,9 +2080,6 @@ static void ib_mad_recv_done(struct ib_cq *cq, struct ib_wc *wc)
recv->header.recv_wc.recv_buf.mad = (struct ib_mad *)recv->mad;
recv->header.recv_wc.recv_buf.grh = &recv->grh;
if (atomic_read(&qp_info->snoop_count))
snoop_recv(qp_info, &recv->header.recv_wc, IB_MAD_SNOOP_RECVS);
/* Validate MAD */
if (!validate_mad((const struct ib_mad_hdr *)recv->mad, qp_info, opa))
goto out;
@ -2538,9 +2326,6 @@ static void ib_mad_send_done(struct ib_cq *cq, struct ib_wc *wc)
mad_send_wc.send_buf = &mad_send_wr->send_buf;
mad_send_wc.status = wc->status;
mad_send_wc.vendor_err = wc->vendor_err;
if (atomic_read(&qp_info->snoop_count))
snoop_send(qp_info, &mad_send_wr->send_buf, &mad_send_wc,
IB_MAD_SNOOP_SEND_COMPLETIONS);
ib_mad_complete_send_wr(mad_send_wr, &mad_send_wc);
if (queued_send_wr) {
@ -2782,10 +2567,6 @@ static void local_completions(struct work_struct *work)
local->mad_priv->header.recv_wc.recv_buf.grh = NULL;
local->mad_priv->header.recv_wc.recv_buf.mad =
(struct ib_mad *)local->mad_priv->mad;
if (atomic_read(&recv_mad_agent->qp_info->snoop_count))
snoop_recv(recv_mad_agent->qp_info,
&local->mad_priv->header.recv_wc,
IB_MAD_SNOOP_RECVS);
recv_mad_agent->agent.recv_handler(
&recv_mad_agent->agent,
&local->mad_send_wr->send_buf,
@ -2800,10 +2581,6 @@ static void local_completions(struct work_struct *work)
mad_send_wc.status = IB_WC_SUCCESS;
mad_send_wc.vendor_err = 0;
mad_send_wc.send_buf = &local->mad_send_wr->send_buf;
if (atomic_read(&mad_agent_priv->qp_info->snoop_count))
snoop_send(mad_agent_priv->qp_info,
&local->mad_send_wr->send_buf,
&mad_send_wc, IB_MAD_SNOOP_SEND_COMPLETIONS);
mad_agent_priv->agent.send_handler(&mad_agent_priv->agent,
&mad_send_wc);
@ -3119,10 +2896,6 @@ static void init_mad_qp(struct ib_mad_port_private *port_priv,
init_mad_queue(qp_info, &qp_info->send_queue);
init_mad_queue(qp_info, &qp_info->recv_queue);
INIT_LIST_HEAD(&qp_info->overflow_list);
spin_lock_init(&qp_info->snoop_lock);
qp_info->snoop_table = NULL;
qp_info->snoop_table_size = 0;
atomic_set(&qp_info->snoop_count, 0);
}
static int create_mad_qp(struct ib_mad_qp_info *qp_info,
@ -3166,7 +2939,6 @@ static void destroy_mad_qp(struct ib_mad_qp_info *qp_info)
return;
ib_destroy_qp(qp_info->qp);
kfree(qp_info->snoop_table);
}
/*
@ -3304,9 +3076,11 @@ static int ib_mad_port_close(struct ib_device *device, int port_num)
return 0;
}
static void ib_mad_init_device(struct ib_device *device)
static int ib_mad_init_device(struct ib_device *device)
{
int start, i;
unsigned int count = 0;
int ret;
start = rdma_start_port(device);
@ -3314,17 +3088,23 @@ static void ib_mad_init_device(struct ib_device *device)
if (!rdma_cap_ib_mad(device, i))
continue;
if (ib_mad_port_open(device, i)) {
ret = ib_mad_port_open(device, i);
if (ret) {
dev_err(&device->dev, "Couldn't open port %d\n", i);
goto error;
}
if (ib_agent_port_open(device, i)) {
ret = ib_agent_port_open(device, i);
if (ret) {
dev_err(&device->dev,
"Couldn't open port %d for agents\n", i);
goto error_agent;
}
count++;
}
return;
if (!count)
return -EOPNOTSUPP;
return 0;
error_agent:
if (ib_mad_port_close(device, i))
@ -3341,6 +3121,7 @@ static void ib_mad_init_device(struct ib_device *device)
if (ib_mad_port_close(device, i))
dev_err(&device->dev, "Couldn't close port %d\n", i);
}
return ret;
}
static void ib_mad_remove_device(struct ib_device *device, void *client_data)

12
drivers/infiniband/core/multicast.c
View File

@ -42,7 +42,7 @@
#include <rdma/ib_cache.h>
#include "sa.h"
static void mcast_add_one(struct ib_device *device);
static int mcast_add_one(struct ib_device *device);
static void mcast_remove_one(struct ib_device *device, void *client_data);
static struct ib_client mcast_client = {
@ -815,7 +815,7 @@ static void mcast_event_handler(struct ib_event_handler *handler,
}
}
static void mcast_add_one(struct ib_device *device)
static int mcast_add_one(struct ib_device *device)
{
struct mcast_device *dev;
struct mcast_port *port;
@ -825,7 +825,7 @@ static void mcast_add_one(struct ib_device *device)
dev = kmalloc(struct_size(dev, port, device->phys_port_cnt),
GFP_KERNEL);
if (!dev)
return;
return -ENOMEM;
dev->start_port = rdma_start_port(device);
dev->end_port = rdma_end_port(device);
@ -845,7 +845,7 @@ static void mcast_add_one(struct ib_device *device)
if (!count) {
kfree(dev);
return;
return -EOPNOTSUPP;
}
dev->device = device;
@ -853,6 +853,7 @@ static void mcast_add_one(struct ib_device *device)
INIT_IB_EVENT_HANDLER(&dev->event_handler, device, mcast_event_handler);
ib_register_event_handler(&dev->event_handler);
return 0;
}
static void mcast_remove_one(struct ib_device *device, void *client_data)
@ -861,9 +862,6 @@ static void mcast_remove_one(struct ib_device *device, void *client_data)
struct mcast_port *port;
int i;
if (!dev)
return;
ib_unregister_event_handler(&dev->event_handler);
flush_workqueue(mcast_wq);

25
drivers/infiniband/core/rdma_core.c
View File

@ -130,6 +130,17 @@ static int uverbs_destroy_uobject(struct ib_uobject *uobj,
lockdep_assert_held(&ufile->hw_destroy_rwsem);
assert_uverbs_usecnt(uobj, UVERBS_LOOKUP_WRITE);
if (reason == RDMA_REMOVE_ABORT_HWOBJ) {
reason = RDMA_REMOVE_ABORT;
ret = uobj->uapi_object->type_class->destroy_hw(uobj, reason,
attrs);
/*
* Drivers are not permitted to ignore RDMA_REMOVE_ABORT, see
* ib_is_destroy_retryable, cleanup_retryable == false here.
*/
WARN_ON(ret);
}
if (reason == RDMA_REMOVE_ABORT) {
WARN_ON(!list_empty(&uobj->list));
WARN_ON(!uobj->context);
@ -653,11 +664,15 @@ void rdma_alloc_commit_uobject(struct ib_uobject *uobj,
* object and anything else connected to uobj before calling this.
*/
void rdma_alloc_abort_uobject(struct ib_uobject *uobj,
struct uverbs_attr_bundle *attrs)
struct uverbs_attr_bundle *attrs,
bool hw_obj_valid)
{
struct ib_uverbs_file *ufile = uobj->ufile;
uverbs_destroy_uobject(uobj, RDMA_REMOVE_ABORT, attrs);
uverbs_destroy_uobject(uobj,
hw_obj_valid ? RDMA_REMOVE_ABORT_HWOBJ :
RDMA_REMOVE_ABORT,
attrs);
/* Matches the down_read in rdma_alloc_begin_uobject */
up_read(&ufile->hw_destroy_rwsem);
@ -927,8 +942,8 @@ uverbs_get_uobject_from_file(u16 object_id, enum uverbs_obj_access access,
}
void uverbs_finalize_object(struct ib_uobject *uobj,
enum uverbs_obj_access access, bool commit,
struct uverbs_attr_bundle *attrs)
enum uverbs_obj_access access, bool hw_obj_valid,
bool commit, struct uverbs_attr_bundle *attrs)
{
/*
* refcounts should be handled at the object level and not at the
@ -951,7 +966,7 @@ void uverbs_finalize_object(struct ib_uobject *uobj,
if (commit)
rdma_alloc_commit_uobject(uobj, attrs);
else
rdma_alloc_abort_uobject(uobj, attrs);
rdma_alloc_abort_uobject(uobj, attrs, hw_obj_valid);
break;
default:
WARN_ON(true);

7
drivers/infiniband/core/rdma_core.h
View File

@ -64,8 +64,8 @@ uverbs_get_uobject_from_file(u16 object_id, enum uverbs_obj_access access,
s64 id, struct uverbs_attr_bundle *attrs);
void uverbs_finalize_object(struct ib_uobject *uobj,
enum uverbs_obj_access access, bool commit,
struct uverbs_attr_bundle *attrs);
enum uverbs_obj_access access, bool hw_obj_valid,
bool commit, struct uverbs_attr_bundle *attrs);
int uverbs_output_written(const struct uverbs_attr_bundle *bundle, size_t idx);
@ -159,6 +159,9 @@ extern const struct uapi_definition uverbs_def_obj_dm[];
extern const struct uapi_definition uverbs_def_obj_flow_action[];
extern const struct uapi_definition uverbs_def_obj_intf[];
extern const struct uapi_definition uverbs_def_obj_mr[];
extern const struct uapi_definition uverbs_def_obj_qp[];
extern const struct uapi_definition uverbs_def_obj_srq[];
extern const struct uapi_definition uverbs_def_obj_wq[];
extern const struct uapi_definition uverbs_def_write_intf[];
static inline const struct uverbs_api_write_method *

2
drivers/infiniband/core/rw.c
View File

@ -129,7 +129,7 @@ static int rdma_rw_init_mr_wrs(struct rdma_rw_ctx *ctx, struct ib_qp *qp,
qp->integrity_en);
int i, j, ret = 0, count = 0;
ctx->nr_ops = (sg_cnt + pages_per_mr - 1) / pages_per_mr;
ctx->nr_ops = DIV_ROUND_UP(sg_cnt, pages_per_mr);
ctx->reg = kcalloc(ctx->nr_ops, sizeof(*ctx->reg), GFP_KERNEL);
if (!ctx->reg) {
ret = -ENOMEM;

51
drivers/infiniband/core/sa_query.c
View File

@ -174,7 +174,7 @@ static const struct nla_policy ib_nl_policy[LS_NLA_TYPE_MAX] = {
};
static void ib_sa_add_one(struct ib_device *device);
static int ib_sa_add_one(struct ib_device *device);
static void ib_sa_remove_one(struct ib_device *device, void *client_data);
static struct ib_client sa_client = {
@ -190,7 +190,7 @@ static u32 tid;
#define PATH_REC_FIELD(field) \
.struct_offset_bytes = offsetof(struct sa_path_rec, field), \
.struct_size_bytes = sizeof((struct sa_path_rec *)0)->field, \
.struct_size_bytes = sizeof_field(struct sa_path_rec, field), \
.field_name = "sa_path_rec:" #field
static const struct ib_field path_rec_table[] = {
@ -292,7 +292,7 @@ static const struct ib_field path_rec_table[] = {
.struct_offset_bytes = \
offsetof(struct sa_path_rec, field), \
.struct_size_bytes = \
sizeof((struct sa_path_rec *)0)->field, \
sizeof_field(struct sa_path_rec, field), \
.field_name = "sa_path_rec:" #field
static const struct ib_field opa_path_rec_table[] = {
@ -420,7 +420,7 @@ static const struct ib_field opa_path_rec_table[] = {
#define MCMEMBER_REC_FIELD(field) \
.struct_offset_bytes = offsetof(struct ib_sa_mcmember_rec, field), \
.struct_size_bytes = sizeof ((struct ib_sa_mcmember_rec *) 0)->field, \
.struct_size_bytes = sizeof_field(struct ib_sa_mcmember_rec, field), \
.field_name = "sa_mcmember_rec:" #field
static const struct ib_field mcmember_rec_table[] = {
@ -504,7 +504,7 @@ static const struct ib_field mcmember_rec_table[] = {
#define SERVICE_REC_FIELD(field) \
.struct_offset_bytes = offsetof(struct ib_sa_service_rec, field), \
.struct_size_bytes = sizeof ((struct ib_sa_service_rec *) 0)->field, \
.struct_size_bytes = sizeof_field(struct ib_sa_service_rec, field), \
.field_name = "sa_service_rec:" #field
static const struct ib_field service_rec_table[] = {
@ -552,7 +552,7 @@ static const struct ib_field service_rec_table[] = {
#define CLASSPORTINFO_REC_FIELD(field) \
.struct_offset_bytes = offsetof(struct ib_class_port_info, field), \
.struct_size_bytes = sizeof((struct ib_class_port_info *)0)->field, \
.struct_size_bytes = sizeof_field(struct ib_class_port_info, field), \
.field_name = "ib_class_port_info:" #field
static const struct ib_field ib_classport_info_rec_table[] = {
@ -630,7 +630,7 @@ static const struct ib_field ib_classport_info_rec_table[] = {
.struct_offset_bytes =\
offsetof(struct opa_class_port_info, field), \
.struct_size_bytes = \
sizeof((struct opa_class_port_info *)0)->field, \
sizeof_field(struct opa_class_port_info, field), \
.field_name = "opa_class_port_info:" #field
static const struct ib_field opa_classport_info_rec_table[] = {
@ -710,7 +710,7 @@ static const struct ib_field opa_classport_info_rec_table[] = {
#define GUIDINFO_REC_FIELD(field) \
.struct_offset_bytes = offsetof(struct ib_sa_guidinfo_rec, field), \
.struct_size_bytes = sizeof((struct ib_sa_guidinfo_rec *) 0)->field, \
.struct_size_bytes = sizeof_field(struct ib_sa_guidinfo_rec, field), \
.field_name = "sa_guidinfo_rec:" #field
static const struct ib_field guidinfo_rec_table[] = {
@ -1412,17 +1412,13 @@ void ib_sa_pack_path(struct sa_path_rec *rec, void *attribute)
EXPORT_SYMBOL(ib_sa_pack_path);
static bool ib_sa_opa_pathrecord_support(struct ib_sa_client *client,
struct ib_device *device,
struct ib_sa_device *sa_dev,
u8 port_num)
{
struct ib_sa_device *sa_dev = ib_get_client_data(device, &sa_client);
struct ib_sa_port *port;
unsigned long flags;
bool ret = false;
if (!sa_dev)
return ret;
port = &sa_dev->port[port_num - sa_dev->start_port];
spin_lock_irqsave(&port->classport_lock, flags);
if (!port->classport_info.valid)
@ -1450,8 +1446,8 @@ enum opa_pr_supported {
* query is possible.
*/
static int opa_pr_query_possible(struct ib_sa_client *client,
struct ib_device *device,
u8 port_num,
struct ib_sa_device *sa_dev,
struct ib_device *device, u8 port_num,
struct sa_path_rec *rec)
{
struct ib_port_attr port_attr;
@ -1459,7 +1455,7 @@ static int opa_pr_query_possible(struct ib_sa_client *client,
if (ib_query_port(device, port_num, &port_attr))
return PR_NOT_SUPPORTED;
if (ib_sa_opa_pathrecord_support(client, device, port_num))
if (ib_sa_opa_pathrecord_support(client, sa_dev, port_num))
return PR_OPA_SUPPORTED;
if (port_attr.lid >= be16_to_cpu(IB_MULTICAST_LID_BASE))
@ -1574,7 +1570,8 @@ int ib_sa_path_rec_get(struct ib_sa_client *client,
query->sa_query.port = port;
if (rec->rec_type == SA_PATH_REC_TYPE_OPA) {
status = opa_pr_query_possible(client, device, port_num, rec);
status = opa_pr_query_possible(client, sa_dev, device, port_num,
rec);
if (status == PR_NOT_SUPPORTED) {
ret = -EINVAL;
goto err1;
@ -2325,18 +2322,19 @@ static void ib_sa_event(struct ib_event_handler *handler,
}
}
static void ib_sa_add_one(struct ib_device *device)
static int ib_sa_add_one(struct ib_device *device)
{
struct ib_sa_device *sa_dev;
int s, e, i;
int count = 0;
int ret;
s = rdma_start_port(device);
e = rdma_end_port(device);
sa_dev = kzalloc(struct_size(sa_dev, port, e - s + 1), GFP_KERNEL);
if (!sa_dev)
return;
return -ENOMEM;
sa_dev->start_port = s;
sa_dev->end_port = e;
@ -2356,8 +2354,10 @@ static void ib_sa_add_one(struct ib_device *device)
ib_register_mad_agent(device, i + s, IB_QPT_GSI,
NULL, 0, send_handler,
recv_handler, sa_dev, 0);
if (IS_ERR(sa_dev->port[i].agent))
if (IS_ERR(sa_dev->port[i].agent)) {
ret = PTR_ERR(sa_dev->port[i].agent);
goto err;
}
INIT_WORK(&sa_dev->port[i].update_task, update_sm_ah);
INIT_DELAYED_WORK(&sa_dev->port[i].ib_cpi_work,
@ -2366,8 +2366,10 @@ static void ib_sa_add_one(struct ib_device *device)
count++;
}
if (!count)
if (!count) {
ret = -EOPNOTSUPP;
goto free;
}
ib_set_client_data(device, &sa_client, sa_dev);
@ -2386,7 +2388,7 @@ static void ib_sa_add_one(struct ib_device *device)
update_sm_ah(&sa_dev->port[i].update_task);
}
return;
return 0;
err:
while (--i >= 0) {
@ -2395,7 +2397,7 @@ static void ib_sa_add_one(struct ib_device *device)
}
free:
kfree(sa_dev);
return;
return ret;
}
static void ib_sa_remove_one(struct ib_device *device, void *client_data)
@ -2403,9 +2405,6 @@ static void ib_sa_remove_one(struct ib_device *device, void *client_data)
struct ib_sa_device *sa_dev = client_data;
int i;
if (!sa_dev)
return;
ib_unregister_event_handler(&sa_dev->event_handler);
flush_workqueue(ib_wq);

10
drivers/infiniband/core/sysfs.c
View File

@ -1058,8 +1058,7 @@ static int add_port(struct ib_core_device *coredev, int port_num)
coredev->ports_kobj,
"%d", port_num);
if (ret) {
kfree(p);
return ret;
goto err_put;
}
p->gid_attr_group = kzalloc(sizeof(*p->gid_attr_group), GFP_KERNEL);
@ -1072,8 +1071,7 @@ static int add_port(struct ib_core_device *coredev, int port_num)
ret = kobject_init_and_add(&p->gid_attr_group->kobj, &gid_attr_type,
&p->kobj, "gid_attrs");
if (ret) {
kfree(p->gid_attr_group);
goto err_put;
goto err_put_gid_attrs;
}
if (device->ops.process_mad && is_full_dev) {
@ -1404,8 +1402,10 @@ int ib_port_register_module_stat(struct ib_device *device, u8 port_num,
ret = kobject_init_and_add(kobj, ktype, &port->kobj, "%s",
name);
if (ret)
if (ret) {
kobject_put(kobj);
return ret;
}
}
return 0;

65
drivers/infiniband/core/ucma.c
View File

@ -52,6 +52,7 @@
#include <rdma/rdma_cm_ib.h>
#include <rdma/ib_addr.h>
#include <rdma/ib.h>
#include <rdma/ib_cm.h>
#include <rdma/rdma_netlink.h>
#include "core_priv.h"
@ -360,6 +361,9 @@ static int ucma_event_handler(struct rdma_cm_id *cm_id,
ucma_copy_conn_event(&uevent->resp.param.conn,
&event->param.conn);
uevent->resp.ece.vendor_id = event->ece.vendor_id;
uevent->resp.ece.attr_mod = event->ece.attr_mod;
if (event->event == RDMA_CM_EVENT_CONNECT_REQUEST) {
if (!ctx->backlog) {
ret = -ENOMEM;
@ -404,7 +408,8 @@ static ssize_t ucma_get_event(struct ucma_file *file, const char __user *inbuf,
* Old 32 bit user space does not send the 4 byte padding in the
* reserved field. We don't care, allow it to keep working.
*/
if (out_len < sizeof(uevent->resp) - sizeof(uevent->resp.reserved))
if (out_len < sizeof(uevent->resp) - sizeof(uevent->resp.reserved) -
sizeof(uevent->resp.ece))
return -ENOSPC;
if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
@ -845,7 +850,7 @@ static ssize_t ucma_query_route(struct ucma_file *file,
struct sockaddr *addr;
int ret = 0;
if (out_len < sizeof(resp))
if (out_len < offsetof(struct rdma_ucm_query_route_resp, ibdev_index))
return -ENOSPC;
if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
@ -869,6 +874,7 @@ static ssize_t ucma_query_route(struct ucma_file *file,
goto out;
resp.node_guid = (__force __u64) ctx->cm_id->device->node_guid;
resp.ibdev_index = ctx->cm_id->device->index;
resp.port_num = ctx->cm_id->port_num;
if (rdma_cap_ib_sa(ctx->cm_id->device, ctx->cm_id->port_num))
@ -880,8 +886,8 @@ static ssize_t ucma_query_route(struct ucma_file *file,
out:
mutex_unlock(&ctx->mutex);
if (copy_to_user(u64_to_user_ptr(cmd.response),
&resp, sizeof(resp)))
if (copy_to_user(u64_to_user_ptr(cmd.response), &resp,
min_t(size_t, out_len, sizeof(resp))))
ret = -EFAULT;
ucma_put_ctx(ctx);
@ -895,6 +901,7 @@ static void ucma_query_device_addr(struct rdma_cm_id *cm_id,
return;
resp->node_guid = (__force __u64) cm_id->device->node_guid;
resp->ibdev_index = cm_id->device->index;
resp->port_num = cm_id->port_num;
resp->pkey = (__force __u16) cpu_to_be16(
ib_addr_get_pkey(&cm_id->route.addr.dev_addr));
@ -907,7 +914,7 @@ static ssize_t ucma_query_addr(struct ucma_context *ctx,
struct sockaddr *addr;
int ret = 0;
if (out_len < sizeof(resp))
if (out_len < offsetof(struct rdma_ucm_query_addr_resp, ibdev_index))
return -ENOSPC;
memset(&resp, 0, sizeof resp);
@ -922,7 +929,7 @@ static ssize_t ucma_query_addr(struct ucma_context *ctx,
ucma_query_device_addr(ctx->cm_id, &resp);
if (copy_to_user(response, &resp, sizeof(resp)))
if (copy_to_user(response, &resp, min_t(size_t, out_len, sizeof(resp))))
ret = -EFAULT;
return ret;
@ -974,7 +981,7 @@ static ssize_t ucma_query_gid(struct ucma_context *ctx,
struct sockaddr_ib *addr;
int ret = 0;
if (out_len < sizeof(resp))
if (out_len < offsetof(struct rdma_ucm_query_addr_resp, ibdev_index))
return -ENOSPC;
memset(&resp, 0, sizeof resp);
@ -1007,7 +1014,7 @@ static ssize_t ucma_query_gid(struct ucma_context *ctx,
&ctx->cm_id->route.addr.dst_addr);
}
if (copy_to_user(response, &resp, sizeof(resp)))
if (copy_to_user(response, &resp, min_t(size_t, out_len, sizeof(resp))))
ret = -EFAULT;
return ret;
@ -1070,12 +1077,15 @@ static void ucma_copy_conn_param(struct rdma_cm_id *id,
static ssize_t ucma_connect(struct ucma_file *file, const char __user *inbuf,
int in_len, int out_len)
{
struct rdma_ucm_connect cmd;
struct rdma_conn_param conn_param;
struct rdma_ucm_ece ece = {};
struct rdma_ucm_connect cmd;
struct ucma_context *ctx;
size_t in_size;
int ret;
if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
in_size = min_t(size_t, in_len, sizeof(cmd));
if (copy_from_user(&cmd, inbuf, in_size))
return -EFAULT;
if (!cmd.conn_param.valid)
@ -1086,8 +1096,13 @@ static ssize_t ucma_connect(struct ucma_file *file, const char __user *inbuf,
return PTR_ERR(ctx);
ucma_copy_conn_param(ctx->cm_id, &conn_param, &cmd.conn_param);
if (offsetofend(typeof(cmd), ece) <= in_size) {
ece.vendor_id = cmd.ece.vendor_id;
ece.attr_mod = cmd.ece.attr_mod;
}
mutex_lock(&ctx->mutex);
ret = rdma_connect(ctx->cm_id, &conn_param);
ret = rdma_connect_ece(ctx->cm_id, &conn_param, &ece);
mutex_unlock(&ctx->mutex);
ucma_put_ctx(ctx);
return ret;
@ -1121,28 +1136,36 @@ static ssize_t ucma_accept(struct ucma_file *file, const char __user *inbuf,
{
struct rdma_ucm_accept cmd;
struct rdma_conn_param conn_param;
struct rdma_ucm_ece ece = {};
struct ucma_context *ctx;
size_t in_size;
int ret;
if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
in_size = min_t(size_t, in_len, sizeof(cmd));
if (copy_from_user(&cmd, inbuf, in_size))
return -EFAULT;
ctx = ucma_get_ctx_dev(file, cmd.id);
if (IS_ERR(ctx))
return PTR_ERR(ctx);
if (offsetofend(typeof(cmd), ece) <= in_size) {
ece.vendor_id = cmd.ece.vendor_id;
ece.attr_mod = cmd.ece.attr_mod;
}
if (cmd.conn_param.valid) {
ucma_copy_conn_param(ctx->cm_id, &conn_param, &cmd.conn_param);
mutex_lock(&file->mut);
mutex_lock(&ctx->mutex);
ret = __rdma_accept(ctx->cm_id, &conn_param, NULL);
ret = __rdma_accept_ece(ctx->cm_id, &conn_param, NULL, &ece);
mutex_unlock(&ctx->mutex);
if (!ret)
ctx->uid = cmd.uid;
mutex_unlock(&file->mut);
} else {
mutex_lock(&ctx->mutex);
ret = __rdma_accept(ctx->cm_id, NULL, NULL);
ret = __rdma_accept_ece(ctx->cm_id, NULL, NULL, &ece);
mutex_unlock(&ctx->mutex);
}
ucma_put_ctx(ctx);
@ -1159,12 +1182,24 @@ static ssize_t ucma_reject(struct ucma_file *file, const char __user *inbuf,
if (copy_from_user(&cmd, inbuf, sizeof(cmd)))
return -EFAULT;
if (!cmd.reason)
cmd.reason = IB_CM_REJ_CONSUMER_DEFINED;
switch (cmd.reason) {
case IB_CM_REJ_CONSUMER_DEFINED:
case IB_CM_REJ_VENDOR_OPTION_NOT_SUPPORTED:
break;
default:
return -EINVAL;
}
ctx = ucma_get_ctx_dev(file, cmd.id);
if (IS_ERR(ctx))
return PTR_ERR(ctx);
mutex_lock(&ctx->mutex);
ret = rdma_reject(ctx->cm_id, cmd.private_data, cmd.private_data_len);
ret = rdma_reject(ctx->cm_id, cmd.private_data, cmd.private_data_len,
cmd.reason);
mutex_unlock(&ctx->mutex);
ucma_put_ctx(ctx);
return ret;

2
drivers/infiniband/core/ud_header.c
View File

@ -41,7 +41,7 @@
#define STRUCT_FIELD(header, field) \
.struct_offset_bytes = offsetof(struct ib_unpacked_ ## header, field), \
.struct_size_bytes = sizeof ((struct ib_unpacked_ ## header *) 0)->field, \
.struct_size_bytes = sizeof_field(struct ib_unpacked_ ## header, field), \
.field_name = #header ":" #field
static const struct ib_field lrh_table[] = {

22
drivers/infiniband/core/user_mad.c
View File

@ -142,7 +142,7 @@ static dev_t dynamic_issm_dev;
static DEFINE_IDA(umad_ida);
static void ib_umad_add_one(struct ib_device *device);
static int ib_umad_add_one(struct ib_device *device);
static void ib_umad_remove_one(struct ib_device *device, void *client_data);
static void ib_umad_dev_free(struct kref *kref)
@ -1352,37 +1352,41 @@ static void ib_umad_kill_port(struct ib_umad_port *port)
put_device(&port->dev);
}
static void ib_umad_add_one(struct ib_device *device)
static int ib_umad_add_one(struct ib_device *device)
{
struct ib_umad_device *umad_dev;
int s, e, i;
int count = 0;
int ret;
s = rdma_start_port(device);
e = rdma_end_port(device);
umad_dev = kzalloc(struct_size(umad_dev, ports, e - s + 1), GFP_KERNEL);
if (!umad_dev)
return;
return -ENOMEM;
kref_init(&umad_dev->kref);
for (i = s; i <= e; ++i) {
if (!rdma_cap_ib_mad(device, i))
continue;
if (ib_umad_init_port(device, i, umad_dev,
&umad_dev->ports[i - s]))
ret = ib_umad_init_port(device, i, umad_dev,
&umad_dev->ports[i - s]);
if (ret)
goto err;
count++;
}
if (!count)
if (!count) {
ret = -EOPNOTSUPP;
goto free;
}
ib_set_client_data(device, &umad_client, umad_dev);
return;
return 0;
err:
while (--i >= s) {
@ -1394,6 +1398,7 @@ static void ib_umad_add_one(struct ib_device *device)
free:
/* balances kref_init */
ib_umad_dev_put(umad_dev);
return ret;
}
static void ib_umad_remove_one(struct ib_device *device, void *client_data)
@ -1401,9 +1406,6 @@ static void ib_umad_remove_one(struct ib_device *device, void *client_data)
struct ib_umad_device *umad_dev = client_data;
unsigned int i;
if (!umad_dev)
return;
rdma_for_each_port (device, i) {
if (rdma_cap_ib_mad(device, i))
ib_umad_kill_port(

21
drivers/infiniband/core/uverbs.h
View File

@ -142,7 +142,7 @@ struct ib_uverbs_file {
* ucontext_lock held
*/
struct ib_ucontext *ucontext;
struct ib_uverbs_async_event_file *async_file;
struct ib_uverbs_async_event_file *default_async_file;
struct list_head list;
/*
@ -180,6 +180,7 @@ struct ib_uverbs_mcast_entry {
struct ib_uevent_object {
struct ib_uobject uobject;
struct ib_uverbs_async_event_file *event_file;
/* List member for ib_uverbs_async_event_file list */
struct list_head event_list;
u32 events_reported;
@ -296,6 +297,24 @@ static inline u32 make_port_cap_flags(const struct ib_port_attr *attr)
return res;
}
static inline struct ib_uverbs_async_event_file *
ib_uverbs_get_async_event(struct uverbs_attr_bundle *attrs,
u16 id)
{
struct ib_uobject *async_ev_file_uobj;
struct ib_uverbs_async_event_file *async_ev_file;
async_ev_file_uobj = uverbs_attr_get_uobject(attrs, id);
if (IS_ERR(async_ev_file_uobj))
async_ev_file = READ_ONCE(attrs->ufile->default_async_file);
else
async_ev_file = container_of(async_ev_file_uobj,
struct ib_uverbs_async_event_file,
uobj);
if (async_ev_file)
uverbs_uobject_get(&async_ev_file->uobj);
return async_ev_file;
}
void copy_port_attr_to_resp(struct ib_port_attr *attr,
struct ib_uverbs_query_port_resp *resp,

76
drivers/infiniband/core/uverbs_cmd.c
View File

@ -311,7 +311,7 @@ static int ib_uverbs_get_context(struct uverbs_attr_bundle *attrs)
return 0;
err_uobj:
rdma_alloc_abort_uobject(uobj, attrs);
rdma_alloc_abort_uobject(uobj, attrs, false);
err_ucontext:
kfree(attrs->context);
attrs->context = NULL;
@ -356,8 +356,6 @@ static void copy_query_dev_fields(struct ib_ucontext *ucontext,
resp->max_mcast_qp_attach = attr->max_mcast_qp_attach;
resp->max_total_mcast_qp_attach = attr->max_total_mcast_qp_attach;
resp->max_ah = attr->max_ah;
resp->max_fmr = attr->max_fmr;
resp->max_map_per_fmr = attr->max_map_per_fmr;
resp->max_srq = attr->max_srq;
resp->max_srq_wr = attr->max_srq_wr;
resp->max_srq_sge = attr->max_srq_sge;
@ -1051,6 +1049,10 @@ static struct ib_ucq_object *create_cq(struct uverbs_attr_bundle *attrs,
goto err_free;
obj->uevent.uobject.object = cq;
obj->uevent.event_file = READ_ONCE(attrs->ufile->default_async_file);
if (obj->uevent.event_file)
uverbs_uobject_get(&obj->uevent.event_file->uobj);
memset(&resp, 0, sizeof resp);
resp.base.cq_handle = obj->uevent.uobject.id;
resp.base.cqe = cq->cqe;
@ -1067,6 +1069,8 @@ static struct ib_ucq_object *create_cq(struct uverbs_attr_bundle *attrs,
return obj;
err_cb:
if (obj->uevent.event_file)
uverbs_uobject_put(&obj->uevent.event_file->uobj);
ib_destroy_cq_user(cq, uverbs_get_cleared_udata(attrs));
cq = NULL;
err_free:
@ -1460,6 +1464,9 @@ static int create_qp(struct uverbs_attr_bundle *attrs,
}
obj->uevent.uobject.object = qp;
obj->uevent.event_file = READ_ONCE(attrs->ufile->default_async_file);
if (obj->uevent.event_file)
uverbs_uobject_get(&obj->uevent.event_file->uobj);
memset(&resp, 0, sizeof resp);
resp.base.qpn = qp->qp_num;
@ -1473,7 +1480,7 @@ static int create_qp(struct uverbs_attr_bundle *attrs,
ret = uverbs_response(attrs, &resp, sizeof(resp));
if (ret)
goto err_cb;
goto err_uevent;
if (xrcd) {
obj->uxrcd = container_of(xrcd_uobj, struct ib_uxrcd_object,
@ -1498,6 +1505,9 @@ static int create_qp(struct uverbs_attr_bundle *attrs,
rdma_alloc_commit_uobject(&obj->uevent.uobject, attrs);
return 0;
err_uevent:
if (obj->uevent.event_file)
uverbs_uobject_put(&obj->uevent.event_file->uobj);
err_cb:
ib_destroy_qp_user(qp, uverbs_get_cleared_udata(attrs));
@ -2954,11 +2964,11 @@ static int ib_uverbs_ex_create_wq(struct uverbs_attr_bundle *attrs)
wq_init_attr.cq = cq;
wq_init_attr.max_sge = cmd.max_sge;
wq_init_attr.max_wr = cmd.max_wr;
wq_init_attr.wq_context = attrs->ufile;
wq_init_attr.wq_type = cmd.wq_type;
wq_init_attr.event_handler = ib_uverbs_wq_event_handler;
wq_init_attr.create_flags = cmd.create_flags;
INIT_LIST_HEAD(&obj->uevent.event_list);
obj->uevent.uobject.user_handle = cmd.user_handle;
wq = pd->device->ops.create_wq(pd, &wq_init_attr, &attrs->driver_udata);
if (IS_ERR(wq)) {
@ -2972,12 +2982,12 @@ static int ib_uverbs_ex_create_wq(struct uverbs_attr_bundle *attrs)
wq->cq = cq;
wq->pd = pd;
wq->device = pd->device;
wq->wq_context = wq_init_attr.wq_context;
atomic_set(&wq->usecnt, 0);
atomic_inc(&pd->usecnt);
atomic_inc(&cq->usecnt);
wq->uobject = obj;
obj->uevent.uobject.object = wq;
obj->uevent.event_file = READ_ONCE(attrs->ufile->default_async_file);
if (obj->uevent.event_file)
uverbs_uobject_get(&obj->uevent.event_file->uobj);
memset(&resp, 0, sizeof(resp));
resp.wq_handle = obj->uevent.uobject.id;
@ -2996,6 +3006,8 @@ static int ib_uverbs_ex_create_wq(struct uverbs_attr_bundle *attrs)
return 0;
err_copy:
if (obj->uevent.event_file)
uverbs_uobject_put(&obj->uevent.event_file->uobj);
ib_destroy_wq(wq, uverbs_get_cleared_udata(attrs));
err_put_cq:
rdma_lookup_put_uobject(&cq->uobject->uevent.uobject,
@ -3441,46 +3453,25 @@ static int __uverbs_create_xsrq(struct uverbs_attr_bundle *attrs,
}
attr.event_handler = ib_uverbs_srq_event_handler;
attr.srq_context = attrs->ufile;
attr.srq_type = cmd->srq_type;
attr.attr.max_wr = cmd->max_wr;
attr.attr.max_sge = cmd->max_sge;
attr.attr.srq_limit = cmd->srq_limit;
INIT_LIST_HEAD(&obj->uevent.event_list);
obj->uevent.uobject.user_handle = cmd->user_handle;
srq = rdma_zalloc_drv_obj(ib_dev, ib_srq);
if (!srq) {
ret = -ENOMEM;
goto err_put;
srq = ib_create_srq_user(pd, &attr, obj, udata);
if (IS_ERR(srq)) {
ret = PTR_ERR(srq);
goto err_put_pd;
}
srq->device = pd->device;
srq->pd = pd;
srq->srq_type = cmd->srq_type;
srq->uobject = obj;
srq->event_handler = attr.event_handler;
srq->srq_context = attr.srq_context;
ret = pd->device->ops.create_srq(srq, &attr, udata);
if (ret)
goto err_free;
if (ib_srq_has_cq(cmd->srq_type)) {
srq->ext.cq = attr.ext.cq;
atomic_inc(&attr.ext.cq->usecnt);
}
if (cmd->srq_type == IB_SRQT_XRC) {
srq->ext.xrc.xrcd = attr.ext.xrc.xrcd;
atomic_inc(&attr.ext.xrc.xrcd->usecnt);
}
atomic_inc(&pd->usecnt);
atomic_set(&srq->usecnt, 0);
obj->uevent.uobject.object = srq;
obj->uevent.uobject.user_handle = cmd->user_handle;
obj->uevent.event_file = READ_ONCE(attrs->ufile->default_async_file);
if (obj->uevent.event_file)
uverbs_uobject_get(&obj->uevent.event_file->uobj);
memset(&resp, 0, sizeof resp);
resp.srq_handle = obj->uevent.uobject.id;
@ -3505,14 +3496,11 @@ static int __uverbs_create_xsrq(struct uverbs_attr_bundle *attrs,
return 0;
err_copy:
if (obj->uevent.event_file)
uverbs_uobject_put(&obj->uevent.event_file->uobj);
ib_destroy_srq_user(srq, uverbs_get_cleared_udata(attrs));
/* It was released in ib_destroy_srq_user */
srq = NULL;
err_free:
kfree(srq);
err_put:
err_put_pd:
uobj_put_obj_read(pd);
err_put_cq:
if (ib_srq_has_cq(cmd->srq_type))
rdma_lookup_put_uobject(&attr.ext.cq->uobject->uevent.uobject,
@ -3751,7 +3739,7 @@ static int ib_uverbs_ex_modify_cq(struct uverbs_attr_bundle *attrs)
#define UAPI_DEF_WRITE_IO(req, resp) \
.write.has_resp = 1 + \
BUILD_BUG_ON_ZERO(offsetof(req, response) != 0) + \
BUILD_BUG_ON_ZERO(sizeof(((req *)0)->response) != \
BUILD_BUG_ON_ZERO(sizeof_field(req, response) != \
sizeof(u64)), \
.write.req_size = sizeof(req), .write.resp_size = sizeof(resp)

24
drivers/infiniband/core/uverbs_ioctl.c
View File

@ -58,6 +58,7 @@ struct bundle_priv {
DECLARE_BITMAP(uobj_finalize, UVERBS_API_ATTR_BKEY_LEN);
DECLARE_BITMAP(spec_finalize, UVERBS_API_ATTR_BKEY_LEN);
DECLARE_BITMAP(uobj_hw_obj_valid, UVERBS_API_ATTR_BKEY_LEN);
/*
* Must be last. bundle ends in a flex array which overlaps
@ -136,7 +137,7 @@ EXPORT_SYMBOL(_uverbs_alloc);
static bool uverbs_is_attr_cleared(const struct ib_uverbs_attr *uattr,
u16 len)
{
if (uattr->len > sizeof(((struct ib_uverbs_attr *)0)->data))
if (uattr->len > sizeof_field(struct ib_uverbs_attr, data))
return ib_is_buffer_cleared(u64_to_user_ptr(uattr->data) + len,
uattr->len - len);
@ -230,7 +231,8 @@ static void uverbs_free_idrs_array(const struct uverbs_api_attr *attr_uapi,
for (i = 0; i != attr->len; i++)
uverbs_finalize_object(attr->uobjects[i],
spec->u2.objs_arr.access, commit, attrs);
spec->u2.objs_arr.access, false, commit,
attrs);
}
static int uverbs_process_attr(struct bundle_priv *pbundle,
@ -502,7 +504,9 @@ static void bundle_destroy(struct bundle_priv *pbundle, bool commit)
uverbs_finalize_object(
attr->obj_attr.uobject,
attr->obj_attr.attr_elm->spec.u.obj.access, commit,
attr->obj_attr.attr_elm->spec.u.obj.access,
test_bit(i, pbundle->uobj_hw_obj_valid),
commit,
&pbundle->bundle);
}
@ -590,6 +594,8 @@ static int ib_uverbs_cmd_verbs(struct ib_uverbs_file *ufile,
sizeof(pbundle->bundle.attr_present));
memset(pbundle->uobj_finalize, 0, sizeof(pbundle->uobj_finalize));
memset(pbundle->spec_finalize, 0, sizeof(pbundle->spec_finalize));
memset(pbundle->uobj_hw_obj_valid, 0,
sizeof(pbundle->uobj_hw_obj_valid));
ret = ib_uverbs_run_method(pbundle, hdr->num_attrs);
bundle_destroy(pbundle, ret == 0);
@ -784,3 +790,15 @@ int uverbs_copy_to_struct_or_zero(const struct uverbs_attr_bundle *bundle,
}
return uverbs_copy_to(bundle, idx, from, size);
}
/* Once called an abort will call through to the type's destroy_hw() */
void uverbs_finalize_uobj_create(const struct uverbs_attr_bundle *bundle,
u16 idx)
{
struct bundle_priv *pbundle =
container_of(bundle, struct bundle_priv, bundle);
__set_bit(uapi_bkey_attr(uapi_key_attr(idx)),
pbundle->uobj_hw_obj_valid);
}
EXPORT_SYMBOL(uverbs_finalize_uobj_create);

40
drivers/infiniband/core/uverbs_main.c
View File

@ -75,7 +75,7 @@ static dev_t dynamic_uverbs_dev;
static struct class *uverbs_class;
static DEFINE_IDA(uverbs_ida);
static void ib_uverbs_add_one(struct ib_device *device);
static int ib_uverbs_add_one(struct ib_device *device);
static void ib_uverbs_remove_one(struct ib_device *device, void *client_data);
/*
@ -146,8 +146,7 @@ void ib_uverbs_release_ucq(struct ib_uverbs_completion_event_file *ev_file,
void ib_uverbs_release_uevent(struct ib_uevent_object *uobj)
{
struct ib_uverbs_async_event_file *async_file =
READ_ONCE(uobj->uobject.ufile->async_file);
struct ib_uverbs_async_event_file *async_file = uobj->event_file;
struct ib_uverbs_event *evt, *tmp;
if (!async_file)
@ -159,6 +158,7 @@ void ib_uverbs_release_uevent(struct ib_uevent_object *uobj)
kfree(evt);
}
spin_unlock_irq(&async_file->ev_queue.lock);
uverbs_uobject_put(&async_file->uobj);
}
void ib_uverbs_detach_umcast(struct ib_qp *qp,
@ -197,8 +197,8 @@ void ib_uverbs_release_file(struct kref *ref)
if (atomic_dec_and_test(&file->device->refcount))
ib_uverbs_comp_dev(file->device);
if (file->async_file)
uverbs_uobject_put(&file->async_file->uobj);
if (file->default_async_file)
uverbs_uobject_put(&file->default_async_file->uobj);
put_device(&file->device->dev);
if (file->disassociate_page)
@ -296,6 +296,8 @@ static __poll_t ib_uverbs_event_poll(struct ib_uverbs_event_queue *ev_queue,
spin_lock_irq(&ev_queue->lock);
if (!list_empty(&ev_queue->event_list))
pollflags = EPOLLIN | EPOLLRDNORM;
else if (ev_queue->is_closed)
pollflags = EPOLLERR;
spin_unlock_irq(&ev_queue->lock);
return pollflags;
@ -425,7 +427,7 @@ void ib_uverbs_async_handler(struct ib_uverbs_async_event_file *async_file,
static void uverbs_uobj_event(struct ib_uevent_object *eobj,
struct ib_event *event)
{
ib_uverbs_async_handler(READ_ONCE(eobj->uobject.ufile->async_file),
ib_uverbs_async_handler(eobj->event_file,
eobj->uobject.user_handle, event->event,
&eobj->event_list, &eobj->events_reported);
}
@ -482,10 +484,10 @@ void ib_uverbs_init_async_event_file(
/* The first async_event_file becomes the default one for the file. */
mutex_lock(&uverbs_file->ucontext_lock);
if (!uverbs_file->async_file) {
if (!uverbs_file->default_async_file) {
/* Pairs with the put in ib_uverbs_release_file */
uverbs_uobject_get(&async_file->uobj);
smp_store_release(&uverbs_file->async_file, async_file);
smp_store_release(&uverbs_file->default_async_file, async_file);
}
mutex_unlock(&uverbs_file->ucontext_lock);
@ -1092,7 +1094,7 @@ static int ib_uverbs_create_uapi(struct ib_device *device,
return 0;
}
static void ib_uverbs_add_one(struct ib_device *device)
static int ib_uverbs_add_one(struct ib_device *device)
{
int devnum;
dev_t base;
@ -1100,16 +1102,16 @@ static void ib_uverbs_add_one(struct ib_device *device)
int ret;
if (!device->ops.alloc_ucontext)
return;
return -EOPNOTSUPP;
uverbs_dev = kzalloc(sizeof(*uverbs_dev), GFP_KERNEL);
if (!uverbs_dev)
return;
return -ENOMEM;
ret = init_srcu_struct(&uverbs_dev->disassociate_srcu);
if (ret) {
kfree(uverbs_dev);
return;
return -ENOMEM;
}
device_initialize(&uverbs_dev->dev);
@ -1129,15 +1131,18 @@ static void ib_uverbs_add_one(struct ib_device *device)
devnum = ida_alloc_max(&uverbs_ida, IB_UVERBS_MAX_DEVICES - 1,
GFP_KERNEL);
if (devnum < 0)
if (devnum < 0) {
ret = -ENOMEM;
goto err;
}
uverbs_dev->devnum = devnum;
if (devnum >= IB_UVERBS_NUM_FIXED_MINOR)
base = dynamic_uverbs_dev + devnum - IB_UVERBS_NUM_FIXED_MINOR;
else
base = IB_UVERBS_BASE_DEV + devnum;
if (ib_uverbs_create_uapi(device, uverbs_dev))
ret = ib_uverbs_create_uapi(device, uverbs_dev);
if (ret)
goto err_uapi;
uverbs_dev->dev.devt = base;
@ -1152,7 +1157,7 @@ static void ib_uverbs_add_one(struct ib_device *device)
goto err_uapi;
ib_set_client_data(device, &uverbs_client, uverbs_dev);
return;
return 0;
err_uapi:
ida_free(&uverbs_ida, devnum);
@ -1161,7 +1166,7 @@ static void ib_uverbs_add_one(struct ib_device *device)
ib_uverbs_comp_dev(uverbs_dev);
wait_for_completion(&uverbs_dev->comp);
put_device(&uverbs_dev->dev);
return;
return ret;
}
static void ib_uverbs_free_hw_resources(struct ib_uverbs_device *uverbs_dev,
@ -1201,9 +1206,6 @@ static void ib_uverbs_remove_one(struct ib_device *device, void *client_data)
struct ib_uverbs_device *uverbs_dev = client_data;
int wait_clients = 1;
if (!uverbs_dev)
return;
cdev_device_del(&uverbs_dev->cdev, &uverbs_dev->dev);
ida_free(&uverbs_ida, uverbs_dev->devnum);

95
drivers/infiniband/core/uverbs_std_types.c
View File

@ -75,40 +75,6 @@ static int uverbs_free_mw(struct ib_uobject *uobject,
return uverbs_dealloc_mw((struct ib_mw *)uobject->object);
}
static int uverbs_free_qp(struct ib_uobject *uobject,
enum rdma_remove_reason why,
struct uverbs_attr_bundle *attrs)
{
struct ib_qp *qp = uobject->object;
struct ib_uqp_object *uqp =
container_of(uobject, struct ib_uqp_object, uevent.uobject);
int ret;
/*
* If this is a user triggered destroy then do not allow destruction
* until the user cleans up all the mcast bindings. Unlike in other
* places we forcibly clean up the mcast attachments for !DESTROY
* because the mcast attaches are not ubojects and will not be
* destroyed by anything else during cleanup processing.
*/
if (why == RDMA_REMOVE_DESTROY) {
if (!list_empty(&uqp->mcast_list))
return -EBUSY;
} else if (qp == qp->real_qp) {
ib_uverbs_detach_umcast(qp, uqp);
}
ret = ib_destroy_qp_user(qp, &attrs->driver_udata);
if (ib_is_destroy_retryable(ret, why, uobject))
return ret;
if (uqp->uxrcd)
atomic_dec(&uqp->uxrcd->refcnt);
ib_uverbs_release_uevent(&uqp->uevent);
return ret;
}
static int uverbs_free_rwq_ind_tbl(struct ib_uobject *uobject,
enum rdma_remove_reason why,
struct uverbs_attr_bundle *attrs)
@ -125,48 +91,6 @@ static int uverbs_free_rwq_ind_tbl(struct ib_uobject *uobject,
return ret;
}
static int uverbs_free_wq(struct ib_uobject *uobject,
enum rdma_remove_reason why,
struct uverbs_attr_bundle *attrs)
{
struct ib_wq *wq = uobject->object;
struct ib_uwq_object *uwq =
container_of(uobject, struct ib_uwq_object, uevent.uobject);
int ret;
ret = ib_destroy_wq(wq, &attrs->driver_udata);
if (ib_is_destroy_retryable(ret, why, uobject))
return ret;
ib_uverbs_release_uevent(&uwq->uevent);
return ret;
}
static int uverbs_free_srq(struct ib_uobject *uobject,
enum rdma_remove_reason why,
struct uverbs_attr_bundle *attrs)
{
struct ib_srq *srq = uobject->object;
struct ib_uevent_object *uevent =
container_of(uobject, struct ib_uevent_object, uobject);
enum ib_srq_type srq_type = srq->srq_type;
int ret;
ret = ib_destroy_srq_user(srq, &attrs->driver_udata);
if (ib_is_destroy_retryable(ret, why, uobject))
return ret;
if (srq_type == IB_SRQT_XRC) {
struct ib_usrq_object *us =
container_of(uevent, struct ib_usrq_object, uevent);
atomic_dec(&us->uxrcd->refcnt);
}
ib_uverbs_release_uevent(uevent);
return ret;
}
static int uverbs_free_xrcd(struct ib_uobject *uobject,
enum rdma_remove_reason why,
struct uverbs_attr_bundle *attrs)
@ -252,10 +176,6 @@ DECLARE_UVERBS_NAMED_OBJECT(
"[infinibandevent]",
O_RDONLY));
DECLARE_UVERBS_NAMED_OBJECT(
UVERBS_OBJECT_QP,
UVERBS_TYPE_ALLOC_IDR_SZ(sizeof(struct ib_uqp_object), uverbs_free_qp));
DECLARE_UVERBS_NAMED_METHOD_DESTROY(
UVERBS_METHOD_MW_DESTROY,
UVERBS_ATTR_IDR(UVERBS_ATTR_DESTROY_MW_HANDLE,
@ -267,11 +187,6 @@ DECLARE_UVERBS_NAMED_OBJECT(UVERBS_OBJECT_MW,
UVERBS_TYPE_ALLOC_IDR(uverbs_free_mw),
&UVERBS_METHOD(UVERBS_METHOD_MW_DESTROY));
DECLARE_UVERBS_NAMED_OBJECT(
UVERBS_OBJECT_SRQ,
UVERBS_TYPE_ALLOC_IDR_SZ(sizeof(struct ib_usrq_object),
uverbs_free_srq));
DECLARE_UVERBS_NAMED_METHOD_DESTROY(
UVERBS_METHOD_AH_DESTROY,
UVERBS_ATTR_IDR(UVERBS_ATTR_DESTROY_AH_HANDLE,
@ -296,10 +211,6 @@ DECLARE_UVERBS_NAMED_OBJECT(
uverbs_free_flow),
&UVERBS_METHOD(UVERBS_METHOD_FLOW_DESTROY));
DECLARE_UVERBS_NAMED_OBJECT(
UVERBS_OBJECT_WQ,
UVERBS_TYPE_ALLOC_IDR_SZ(sizeof(struct ib_uwq_object), uverbs_free_wq));
DECLARE_UVERBS_NAMED_METHOD_DESTROY(
UVERBS_METHOD_RWQ_IND_TBL_DESTROY,
UVERBS_ATTR_IDR(UVERBS_ATTR_DESTROY_RWQ_IND_TBL_HANDLE,
@ -340,18 +251,12 @@ const struct uapi_definition uverbs_def_obj_intf[] = {
UAPI_DEF_OBJ_NEEDS_FN(dealloc_pd)),
UAPI_DEF_CHAIN_OBJ_TREE_NAMED(UVERBS_OBJECT_COMP_CHANNEL,
UAPI_DEF_OBJ_NEEDS_FN(dealloc_pd)),
UAPI_DEF_CHAIN_OBJ_TREE_NAMED(UVERBS_OBJECT_QP,
UAPI_DEF_OBJ_NEEDS_FN(destroy_qp)),
UAPI_DEF_CHAIN_OBJ_TREE_NAMED(UVERBS_OBJECT_AH,
UAPI_DEF_OBJ_NEEDS_FN(destroy_ah)),
UAPI_DEF_CHAIN_OBJ_TREE_NAMED(UVERBS_OBJECT_MW,
UAPI_DEF_OBJ_NEEDS_FN(dealloc_mw)),
UAPI_DEF_CHAIN_OBJ_TREE_NAMED(UVERBS_OBJECT_SRQ,
UAPI_DEF_OBJ_NEEDS_FN(destroy_srq)),
UAPI_DEF_CHAIN_OBJ_TREE_NAMED(UVERBS_OBJECT_FLOW,
UAPI_DEF_OBJ_NEEDS_FN(destroy_flow)),
UAPI_DEF_CHAIN_OBJ_TREE_NAMED(UVERBS_OBJECT_WQ,
UAPI_DEF_OBJ_NEEDS_FN(destroy_wq)),
UAPI_DEF_CHAIN_OBJ_TREE_NAMED(
UVERBS_OBJECT_RWQ_IND_TBL,
UAPI_DEF_OBJ_NEEDS_FN(destroy_rwq_ind_table)),

17
drivers/infiniband/core/uverbs_std_types_cq.c
View File

@ -100,6 +100,9 @@ static int UVERBS_HANDLER(UVERBS_METHOD_CQ_CREATE)(
uverbs_uobject_get(ev_file_uobj);
}
obj->uevent.event_file = ib_uverbs_get_async_event(
attrs, UVERBS_ATTR_CREATE_CQ_EVENT_FD);
if (attr.comp_vector >= attrs->ufile->device->num_comp_vectors) {
ret = -EINVAL;
goto err_event_file;
@ -129,19 +132,17 @@ static int UVERBS_HANDLER(UVERBS_METHOD_CQ_CREATE)(
obj->uevent.uobject.object = cq;
obj->uevent.uobject.user_handle = user_handle;
rdma_restrack_uadd(&cq->res);
uverbs_finalize_uobj_create(attrs, UVERBS_ATTR_CREATE_CQ_HANDLE);
ret = uverbs_copy_to(attrs, UVERBS_ATTR_CREATE_CQ_RESP_CQE, &cq->cqe,
sizeof(cq->cqe));
if (ret)
goto err_cq;
return ret;
return 0;
err_cq:
ib_destroy_cq_user(cq, uverbs_get_cleared_udata(attrs));
cq = NULL;
err_free:
kfree(cq);
err_event_file:
if (obj->uevent.event_file)
uverbs_uobject_put(&obj->uevent.event_file->uobj);
if (ev_file)
uverbs_uobject_put(ev_file_uobj);
return ret;
@ -171,6 +172,10 @@ DECLARE_UVERBS_NAMED_METHOD(
UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_CREATE_CQ_RESP_CQE,
UVERBS_ATTR_TYPE(u32),
UA_MANDATORY),
UVERBS_ATTR_FD(UVERBS_ATTR_CREATE_CQ_EVENT_FD,
UVERBS_OBJECT_ASYNC_EVENT,
UVERBS_ACCESS_READ,
UA_OPTIONAL),
UVERBS_ATTR_UHW());
static int UVERBS_HANDLER(UVERBS_METHOD_CQ_DESTROY)(

12
drivers/infiniband/core/uverbs_std_types_mr.c
View File

@ -136,21 +136,15 @@ static int UVERBS_HANDLER(UVERBS_METHOD_DM_MR_REG)(
uobj->object = mr;
uverbs_finalize_uobj_create(attrs, UVERBS_ATTR_REG_DM_MR_HANDLE);
ret = uverbs_copy_to(attrs, UVERBS_ATTR_REG_DM_MR_RESP_LKEY, &mr->lkey,
sizeof(mr->lkey));
if (ret)
goto err_dereg;
return ret;
ret = uverbs_copy_to(attrs, UVERBS_ATTR_REG_DM_MR_RESP_RKEY,
&mr->rkey, sizeof(mr->rkey));
if (ret)
goto err_dereg;
return 0;
err_dereg:
ib_dereg_mr_user(mr, uverbs_get_cleared_udata(attrs));
return ret;
}

401
drivers/infiniband/core/uverbs_std_types_qp.c Normal file
View File

@ -0,0 +1,401 @@
// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
/*
* Copyright (c) 2020, Mellanox Technologies inc. All rights reserved.
*/
#include <rdma/uverbs_std_types.h>
#include "rdma_core.h"
#include "uverbs.h"
#include "core_priv.h"
static int uverbs_free_qp(struct ib_uobject *uobject,
enum rdma_remove_reason why,
struct uverbs_attr_bundle *attrs)
{
struct ib_qp *qp = uobject->object;
struct ib_uqp_object *uqp =
container_of(uobject, struct ib_uqp_object, uevent.uobject);
int ret;
/*
* If this is a user triggered destroy then do not allow destruction
* until the user cleans up all the mcast bindings. Unlike in other
* places we forcibly clean up the mcast attachments for !DESTROY
* because the mcast attaches are not ubojects and will not be
* destroyed by anything else during cleanup processing.
*/
if (why == RDMA_REMOVE_DESTROY) {
if (!list_empty(&uqp->mcast_list))
return -EBUSY;
} else if (qp == qp->real_qp) {
ib_uverbs_detach_umcast(qp, uqp);
}
ret = ib_destroy_qp_user(qp, &attrs->driver_udata);
if (ib_is_destroy_retryable(ret, why, uobject))
return ret;
if (uqp->uxrcd)
atomic_dec(&uqp->uxrcd->refcnt);
ib_uverbs_release_uevent(&uqp->uevent);
return ret;
}
static int check_creation_flags(enum ib_qp_type qp_type,
u32 create_flags)
{
create_flags &= ~IB_UVERBS_QP_CREATE_SQ_SIG_ALL;
if (!create_flags || qp_type == IB_QPT_DRIVER)
return 0;
if (qp_type != IB_QPT_RAW_PACKET && qp_type != IB_QPT_UD)
return -EINVAL;
if ((create_flags & IB_UVERBS_QP_CREATE_SCATTER_FCS ||
create_flags & IB_UVERBS_QP_CREATE_CVLAN_STRIPPING) &&
qp_type != IB_QPT_RAW_PACKET)
return -EINVAL;
return 0;
}
static void set_caps(struct ib_qp_init_attr *attr,
struct ib_uverbs_qp_cap *cap, bool req)
{
if (req) {
attr->cap.max_send_wr = cap->max_send_wr;
attr->cap.max_recv_wr = cap->max_recv_wr;
attr->cap.max_send_sge = cap->max_send_sge;
attr->cap.max_recv_sge = cap->max_recv_sge;
attr->cap.max_inline_data = cap->max_inline_data;
} else {
cap->max_send_wr = attr->cap.max_send_wr;
cap->max_recv_wr = attr->cap.max_recv_wr;
cap->max_send_sge = attr->cap.max_send_sge;
cap->max_recv_sge = attr->cap.max_recv_sge;
cap->max_inline_data = attr->cap.max_inline_data;
}
}
static int UVERBS_HANDLER(UVERBS_METHOD_QP_CREATE)(
struct uverbs_attr_bundle *attrs)
{
struct ib_uqp_object *obj = container_of(
uverbs_attr_get_uobject(attrs, UVERBS_ATTR_CREATE_QP_HANDLE),
typeof(*obj), uevent.uobject);
struct ib_qp_init_attr attr = {};
struct ib_uverbs_qp_cap cap = {};
struct ib_rwq_ind_table *rwq_ind_tbl = NULL;
struct ib_qp *qp;
struct ib_pd *pd = NULL;
struct ib_srq *srq = NULL;
struct ib_cq *recv_cq = NULL;
struct ib_cq *send_cq = NULL;
struct ib_xrcd *xrcd = NULL;
struct ib_uobject *xrcd_uobj = NULL;
struct ib_device *device;
u64 user_handle;
int ret;
ret = uverbs_copy_from_or_zero(&cap, attrs,
UVERBS_ATTR_CREATE_QP_CAP);
if (!ret)
ret = uverbs_copy_from(&user_handle, attrs,
UVERBS_ATTR_CREATE_QP_USER_HANDLE);
if (!ret)
ret = uverbs_get_const(&attr.qp_type, attrs,
UVERBS_ATTR_CREATE_QP_TYPE);
if (ret)
return ret;
switch (attr.qp_type) {
case IB_QPT_XRC_TGT:
if (uverbs_attr_is_valid(attrs,
UVERBS_ATTR_CREATE_QP_RECV_CQ_HANDLE) ||
uverbs_attr_is_valid(attrs,
UVERBS_ATTR_CREATE_QP_SEND_CQ_HANDLE) ||
uverbs_attr_is_valid(attrs,
UVERBS_ATTR_CREATE_QP_PD_HANDLE) ||
uverbs_attr_is_valid(attrs,
UVERBS_ATTR_CREATE_QP_IND_TABLE_HANDLE))
return -EINVAL;
xrcd_uobj = uverbs_attr_get_uobject(attrs,
UVERBS_ATTR_CREATE_QP_XRCD_HANDLE);
if (IS_ERR(xrcd_uobj))
return PTR_ERR(xrcd_uobj);
xrcd = (struct ib_xrcd *)xrcd_uobj->object;
if (!xrcd)
return -EINVAL;
device = xrcd->device;
break;
case IB_UVERBS_QPT_RAW_PACKET:
if (!capable(CAP_NET_RAW))
return -EPERM;
fallthrough;
case IB_UVERBS_QPT_RC:
case IB_UVERBS_QPT_UC:
case IB_UVERBS_QPT_UD:
case IB_UVERBS_QPT_XRC_INI:
case IB_UVERBS_QPT_DRIVER:
if (uverbs_attr_is_valid(attrs,
UVERBS_ATTR_CREATE_QP_XRCD_HANDLE) ||
(uverbs_attr_is_valid(attrs,
UVERBS_ATTR_CREATE_QP_SRQ_HANDLE) &&
attr.qp_type == IB_QPT_XRC_INI))
return -EINVAL;
pd = uverbs_attr_get_obj(attrs,
UVERBS_ATTR_CREATE_QP_PD_HANDLE);
if (IS_ERR(pd))
return PTR_ERR(pd);
rwq_ind_tbl = uverbs_attr_get_obj(attrs,
UVERBS_ATTR_CREATE_QP_IND_TABLE_HANDLE);
if (!IS_ERR(rwq_ind_tbl)) {
if (cap.max_recv_wr || cap.max_recv_sge ||
uverbs_attr_is_valid(attrs,
UVERBS_ATTR_CREATE_QP_RECV_CQ_HANDLE) ||
uverbs_attr_is_valid(attrs,
UVERBS_ATTR_CREATE_QP_SRQ_HANDLE))
return -EINVAL;
/* send_cq is optinal */
if (cap.max_send_wr) {
send_cq = uverbs_attr_get_obj(attrs,
UVERBS_ATTR_CREATE_QP_SEND_CQ_HANDLE);
if (IS_ERR(send_cq))
return PTR_ERR(send_cq);
}
attr.rwq_ind_tbl = rwq_ind_tbl;
} else {
send_cq = uverbs_attr_get_obj(attrs,
UVERBS_ATTR_CREATE_QP_SEND_CQ_HANDLE);
if (IS_ERR(send_cq))
return PTR_ERR(send_cq);
if (attr.qp_type != IB_QPT_XRC_INI) {
recv_cq = uverbs_attr_get_obj(attrs,
UVERBS_ATTR_CREATE_QP_RECV_CQ_HANDLE);
if (IS_ERR(recv_cq))
return PTR_ERR(recv_cq);
}
}
device = pd->device;
break;
default:
return -EINVAL;
}
ret = uverbs_get_flags32(&attr.create_flags, attrs,
UVERBS_ATTR_CREATE_QP_FLAGS,
IB_UVERBS_QP_CREATE_BLOCK_MULTICAST_LOOPBACK |
IB_UVERBS_QP_CREATE_SCATTER_FCS |
IB_UVERBS_QP_CREATE_CVLAN_STRIPPING |
IB_UVERBS_QP_CREATE_PCI_WRITE_END_PADDING |
IB_UVERBS_QP_CREATE_SQ_SIG_ALL);
if (ret)
return ret;
ret = check_creation_flags(attr.qp_type, attr.create_flags);
if (ret)
return ret;
if (uverbs_attr_is_valid(attrs,
UVERBS_ATTR_CREATE_QP_SOURCE_QPN)) {
ret = uverbs_copy_from(&attr.source_qpn, attrs,
UVERBS_ATTR_CREATE_QP_SOURCE_QPN);
if (ret)
return ret;
attr.create_flags |= IB_QP_CREATE_SOURCE_QPN;
}
srq = uverbs_attr_get_obj(attrs,
UVERBS_ATTR_CREATE_QP_SRQ_HANDLE);
if (!IS_ERR(srq)) {
if ((srq->srq_type == IB_SRQT_XRC &&
attr.qp_type != IB_QPT_XRC_TGT) ||
(srq->srq_type != IB_SRQT_XRC &&
attr.qp_type == IB_QPT_XRC_TGT))
return -EINVAL;
attr.srq = srq;
}
obj->uevent.event_file = ib_uverbs_get_async_event(attrs,
UVERBS_ATTR_CREATE_QP_EVENT_FD);
INIT_LIST_HEAD(&obj->uevent.event_list);
INIT_LIST_HEAD(&obj->mcast_list);
obj->uevent.uobject.user_handle = user_handle;
attr.event_handler = ib_uverbs_qp_event_handler;
attr.send_cq = send_cq;
attr.recv_cq = recv_cq;
attr.xrcd = xrcd;
if (attr.create_flags & IB_UVERBS_QP_CREATE_SQ_SIG_ALL) {
/* This creation bit is uverbs one, need to mask before
* calling drivers. It was added to prevent an extra user attr
* only for that when using ioctl.
*/
attr.create_flags &= ~IB_UVERBS_QP_CREATE_SQ_SIG_ALL;
attr.sq_sig_type = IB_SIGNAL_ALL_WR;
} else {
attr.sq_sig_type = IB_SIGNAL_REQ_WR;
}
set_caps(&attr, &cap, true);
mutex_init(&obj->mcast_lock);
if (attr.qp_type == IB_QPT_XRC_TGT)
qp = ib_create_qp(pd, &attr);
else
qp = _ib_create_qp(device, pd, &attr, &attrs->driver_udata,
obj);
if (IS_ERR(qp)) {
ret = PTR_ERR(qp);
goto err_put;
}
if (attr.qp_type != IB_QPT_XRC_TGT) {
atomic_inc(&pd->usecnt);
if (attr.send_cq)
atomic_inc(&attr.send_cq->usecnt);
if (attr.recv_cq)
atomic_inc(&attr.recv_cq->usecnt);
if (attr.srq)
atomic_inc(&attr.srq->usecnt);
if (attr.rwq_ind_tbl)
atomic_inc(&attr.rwq_ind_tbl->usecnt);
} else {
obj->uxrcd = container_of(xrcd_uobj, struct ib_uxrcd_object,
uobject);
atomic_inc(&obj->uxrcd->refcnt);
/* It is done in _ib_create_qp for other QP types */
qp->uobject = obj;
}
obj->uevent.uobject.object = qp;
uverbs_finalize_uobj_create(attrs, UVERBS_ATTR_CREATE_QP_HANDLE);
if (attr.qp_type != IB_QPT_XRC_TGT) {
ret = ib_create_qp_security(qp, device);
if (ret)
return ret;
}
set_caps(&attr, &cap, false);
ret = uverbs_copy_to_struct_or_zero(attrs,
UVERBS_ATTR_CREATE_QP_RESP_CAP, &cap,
sizeof(cap));
if (ret)
return ret;
ret = uverbs_copy_to(attrs, UVERBS_ATTR_CREATE_QP_RESP_QP_NUM,
&qp->qp_num,
sizeof(qp->qp_num));
return ret;
err_put:
if (obj->uevent.event_file)
uverbs_uobject_put(&obj->uevent.event_file->uobj);
return ret;
};
DECLARE_UVERBS_NAMED_METHOD(
UVERBS_METHOD_QP_CREATE,
UVERBS_ATTR_IDR(UVERBS_ATTR_CREATE_QP_HANDLE,
UVERBS_OBJECT_QP,
UVERBS_ACCESS_NEW,
UA_MANDATORY),
UVERBS_ATTR_IDR(UVERBS_ATTR_CREATE_QP_XRCD_HANDLE,
UVERBS_OBJECT_XRCD,
UVERBS_ACCESS_READ,
UA_OPTIONAL),
UVERBS_ATTR_IDR(UVERBS_ATTR_CREATE_QP_PD_HANDLE,
UVERBS_OBJECT_PD,
UVERBS_ACCESS_READ,
UA_OPTIONAL),
UVERBS_ATTR_IDR(UVERBS_ATTR_CREATE_QP_SRQ_HANDLE,
UVERBS_OBJECT_SRQ,
UVERBS_ACCESS_READ,
UA_OPTIONAL),
UVERBS_ATTR_IDR(UVERBS_ATTR_CREATE_QP_SEND_CQ_HANDLE,
UVERBS_OBJECT_CQ,
UVERBS_ACCESS_READ,
UA_OPTIONAL),
UVERBS_ATTR_IDR(UVERBS_ATTR_CREATE_QP_RECV_CQ_HANDLE,
UVERBS_OBJECT_CQ,
UVERBS_ACCESS_READ,
UA_OPTIONAL),
UVERBS_ATTR_IDR(UVERBS_ATTR_CREATE_QP_IND_TABLE_HANDLE,
UVERBS_OBJECT_RWQ_IND_TBL,
UVERBS_ACCESS_READ,
UA_OPTIONAL),
UVERBS_ATTR_PTR_IN(UVERBS_ATTR_CREATE_QP_USER_HANDLE,
UVERBS_ATTR_TYPE(u64),
UA_MANDATORY),
UVERBS_ATTR_PTR_IN(UVERBS_ATTR_CREATE_QP_CAP,
UVERBS_ATTR_STRUCT(struct ib_uverbs_qp_cap,
max_inline_data),
UA_MANDATORY),
UVERBS_ATTR_CONST_IN(UVERBS_ATTR_CREATE_QP_TYPE,
enum ib_uverbs_qp_type,
UA_MANDATORY),
UVERBS_ATTR_FLAGS_IN(UVERBS_ATTR_CREATE_QP_FLAGS,
enum ib_uverbs_qp_create_flags,
UA_OPTIONAL),
UVERBS_ATTR_PTR_IN(UVERBS_ATTR_CREATE_QP_SOURCE_QPN,
UVERBS_ATTR_TYPE(u32),
UA_OPTIONAL),
UVERBS_ATTR_FD(UVERBS_ATTR_CREATE_QP_EVENT_FD,
UVERBS_OBJECT_ASYNC_EVENT,
UVERBS_ACCESS_READ,
UA_OPTIONAL),
UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_CREATE_QP_RESP_CAP,
UVERBS_ATTR_STRUCT(struct ib_uverbs_qp_cap,
max_inline_data),
UA_MANDATORY),
UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_CREATE_QP_RESP_QP_NUM,
UVERBS_ATTR_TYPE(u32),
UA_MANDATORY),
UVERBS_ATTR_UHW());
static int UVERBS_HANDLER(UVERBS_METHOD_QP_DESTROY)(
struct uverbs_attr_bundle *attrs)
{
struct ib_uobject *uobj =
uverbs_attr_get_uobject(attrs, UVERBS_ATTR_DESTROY_QP_HANDLE);
struct ib_uqp_object *obj =
container_of(uobj, struct ib_uqp_object, uevent.uobject);
struct ib_uverbs_destroy_qp_resp resp = {
.events_reported = obj->uevent.events_reported
};
return uverbs_copy_to(attrs, UVERBS_ATTR_DESTROY_QP_RESP, &resp,
sizeof(resp));
}
DECLARE_UVERBS_NAMED_METHOD(
UVERBS_METHOD_QP_DESTROY,
UVERBS_ATTR_IDR(UVERBS_ATTR_DESTROY_QP_HANDLE,
UVERBS_OBJECT_QP,
UVERBS_ACCESS_DESTROY,
UA_MANDATORY),
UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_DESTROY_QP_RESP,
UVERBS_ATTR_TYPE(struct ib_uverbs_destroy_qp_resp),
UA_MANDATORY));
DECLARE_UVERBS_NAMED_OBJECT(
UVERBS_OBJECT_QP,
UVERBS_TYPE_ALLOC_IDR_SZ(sizeof(struct ib_uqp_object), uverbs_free_qp),
&UVERBS_METHOD(UVERBS_METHOD_QP_CREATE),
&UVERBS_METHOD(UVERBS_METHOD_QP_DESTROY));
const struct uapi_definition uverbs_def_obj_qp[] = {
UAPI_DEF_CHAIN_OBJ_TREE_NAMED(UVERBS_OBJECT_QP,
UAPI_DEF_OBJ_NEEDS_FN(destroy_qp)),
{}
};

234
drivers/infiniband/core/uverbs_std_types_srq.c Normal file
View File

@ -0,0 +1,234 @@
// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
/*
* Copyright (c) 2020, Mellanox Technologies inc. All rights reserved.
*/
#include <rdma/uverbs_std_types.h>
#include "rdma_core.h"
#include "uverbs.h"
static int uverbs_free_srq(struct ib_uobject *uobject,
enum rdma_remove_reason why,
struct uverbs_attr_bundle *attrs)
{
struct ib_srq *srq = uobject->object;
struct ib_uevent_object *uevent =
container_of(uobject, struct ib_uevent_object, uobject);
enum ib_srq_type srq_type = srq->srq_type;
int ret;
ret = ib_destroy_srq_user(srq, &attrs->driver_udata);
if (ib_is_destroy_retryable(ret, why, uobject))
return ret;
if (srq_type == IB_SRQT_XRC) {
struct ib_usrq_object *us =
container_of(uobject, struct ib_usrq_object,
uevent.uobject);
atomic_dec(&us->uxrcd->refcnt);
}
ib_uverbs_release_uevent(uevent);
return ret;
}
static int UVERBS_HANDLER(UVERBS_METHOD_SRQ_CREATE)(
struct uverbs_attr_bundle *attrs)
{
struct ib_usrq_object *obj = container_of(
uverbs_attr_get_uobject(attrs, UVERBS_ATTR_CREATE_SRQ_HANDLE),
typeof(*obj), uevent.uobject);
struct ib_pd *pd =
uverbs_attr_get_obj(attrs, UVERBS_ATTR_CREATE_SRQ_PD_HANDLE);
struct ib_srq_init_attr attr = {};
struct ib_uobject *xrcd_uobj;
struct ib_srq *srq;
u64 user_handle;
int ret;
ret = uverbs_copy_from(&attr.attr.max_sge, attrs,
UVERBS_ATTR_CREATE_SRQ_MAX_SGE);
if (!ret)
ret = uverbs_copy_from(&attr.attr.max_wr, attrs,
UVERBS_ATTR_CREATE_SRQ_MAX_WR);
if (!ret)
ret = uverbs_copy_from(&attr.attr.srq_limit, attrs,
UVERBS_ATTR_CREATE_SRQ_LIMIT);
if (!ret)
ret = uverbs_copy_from(&user_handle, attrs,
UVERBS_ATTR_CREATE_SRQ_USER_HANDLE);
if (!ret)
ret = uverbs_get_const(&attr.srq_type, attrs,
UVERBS_ATTR_CREATE_SRQ_TYPE);
if (ret)
return ret;
if (ib_srq_has_cq(attr.srq_type)) {
attr.ext.cq = uverbs_attr_get_obj(attrs,
UVERBS_ATTR_CREATE_SRQ_CQ_HANDLE);
if (IS_ERR(attr.ext.cq))
return PTR_ERR(attr.ext.cq);
}
switch (attr.srq_type) {
case IB_UVERBS_SRQT_XRC:
xrcd_uobj = uverbs_attr_get_uobject(attrs,
UVERBS_ATTR_CREATE_SRQ_XRCD_HANDLE);
if (IS_ERR(xrcd_uobj))
return PTR_ERR(xrcd_uobj);
attr.ext.xrc.xrcd = (struct ib_xrcd *)xrcd_uobj->object;
if (!attr.ext.xrc.xrcd)
return -EINVAL;
obj->uxrcd = container_of(xrcd_uobj, struct ib_uxrcd_object,
uobject);
atomic_inc(&obj->uxrcd->refcnt);
break;
case IB_UVERBS_SRQT_TM:
ret = uverbs_copy_from(&attr.ext.tag_matching.max_num_tags,
attrs,
UVERBS_ATTR_CREATE_SRQ_MAX_NUM_TAGS);
if (ret)
return ret;
break;
case IB_UVERBS_SRQT_BASIC:
break;
default:
return -EINVAL;
}
obj->uevent.event_file = ib_uverbs_get_async_event(attrs,
UVERBS_ATTR_CREATE_SRQ_EVENT_FD);
INIT_LIST_HEAD(&obj->uevent.event_list);
attr.event_handler = ib_uverbs_srq_event_handler;
obj->uevent.uobject.user_handle = user_handle;
srq = ib_create_srq_user(pd, &attr, obj, &attrs->driver_udata);
if (IS_ERR(srq)) {
ret = PTR_ERR(srq);
goto err;
}
obj->uevent.uobject.object = srq;
uverbs_finalize_uobj_create(attrs, UVERBS_ATTR_CREATE_SRQ_HANDLE);
ret = uverbs_copy_to(attrs, UVERBS_ATTR_CREATE_SRQ_RESP_MAX_WR,
&attr.attr.max_wr,
sizeof(attr.attr.max_wr));
if (ret)
return ret;
ret = uverbs_copy_to(attrs, UVERBS_ATTR_CREATE_SRQ_RESP_MAX_SGE,
&attr.attr.max_sge,
sizeof(attr.attr.max_sge));
if (ret)
return ret;
if (attr.srq_type == IB_SRQT_XRC) {
ret = uverbs_copy_to(attrs,
UVERBS_ATTR_CREATE_SRQ_RESP_SRQ_NUM,
&srq->ext.xrc.srq_num,
sizeof(srq->ext.xrc.srq_num));
if (ret)
return ret;
}
return 0;
err:
if (obj->uevent.event_file)
uverbs_uobject_put(&obj->uevent.event_file->uobj);
if (attr.srq_type == IB_SRQT_XRC)
atomic_dec(&obj->uxrcd->refcnt);
return ret;
};
DECLARE_UVERBS_NAMED_METHOD(
UVERBS_METHOD_SRQ_CREATE,
UVERBS_ATTR_IDR(UVERBS_ATTR_CREATE_SRQ_HANDLE,
UVERBS_OBJECT_SRQ,
UVERBS_ACCESS_NEW,
UA_MANDATORY),
UVERBS_ATTR_IDR(UVERBS_ATTR_CREATE_SRQ_PD_HANDLE,
UVERBS_OBJECT_PD,
UVERBS_ACCESS_READ,
UA_MANDATORY),
UVERBS_ATTR_CONST_IN(UVERBS_ATTR_CREATE_SRQ_TYPE,
enum ib_uverbs_srq_type,
UA_MANDATORY),
UVERBS_ATTR_PTR_IN(UVERBS_ATTR_CREATE_SRQ_USER_HANDLE,
UVERBS_ATTR_TYPE(u64),
UA_MANDATORY),
UVERBS_ATTR_PTR_IN(UVERBS_ATTR_CREATE_SRQ_MAX_WR,
UVERBS_ATTR_TYPE(u32),
UA_MANDATORY),
UVERBS_ATTR_PTR_IN(UVERBS_ATTR_CREATE_SRQ_MAX_SGE,
UVERBS_ATTR_TYPE(u32),
UA_MANDATORY),
UVERBS_ATTR_PTR_IN(UVERBS_ATTR_CREATE_SRQ_LIMIT,
UVERBS_ATTR_TYPE(u32),
UA_MANDATORY),
UVERBS_ATTR_IDR(UVERBS_ATTR_CREATE_SRQ_XRCD_HANDLE,
UVERBS_OBJECT_XRCD,
UVERBS_ACCESS_READ,
UA_OPTIONAL),
UVERBS_ATTR_IDR(UVERBS_ATTR_CREATE_SRQ_CQ_HANDLE,
UVERBS_OBJECT_CQ,
UVERBS_ACCESS_READ,
UA_OPTIONAL),
UVERBS_ATTR_PTR_IN(UVERBS_ATTR_CREATE_SRQ_MAX_NUM_TAGS,
UVERBS_ATTR_TYPE(u32),
UA_OPTIONAL),
UVERBS_ATTR_FD(UVERBS_ATTR_CREATE_SRQ_EVENT_FD,
UVERBS_OBJECT_ASYNC_EVENT,
UVERBS_ACCESS_READ,
UA_OPTIONAL),
UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_CREATE_SRQ_RESP_MAX_WR,
UVERBS_ATTR_TYPE(u32),
UA_MANDATORY),
UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_CREATE_SRQ_RESP_MAX_SGE,
UVERBS_ATTR_TYPE(u32),
UA_MANDATORY),
UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_CREATE_SRQ_RESP_SRQ_NUM,
UVERBS_ATTR_TYPE(u32),
UA_OPTIONAL),
UVERBS_ATTR_UHW());
static int UVERBS_HANDLER(UVERBS_METHOD_SRQ_DESTROY)(
struct uverbs_attr_bundle *attrs)
{
struct ib_uobject *uobj =
uverbs_attr_get_uobject(attrs, UVERBS_ATTR_DESTROY_SRQ_HANDLE);
struct ib_usrq_object *obj =
container_of(uobj, struct ib_usrq_object, uevent.uobject);
struct ib_uverbs_destroy_srq_resp resp = {
.events_reported = obj->uevent.events_reported
};
return uverbs_copy_to(attrs, UVERBS_ATTR_DESTROY_SRQ_RESP, &resp,
sizeof(resp));
}
DECLARE_UVERBS_NAMED_METHOD(
UVERBS_METHOD_SRQ_DESTROY,
UVERBS_ATTR_IDR(UVERBS_ATTR_DESTROY_SRQ_HANDLE,
UVERBS_OBJECT_SRQ,
UVERBS_ACCESS_DESTROY,
UA_MANDATORY),
UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_DESTROY_SRQ_RESP,
UVERBS_ATTR_TYPE(struct ib_uverbs_destroy_srq_resp),
UA_MANDATORY));
DECLARE_UVERBS_NAMED_OBJECT(
UVERBS_OBJECT_SRQ,
UVERBS_TYPE_ALLOC_IDR_SZ(sizeof(struct ib_usrq_object),
uverbs_free_srq),
&UVERBS_METHOD(UVERBS_METHOD_SRQ_CREATE),
&UVERBS_METHOD(UVERBS_METHOD_SRQ_DESTROY)
);
const struct uapi_definition uverbs_def_obj_srq[] = {
UAPI_DEF_CHAIN_OBJ_TREE_NAMED(UVERBS_OBJECT_SRQ,
UAPI_DEF_OBJ_NEEDS_FN(destroy_srq)),
{}
};

194
drivers/infiniband/core/uverbs_std_types_wq.c Normal file
View File

@ -0,0 +1,194 @@
// SPDX-License-Identifier: GPL-2.0 OR Linux-OpenIB
/*
* Copyright (c) 2020, Mellanox Technologies inc. All rights reserved.
*/
#include <rdma/uverbs_std_types.h>
#include "rdma_core.h"
#include "uverbs.h"
static int uverbs_free_wq(struct ib_uobject *uobject,
enum rdma_remove_reason why,
struct uverbs_attr_bundle *attrs)
{
struct ib_wq *wq = uobject->object;
struct ib_uwq_object *uwq =
container_of(uobject, struct ib_uwq_object, uevent.uobject);
int ret;
ret = ib_destroy_wq(wq, &attrs->driver_udata);
if (ib_is_destroy_retryable(ret, why, uobject))
return ret;
ib_uverbs_release_uevent(&uwq->uevent);
return ret;
}
static int UVERBS_HANDLER(UVERBS_METHOD_WQ_CREATE)(
struct uverbs_attr_bundle *attrs)
{
struct ib_uwq_object *obj = container_of(
uverbs_attr_get_uobject(attrs, UVERBS_ATTR_CREATE_WQ_HANDLE),
typeof(*obj), uevent.uobject);
struct ib_pd *pd =
uverbs_attr_get_obj(attrs, UVERBS_ATTR_CREATE_WQ_PD_HANDLE);
struct ib_cq *cq =
uverbs_attr_get_obj(attrs, UVERBS_ATTR_CREATE_WQ_CQ_HANDLE);
struct ib_wq_init_attr wq_init_attr = {};
struct ib_wq *wq;
u64 user_handle;
int ret;
ret = uverbs_get_flags32(&wq_init_attr.create_flags, attrs,
UVERBS_ATTR_CREATE_WQ_FLAGS,
IB_UVERBS_WQ_FLAGS_CVLAN_STRIPPING |
IB_UVERBS_WQ_FLAGS_SCATTER_FCS |
IB_UVERBS_WQ_FLAGS_DELAY_DROP |
IB_UVERBS_WQ_FLAGS_PCI_WRITE_END_PADDING);
if (!ret)
ret = uverbs_copy_from(&wq_init_attr.max_sge, attrs,
UVERBS_ATTR_CREATE_WQ_MAX_SGE);
if (!ret)
ret = uverbs_copy_from(&wq_init_attr.max_wr, attrs,
UVERBS_ATTR_CREATE_WQ_MAX_WR);
if (!ret)
ret = uverbs_copy_from(&user_handle, attrs,
UVERBS_ATTR_CREATE_WQ_USER_HANDLE);
if (!ret)
ret = uverbs_get_const(&wq_init_attr.wq_type, attrs,
UVERBS_ATTR_CREATE_WQ_TYPE);
if (ret)
return ret;
if (wq_init_attr.wq_type != IB_WQT_RQ)
return -EINVAL;
obj->uevent.event_file = ib_uverbs_get_async_event(attrs,
UVERBS_ATTR_CREATE_WQ_EVENT_FD);
obj->uevent.uobject.user_handle = user_handle;
INIT_LIST_HEAD(&obj->uevent.event_list);
wq_init_attr.event_handler = ib_uverbs_wq_event_handler;
wq_init_attr.wq_context = attrs->ufile;
wq_init_attr.cq = cq;
wq = pd->device->ops.create_wq(pd, &wq_init_attr, &attrs->driver_udata);
if (IS_ERR(wq)) {
ret = PTR_ERR(wq);
goto err;
}
obj->uevent.uobject.object = wq;
wq->wq_type = wq_init_attr.wq_type;
wq->cq = cq;
wq->pd = pd;
wq->device = pd->device;
wq->wq_context = wq_init_attr.wq_context;
atomic_set(&wq->usecnt, 0);
atomic_inc(&pd->usecnt);
atomic_inc(&cq->usecnt);
wq->uobject = obj;
uverbs_finalize_uobj_create(attrs, UVERBS_ATTR_CREATE_WQ_HANDLE);
ret = uverbs_copy_to(attrs, UVERBS_ATTR_CREATE_WQ_RESP_MAX_WR,
&wq_init_attr.max_wr,
sizeof(wq_init_attr.max_wr));
if (ret)
return ret;
ret = uverbs_copy_to(attrs, UVERBS_ATTR_CREATE_WQ_RESP_MAX_SGE,
&wq_init_attr.max_sge,
sizeof(wq_init_attr.max_sge));
if (ret)
return ret;
ret = uverbs_copy_to(attrs, UVERBS_ATTR_CREATE_WQ_RESP_WQ_NUM,
&wq->wq_num,
sizeof(wq->wq_num));
return ret;
err:
if (obj->uevent.event_file)
uverbs_uobject_put(&obj->uevent.event_file->uobj);
return ret;
};
DECLARE_UVERBS_NAMED_METHOD(
UVERBS_METHOD_WQ_CREATE,
UVERBS_ATTR_IDR(UVERBS_ATTR_CREATE_WQ_HANDLE,
UVERBS_OBJECT_WQ,
UVERBS_ACCESS_NEW,
UA_MANDATORY),
UVERBS_ATTR_IDR(UVERBS_ATTR_CREATE_WQ_PD_HANDLE,
UVERBS_OBJECT_PD,
UVERBS_ACCESS_READ,
UA_MANDATORY),
UVERBS_ATTR_CONST_IN(UVERBS_ATTR_CREATE_WQ_TYPE,
enum ib_wq_type,
UA_MANDATORY),
UVERBS_ATTR_PTR_IN(UVERBS_ATTR_CREATE_WQ_USER_HANDLE,
UVERBS_ATTR_TYPE(u64),
UA_MANDATORY),
UVERBS_ATTR_PTR_IN(UVERBS_ATTR_CREATE_WQ_MAX_WR,
UVERBS_ATTR_TYPE(u32),
UA_MANDATORY),
UVERBS_ATTR_PTR_IN(UVERBS_ATTR_CREATE_WQ_MAX_SGE,
UVERBS_ATTR_TYPE(u32),
UA_MANDATORY),
UVERBS_ATTR_FLAGS_IN(UVERBS_ATTR_CREATE_WQ_FLAGS,
enum ib_uverbs_wq_flags,
UA_MANDATORY),
UVERBS_ATTR_IDR(UVERBS_ATTR_CREATE_WQ_CQ_HANDLE,
UVERBS_OBJECT_CQ,
UVERBS_ACCESS_READ,
UA_OPTIONAL),
UVERBS_ATTR_FD(UVERBS_ATTR_CREATE_WQ_EVENT_FD,
UVERBS_OBJECT_ASYNC_EVENT,
UVERBS_ACCESS_READ,
UA_OPTIONAL),
UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_CREATE_WQ_RESP_MAX_WR,
UVERBS_ATTR_TYPE(u32),
UA_MANDATORY),
UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_CREATE_WQ_RESP_MAX_SGE,
UVERBS_ATTR_TYPE(u32),
UA_MANDATORY),
UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_CREATE_WQ_RESP_WQ_NUM,
UVERBS_ATTR_TYPE(u32),
UA_OPTIONAL),
UVERBS_ATTR_UHW());
static int UVERBS_HANDLER(UVERBS_METHOD_WQ_DESTROY)(
struct uverbs_attr_bundle *attrs)
{
struct ib_uobject *uobj =
uverbs_attr_get_uobject(attrs, UVERBS_ATTR_DESTROY_WQ_HANDLE);
struct ib_uwq_object *obj =
container_of(uobj, struct ib_uwq_object, uevent.uobject);
return uverbs_copy_to(attrs, UVERBS_ATTR_DESTROY_WQ_RESP,
&obj->uevent.events_reported,
sizeof(obj->uevent.events_reported));
}
DECLARE_UVERBS_NAMED_METHOD(
UVERBS_METHOD_WQ_DESTROY,
UVERBS_ATTR_IDR(UVERBS_ATTR_DESTROY_WQ_HANDLE,
UVERBS_OBJECT_WQ,
UVERBS_ACCESS_DESTROY,
UA_MANDATORY),
UVERBS_ATTR_PTR_OUT(UVERBS_ATTR_DESTROY_WQ_RESP,
UVERBS_ATTR_TYPE(u32),
UA_MANDATORY));
DECLARE_UVERBS_NAMED_OBJECT(
UVERBS_OBJECT_WQ,
UVERBS_TYPE_ALLOC_IDR_SZ(sizeof(struct ib_uwq_object), uverbs_free_wq),
&UVERBS_METHOD(UVERBS_METHOD_WQ_CREATE),
&UVERBS_METHOD(UVERBS_METHOD_WQ_DESTROY)
);
const struct uapi_definition uverbs_def_obj_wq[] = {
UAPI_DEF_CHAIN_OBJ_TREE_NAMED(UVERBS_OBJECT_WQ,
UAPI_DEF_OBJ_NEEDS_FN(destroy_wq)),
{}
};

3
drivers/infiniband/core/uverbs_uapi.c
View File

@ -634,6 +634,9 @@ static const struct uapi_definition uverbs_core_api[] = {
UAPI_DEF_CHAIN(uverbs_def_obj_flow_action),
UAPI_DEF_CHAIN(uverbs_def_obj_intf),
UAPI_DEF_CHAIN(uverbs_def_obj_mr),
UAPI_DEF_CHAIN(uverbs_def_obj_qp),
UAPI_DEF_CHAIN(uverbs_def_obj_srq),
UAPI_DEF_CHAIN(uverbs_def_obj_wq),
UAPI_DEF_CHAIN(uverbs_def_write_intf),
{},
};

159
drivers/infiniband/core/verbs.c
View File

@ -50,6 +50,7 @@
#include <rdma/ib_cache.h>
#include <rdma/ib_addr.h>
#include <rdma/rw.h>
#include <rdma/lag.h>
#include "core_priv.h"
#include <trace/events/rdma_core.h>
@ -500,8 +501,10 @@ rdma_update_sgid_attr(struct rdma_ah_attr *ah_attr,
static struct ib_ah *_rdma_create_ah(struct ib_pd *pd,
struct rdma_ah_attr *ah_attr,
u32 flags,
struct ib_udata *udata)
struct ib_udata *udata,
struct net_device *xmit_slave)
{
struct rdma_ah_init_attr init_attr = {};
struct ib_device *device = pd->device;
struct ib_ah *ah;
int ret;
@ -521,8 +524,11 @@ static struct ib_ah *_rdma_create_ah(struct ib_pd *pd,
ah->pd = pd;
ah->type = ah_attr->type;
ah->sgid_attr = rdma_update_sgid_attr(ah_attr, NULL);
init_attr.ah_attr = ah_attr;
init_attr.flags = flags;
init_attr.xmit_slave = xmit_slave;
ret = device->ops.create_ah(ah, ah_attr, flags, udata);
ret = device->ops.create_ah(ah, &init_attr, udata);
if (ret) {
kfree(ah);
return ERR_PTR(ret);
@ -547,15 +553,22 @@ struct ib_ah *rdma_create_ah(struct ib_pd *pd, struct rdma_ah_attr *ah_attr,
u32 flags)
{
const struct ib_gid_attr *old_sgid_attr;
struct net_device *slave;
struct ib_ah *ah;
int ret;
ret = rdma_fill_sgid_attr(pd->device, ah_attr, &old_sgid_attr);
if (ret)
return ERR_PTR(ret);
ah = _rdma_create_ah(pd, ah_attr, flags, NULL);
slave = rdma_lag_get_ah_roce_slave(pd->device, ah_attr,
(flags & RDMA_CREATE_AH_SLEEPABLE) ?
GFP_KERNEL : GFP_ATOMIC);
if (IS_ERR(slave)) {
rdma_unfill_sgid_attr(ah_attr, old_sgid_attr);
return (void *)slave;
}
ah = _rdma_create_ah(pd, ah_attr, flags, NULL, slave);
rdma_lag_put_ah_roce_slave(slave);
rdma_unfill_sgid_attr(ah_attr, old_sgid_attr);
return ah;
}
@ -594,7 +607,8 @@ struct ib_ah *rdma_create_user_ah(struct ib_pd *pd,
}
}
ah = _rdma_create_ah(pd, ah_attr, RDMA_CREATE_AH_SLEEPABLE, udata);
ah = _rdma_create_ah(pd, ah_attr, RDMA_CREATE_AH_SLEEPABLE,
udata, NULL);
out:
rdma_unfill_sgid_attr(ah_attr, old_sgid_attr);
@ -967,15 +981,29 @@ EXPORT_SYMBOL(rdma_destroy_ah_user);
/* Shared receive queues */
struct ib_srq *ib_create_srq(struct ib_pd *pd,
struct ib_srq_init_attr *srq_init_attr)
/**
* ib_create_srq_user - Creates a SRQ associated with the specified protection
* domain.
* @pd: The protection domain associated with the SRQ.
* @srq_init_attr: A list of initial attributes required to create the
* SRQ. If SRQ creation succeeds, then the attributes are updated to
* the actual capabilities of the created SRQ.
* @uobject - uobject pointer if this is not a kernel SRQ
* @udata - udata pointer if this is not a kernel SRQ
*
* srq_attr->max_wr and srq_attr->max_sge are read the determine the
* requested size of the SRQ, and set to the actual values allocated
* on return. If ib_create_srq() succeeds, then max_wr and max_sge
* will always be at least as large as the requested values.
*/
struct ib_srq *ib_create_srq_user(struct ib_pd *pd,
struct ib_srq_init_attr *srq_init_attr,
struct ib_usrq_object *uobject,
struct ib_udata *udata)
{
struct ib_srq *srq;
int ret;
if (!pd->device->ops.create_srq)
return ERR_PTR(-EOPNOTSUPP);
srq = rdma_zalloc_drv_obj(pd->device, ib_srq);
if (!srq)
return ERR_PTR(-ENOMEM);
@ -985,6 +1013,7 @@ struct ib_srq *ib_create_srq(struct ib_pd *pd,
srq->event_handler = srq_init_attr->event_handler;
srq->srq_context = srq_init_attr->srq_context;
srq->srq_type = srq_init_attr->srq_type;
srq->uobject = uobject;
if (ib_srq_has_cq(srq->srq_type)) {
srq->ext.cq = srq_init_attr->ext.cq;
@ -996,7 +1025,7 @@ struct ib_srq *ib_create_srq(struct ib_pd *pd,
}
atomic_inc(&pd->usecnt);
ret = pd->device->ops.create_srq(srq, srq_init_attr, NULL);
ret = pd->device->ops.create_srq(srq, srq_init_attr, udata);
if (ret) {
atomic_dec(&srq->pd->usecnt);
if (srq->srq_type == IB_SRQT_XRC)
@ -1009,7 +1038,7 @@ struct ib_srq *ib_create_srq(struct ib_pd *pd,
return srq;
}
EXPORT_SYMBOL(ib_create_srq);
EXPORT_SYMBOL(ib_create_srq_user);
int ib_modify_srq(struct ib_srq *srq,
struct ib_srq_attr *srq_attr,
@ -1633,11 +1662,35 @@ static int _ib_modify_qp(struct ib_qp *qp, struct ib_qp_attr *attr,
const struct ib_gid_attr *old_sgid_attr_alt_av;
int ret;
attr->xmit_slave = NULL;
if (attr_mask & IB_QP_AV) {
ret = rdma_fill_sgid_attr(qp->device, &attr->ah_attr,
&old_sgid_attr_av);
if (ret)
return ret;
if (attr->ah_attr.type == RDMA_AH_ATTR_TYPE_ROCE &&
is_qp_type_connected(qp)) {
struct net_device *slave;
/*
* If the user provided the qp_attr then we have to
* resolve it. Kerne users have to provide already
* resolved rdma_ah_attr's.
*/
if (udata) {
ret = ib_resolve_eth_dmac(qp->device,
&attr->ah_attr);
if (ret)
goto out_av;
}
slave = rdma_lag_get_ah_roce_slave(qp->device,
&attr->ah_attr,
GFP_KERNEL);
if (IS_ERR(slave))
goto out_av;
attr->xmit_slave = slave;
}
}
if (attr_mask & IB_QP_ALT_PATH) {
/*
@ -1664,18 +1717,6 @@ static int _ib_modify_qp(struct ib_qp *qp, struct ib_qp_attr *attr,
}
}
/*
* If the user provided the qp_attr then we have to resolve it. Kernel
* users have to provide already resolved rdma_ah_attr's
*/
if (udata && (attr_mask & IB_QP_AV) &&
attr->ah_attr.type == RDMA_AH_ATTR_TYPE_ROCE &&
is_qp_type_connected(qp)) {
ret = ib_resolve_eth_dmac(qp->device, &attr->ah_attr);
if (ret)
goto out;
}
if (rdma_ib_or_roce(qp->device, port)) {
if (attr_mask & IB_QP_RQ_PSN && attr->rq_psn & ~0xffffff) {
dev_warn(&qp->device->dev,
@ -1717,8 +1758,10 @@ static int _ib_modify_qp(struct ib_qp *qp, struct ib_qp_attr *attr,
if (attr_mask & IB_QP_ALT_PATH)
rdma_unfill_sgid_attr(&attr->alt_ah_attr, old_sgid_attr_alt_av);
out_av:
if (attr_mask & IB_QP_AV)
if (attr_mask & IB_QP_AV) {
rdma_lag_put_ah_roce_slave(attr->xmit_slave);
rdma_unfill_sgid_attr(&attr->ah_attr, old_sgid_attr_av);
}
return ret;
}
@ -1962,6 +2005,9 @@ EXPORT_SYMBOL(__ib_create_cq);
int rdma_set_cq_moderation(struct ib_cq *cq, u16 cq_count, u16 cq_period)
{
if (cq->shared)
return -EOPNOTSUPP;
return cq->device->ops.modify_cq ?
cq->device->ops.modify_cq(cq, cq_count,
cq_period) : -EOPNOTSUPP;
@ -1970,6 +2016,9 @@ EXPORT_SYMBOL(rdma_set_cq_moderation);
int ib_destroy_cq_user(struct ib_cq *cq, struct ib_udata *udata)
{
if (WARN_ON_ONCE(cq->shared))
return -EOPNOTSUPP;
if (atomic_read(&cq->usecnt))
return -EBUSY;
@ -1982,6 +2031,9 @@ EXPORT_SYMBOL(ib_destroy_cq_user);
int ib_resize_cq(struct ib_cq *cq, int cqe)
{
if (cq->shared)
return -EOPNOTSUPP;
return cq->device->ops.resize_cq ?
cq->device->ops.resize_cq(cq, cqe, NULL) : -EOPNOTSUPP;
}
@ -2160,54 +2212,6 @@ struct ib_mr *ib_alloc_mr_integrity(struct ib_pd *pd,
}
EXPORT_SYMBOL(ib_alloc_mr_integrity);
/* "Fast" memory regions */
struct ib_fmr *ib_alloc_fmr(struct ib_pd *pd,
int mr_access_flags,
struct ib_fmr_attr *fmr_attr)
{
struct ib_fmr *fmr;
if (!pd->device->ops.alloc_fmr)
return ERR_PTR(-EOPNOTSUPP);
fmr = pd->device->ops.alloc_fmr(pd, mr_access_flags, fmr_attr);
if (!IS_ERR(fmr)) {
fmr->device = pd->device;
fmr->pd = pd;
atomic_inc(&pd->usecnt);
}
return fmr;
}
EXPORT_SYMBOL(ib_alloc_fmr);
int ib_unmap_fmr(struct list_head *fmr_list)
{
struct ib_fmr *fmr;
if (list_empty(fmr_list))
return 0;
fmr = list_entry(fmr_list->next, struct ib_fmr, list);
return fmr->device->ops.unmap_fmr(fmr_list);
}
EXPORT_SYMBOL(ib_unmap_fmr);
int ib_dealloc_fmr(struct ib_fmr *fmr)
{
struct ib_pd *pd;
int ret;
pd = fmr->pd;
ret = fmr->device->ops.dealloc_fmr(fmr);
if (!ret)
atomic_dec(&pd->usecnt);
return ret;
}
EXPORT_SYMBOL(ib_dealloc_fmr);
/* Multicast groups */
static bool is_valid_mcast_lid(struct ib_qp *qp, u16 lid)
@ -2574,6 +2578,7 @@ EXPORT_SYMBOL(ib_map_mr_sg_pi);
* @page_size: page vector desired page size
*
* Constraints:
*
* - The first sg element is allowed to have an offset.
* - Each sg element must either be aligned to page_size or virtually
* contiguous to the previous element. In case an sg element has a
@ -2607,10 +2612,12 @@ EXPORT_SYMBOL(ib_map_mr_sg);
* @mr: memory region
* @sgl: dma mapped scatterlist
* @sg_nents: number of entries in sg
* @sg_offset_p: IN: start offset in bytes into sg
* OUT: offset in bytes for element n of the sg of the first
* @sg_offset_p: ==== =======================================================
* IN start offset in bytes into sg
* OUT offset in bytes for element n of the sg of the first
* byte that has not been processed where n is the return
* value of this function.
* ==== =======================================================
* @set_page: driver page assignment function pointer
*
* Core service helper for drivers to convert the largest

76
drivers/infiniband/hw/bnxt_re/ib_verbs.c
View File

@ -177,9 +177,6 @@ int bnxt_re_query_device(struct ib_device *ibdev,
ib_attr->max_total_mcast_qp_attach = 0;
ib_attr->max_ah = dev_attr->max_ah;
ib_attr->max_fmr = 0;
ib_attr->max_map_per_fmr = 0;
ib_attr->max_srq = dev_attr->max_srq;
ib_attr->max_srq_wr = dev_attr->max_srq_wqes;
ib_attr->max_srq_sge = dev_attr->max_srq_sges;
@ -631,11 +628,12 @@ static u8 bnxt_re_stack_to_dev_nw_type(enum rdma_network_type ntype)
return nw_type;
}
int bnxt_re_create_ah(struct ib_ah *ib_ah, struct rdma_ah_attr *ah_attr,
u32 flags, struct ib_udata *udata)
int bnxt_re_create_ah(struct ib_ah *ib_ah, struct rdma_ah_init_attr *init_attr,
struct ib_udata *udata)
{
struct ib_pd *ib_pd = ib_ah->pd;
struct bnxt_re_pd *pd = container_of(ib_pd, struct bnxt_re_pd, ib_pd);
struct rdma_ah_attr *ah_attr = init_attr->ah_attr;
const struct ib_global_route *grh = rdma_ah_read_grh(ah_attr);
struct bnxt_re_dev *rdev = pd->rdev;
const struct ib_gid_attr *sgid_attr;
@ -673,7 +671,8 @@ int bnxt_re_create_ah(struct ib_ah *ib_ah, struct rdma_ah_attr *ah_attr,
memcpy(ah->qplib_ah.dmac, ah_attr->roce.dmac, ETH_ALEN);
rc = bnxt_qplib_create_ah(&rdev->qplib_res, &ah->qplib_ah,
!(flags & RDMA_CREATE_AH_SLEEPABLE));
!(init_attr->flags &
RDMA_CREATE_AH_SLEEPABLE));
if (rc) {
ibdev_err(&rdev->ibdev, "Failed to allocate HW AH");
return rc;
@ -856,7 +855,7 @@ static int bnxt_re_init_user_qp(struct bnxt_re_dev *rdev, struct bnxt_re_pd *pd,
if (ib_copy_from_udata(&ureq, udata, sizeof(ureq)))
return -EFAULT;
bytes = (qplib_qp->sq.max_wqe * BNXT_QPLIB_MAX_SQE_ENTRY_SIZE);
bytes = (qplib_qp->sq.max_wqe * qplib_qp->sq.wqe_size);
/* Consider mapping PSN search memory only for RC QPs. */
if (qplib_qp->type == CMDQ_CREATE_QP_TYPE_RC) {
psn_sz = bnxt_qplib_is_chip_gen_p5(rdev->chip_ctx) ?
@ -879,7 +878,7 @@ static int bnxt_re_init_user_qp(struct bnxt_re_dev *rdev, struct bnxt_re_pd *pd,
qplib_qp->qp_handle = ureq.qp_handle;
if (!qp->qplib_qp.srq) {
bytes = (qplib_qp->rq.max_wqe * BNXT_QPLIB_MAX_RQE_ENTRY_SIZE);
bytes = (qplib_qp->rq.max_wqe * qplib_qp->rq.wqe_size);
bytes = PAGE_ALIGN(bytes);
umem = ib_umem_get(&rdev->ibdev, ureq.qprva, bytes,
IB_ACCESS_LOCAL_WRITE);
@ -976,6 +975,7 @@ static struct bnxt_re_qp *bnxt_re_create_shadow_qp
qp->qplib_qp.sig_type = true;
/* Shadow QP SQ depth should be same as QP1 RQ depth */
qp->qplib_qp.sq.wqe_size = bnxt_re_get_swqe_size();
qp->qplib_qp.sq.max_wqe = qp1_qp->rq.max_wqe;
qp->qplib_qp.sq.max_sge = 2;
/* Q full delta can be 1 since it is internal QP */
@ -986,6 +986,7 @@ static struct bnxt_re_qp *bnxt_re_create_shadow_qp
qp->qplib_qp.scq = qp1_qp->scq;
qp->qplib_qp.rcq = qp1_qp->rcq;
qp->qplib_qp.rq.wqe_size = bnxt_re_get_rwqe_size();
qp->qplib_qp.rq.max_wqe = qp1_qp->rq.max_wqe;
qp->qplib_qp.rq.max_sge = qp1_qp->rq.max_sge;
/* Q full delta can be 1 since it is internal QP */
@ -1021,10 +1022,12 @@ static int bnxt_re_init_rq_attr(struct bnxt_re_qp *qp,
struct bnxt_qplib_dev_attr *dev_attr;
struct bnxt_qplib_qp *qplqp;
struct bnxt_re_dev *rdev;
struct bnxt_qplib_q *rq;
int entries;
rdev = qp->rdev;
qplqp = &qp->qplib_qp;
rq = &qplqp->rq;
dev_attr = &rdev->dev_attr;
if (init_attr->srq) {
@ -1036,23 +1039,21 @@ static int bnxt_re_init_rq_attr(struct bnxt_re_qp *qp,
return -EINVAL;
}
qplqp->srq = &srq->qplib_srq;
qplqp->rq.max_wqe = 0;
rq->max_wqe = 0;
} else {
rq->wqe_size = bnxt_re_get_rwqe_size();
/* Allocate 1 more than what's provided so posting max doesn't
* mean empty.
*/
entries = roundup_pow_of_two(init_attr->cap.max_recv_wr + 1);
qplqp->rq.max_wqe = min_t(u32, entries,
dev_attr->max_qp_wqes + 1);
qplqp->rq.q_full_delta = qplqp->rq.max_wqe -
init_attr->cap.max_recv_wr;
qplqp->rq.max_sge = init_attr->cap.max_recv_sge;
if (qplqp->rq.max_sge > dev_attr->max_qp_sges)
qplqp->rq.max_sge = dev_attr->max_qp_sges;
rq->max_wqe = min_t(u32, entries, dev_attr->max_qp_wqes + 1);
rq->q_full_delta = rq->max_wqe - init_attr->cap.max_recv_wr;
rq->max_sge = init_attr->cap.max_recv_sge;
if (rq->max_sge > dev_attr->max_qp_sges)
rq->max_sge = dev_attr->max_qp_sges;
}
qplqp->rq.sg_info.pgsize = PAGE_SIZE;
qplqp->rq.sg_info.pgshft = PAGE_SHIFT;
rq->sg_info.pgsize = PAGE_SIZE;
rq->sg_info.pgshft = PAGE_SHIFT;
return 0;
}
@ -1080,15 +1081,18 @@ static void bnxt_re_init_sq_attr(struct bnxt_re_qp *qp,
struct bnxt_qplib_dev_attr *dev_attr;
struct bnxt_qplib_qp *qplqp;
struct bnxt_re_dev *rdev;
struct bnxt_qplib_q *sq;
int entries;
rdev = qp->rdev;
qplqp = &qp->qplib_qp;
sq = &qplqp->sq;
dev_attr = &rdev->dev_attr;
qplqp->sq.max_sge = init_attr->cap.max_send_sge;
if (qplqp->sq.max_sge > dev_attr->max_qp_sges)
qplqp->sq.max_sge = dev_attr->max_qp_sges;
sq->wqe_size = bnxt_re_get_swqe_size();
sq->max_sge = init_attr->cap.max_send_sge;
if (sq->max_sge > dev_attr->max_qp_sges)
sq->max_sge = dev_attr->max_qp_sges;
/*
* Change the SQ depth if user has requested minimum using
* configfs. Only supported for kernel consumers
@ -1096,9 +1100,9 @@ static void bnxt_re_init_sq_attr(struct bnxt_re_qp *qp,
entries = init_attr->cap.max_send_wr;
/* Allocate 128 + 1 more than what's provided */
entries = roundup_pow_of_two(entries + BNXT_QPLIB_RESERVED_QP_WRS + 1);
qplqp->sq.max_wqe = min_t(u32, entries, dev_attr->max_qp_wqes +
BNXT_QPLIB_RESERVED_QP_WRS + 1);
qplqp->sq.q_full_delta = BNXT_QPLIB_RESERVED_QP_WRS + 1;
sq->max_wqe = min_t(u32, entries, dev_attr->max_qp_wqes +
BNXT_QPLIB_RESERVED_QP_WRS + 1);
sq->q_full_delta = BNXT_QPLIB_RESERVED_QP_WRS + 1;
/*
* Reserving one slot for Phantom WQE. Application can
* post one extra entry in this case. But allowing this to avoid
@ -1511,7 +1515,7 @@ static int bnxt_re_init_user_srq(struct bnxt_re_dev *rdev,
if (ib_copy_from_udata(&ureq, udata, sizeof(ureq)))
return -EFAULT;
bytes = (qplib_srq->max_wqe * BNXT_QPLIB_MAX_RQE_ENTRY_SIZE);
bytes = (qplib_srq->max_wqe * qplib_srq->wqe_size);
bytes = PAGE_ALIGN(bytes);
umem = ib_umem_get(&rdev->ibdev, ureq.srqva, bytes,
IB_ACCESS_LOCAL_WRITE);
@ -1534,15 +1538,20 @@ int bnxt_re_create_srq(struct ib_srq *ib_srq,
struct ib_srq_init_attr *srq_init_attr,
struct ib_udata *udata)
{
struct ib_pd *ib_pd = ib_srq->pd;
struct bnxt_re_pd *pd = container_of(ib_pd, struct bnxt_re_pd, ib_pd);
struct bnxt_re_dev *rdev = pd->rdev;
struct bnxt_qplib_dev_attr *dev_attr = &rdev->dev_attr;
struct bnxt_re_srq *srq =
container_of(ib_srq, struct bnxt_re_srq, ib_srq);
struct bnxt_qplib_dev_attr *dev_attr;
struct bnxt_qplib_nq *nq = NULL;
struct bnxt_re_dev *rdev;
struct bnxt_re_srq *srq;
struct bnxt_re_pd *pd;
struct ib_pd *ib_pd;
int rc, entries;
ib_pd = ib_srq->pd;
pd = container_of(ib_pd, struct bnxt_re_pd, ib_pd);
rdev = pd->rdev;
dev_attr = &rdev->dev_attr;
srq = container_of(ib_srq, struct bnxt_re_srq, ib_srq);
if (srq_init_attr->attr.max_wr >= dev_attr->max_srq_wqes) {
ibdev_err(&rdev->ibdev, "Create CQ failed - max exceeded");
rc = -EINVAL;
@ -1563,8 +1572,9 @@ int bnxt_re_create_srq(struct ib_srq *ib_srq,
entries = roundup_pow_of_two(srq_init_attr->attr.max_wr + 1);
if (entries > dev_attr->max_srq_wqes + 1)
entries = dev_attr->max_srq_wqes + 1;
srq->qplib_srq.max_wqe = entries;
srq->qplib_srq.wqe_size = bnxt_re_get_rwqe_size();
srq->qplib_srq.max_sge = srq_init_attr->attr.max_sge;
srq->qplib_srq.threshold = srq_init_attr->attr.srq_limit;
srq->srq_limit = srq_init_attr->attr.srq_limit;

18
drivers/infiniband/hw/bnxt_re/ib_verbs.h
View File

@ -122,12 +122,6 @@ struct bnxt_re_frpl {
u64 *page_list;
};
struct bnxt_re_fmr {
struct bnxt_re_dev *rdev;
struct ib_fmr ib_fmr;
struct bnxt_qplib_mrw qplib_fmr;
};
struct bnxt_re_mw {
struct bnxt_re_dev *rdev;
struct ib_mw ib_mw;
@ -142,6 +136,16 @@ struct bnxt_re_ucontext {
spinlock_t sh_lock; /* protect shpg */
};
static inline u16 bnxt_re_get_swqe_size(void)
{
return sizeof(struct sq_send);
}
static inline u16 bnxt_re_get_rwqe_size(void)
{
return sizeof(struct rq_wqe);
}
int bnxt_re_query_device(struct ib_device *ibdev,
struct ib_device_attr *ib_attr,
struct ib_udata *udata);
@ -160,7 +164,7 @@ enum rdma_link_layer bnxt_re_get_link_layer(struct ib_device *ibdev,
u8 port_num);
int bnxt_re_alloc_pd(struct ib_pd *pd, struct ib_udata *udata);
void bnxt_re_dealloc_pd(struct ib_pd *pd, struct ib_udata *udata);
int bnxt_re_create_ah(struct ib_ah *ah, struct rdma_ah_attr *ah_attr, u32 flags,
int bnxt_re_create_ah(struct ib_ah *ah, struct rdma_ah_init_attr *init_attr,
struct ib_udata *udata);
int bnxt_re_modify_ah(struct ib_ah *ah, struct rdma_ah_attr *ah_attr);
int bnxt_re_query_ah(struct ib_ah *ah, struct rdma_ah_attr *ah_attr);

Some files were not shown because too many files have changed in this diff Show More