mirror of
https://github.com/torvalds/linux.git
synced 2026-07-31 19:47:08 +02:00
Linux kernel source tree
152ba35c04
Tariq Toukan says: ==================== net/mlx5e: Use multiple doorbells mlx5e uses a single MMIO-mapped doorbell per netdevice for all send and receive operations. Writes to the doorbell go over the PCIe bus directly to the device, which then services the indicated queues. On certain architectures and with sufficiently high volume of doorbell ringing (many cores, many active channels, small MTU, no GSO, etc.), the MMIO-mapped doorbell address can become contended, leading to delays in servicing writes to that address and a global slowdown of all traffic for that netdevice. mlx5 NICs have supported using multiple doorbells for many years, the mlx5_ib driver for the same hardware has been using multiple doorbells traditionally. This patch series extends the mlx5 Ethernet driver to also use multiple doorbells to solve the MMIO contention issues. By allocating and using more doorbells for all channel queues (TX and RX), the MMIO contention on any particular doorbell address is reduced significantly. The first patches are cleanups: net/mlx5: Fix typo of MLX5_EQ_DOORBEL_OFFSET net/mlx5: Remove unused 'offset' field from struct mlx5_sq_bfreg' net/mlx5e: Remove unused 'xsk' param of mlx5e_build_xdpsq_param The next patch separates the global doorbell from Ethernet-specific resources: net/mlx5: Store the global doorbell in mlx5_priv Next, plumbing to allow a different doorbell to be used for channel TX and RX queues: net/mlx5e: Prepare for using multiple TX doorbells net/mlx5e: Prepare for using different CQ doorbells Then, enable using multiple doorbells for channel queues: net/mlx5e: Use multiple TX doorbells net/mlx5e: Use multiple CQ doorbells Finally, introduce a devlink parameter to control this: devlink: Add a 'num_doorbells' driverinit param net/mlx5e: Use the 'num_doorbells' devlink param Some performance results, done with the Linux pktgen script, running b2b over Connect-X 8 NICs: samples/pktgen/pktgen_sample02_multiqueue.sh -i $NIC -s 64 -d $DST_IP \ -m $MAC -t 64 Baseline (1 doorbell): 9 Mpps This series (8 doorbells): 56 Mpps Note that pktgen without 'burst' rings the doorbell after every packet, while real packet TX using NAPI usually batches multiple pending packets with the xmit_more mechanism. So this is in essence a micro-benchmark showcasing the improvement of using multiple doorbells on platforms affected by MMIO contention. Real life traffic usually sees little movement either way. ==================== Link: https://patch.msgid.link/1758031904-634231-1-git-send-email-tariqt@nvidia.com Signed-off-by: Jakub Kicinski <kuba@kernel.org> |
||
|---|---|---|
| arch | ||
| block | ||
| certs | ||
| crypto | ||
| Documentation | ||
| drivers | ||
| fs | ||
| include | ||
| init | ||
| io_uring | ||
| ipc | ||
| kernel | ||
| lib | ||
| LICENSES | ||
| mm | ||
| net | ||
| rust | ||
| samples | ||
| scripts | ||
| security | ||
| sound | ||
| tools | ||
| usr | ||
| virt | ||
| .clang-format | ||
| .clippy.toml | ||
| .cocciconfig | ||
| .editorconfig | ||
| .get_maintainer.ignore | ||
| .gitattributes | ||
| .gitignore | ||
| .mailmap | ||
| .pylintrc | ||
| .rustfmt.toml | ||
| COPYING | ||
| CREDITS | ||
| Kbuild | ||
| Kconfig | ||
| MAINTAINERS | ||
| Makefile | ||
| README | ||
Linux kernel
============
There are several guides for kernel developers and users. These guides can
be rendered in a number of formats, like HTML and PDF. Please read
Documentation/admin-guide/README.rst first.
In order to build the documentation, use ``make htmldocs`` or
``make pdfdocs``. The formatted documentation can also be read online at:
https://www.kernel.org/doc/html/latest/
There are various text files in the Documentation/ subdirectory,
several of them using the reStructuredText markup notation.
Please read the Documentation/process/changes.rst file, as it contains the
requirements for building and running the kernel, and information about
the problems which may result by upgrading your kernel.