Merge branch 'docs-net-updates-for-old-and-cobwebbed-docs'

Jakub Kicinski says:

====================
docs: net: updates for old and cobwebbed docs

I'm hoping to start feeding our docs into the AI review tools, instead
of maintaining a separate repo with review prompts. To experiment with
that we have to refresh the docs a little bit.

A read thru our current docs makes one slightly question the value
of including them in reviews. But directionally, I feel, it's probably
still right. I'm hoping the Rx Checksum section about not dropping packets
for example to be impactful. I don't think the current AI agents or
review docs include this guidance.
====================

Link: https://patch.msgid.link/20260526160151.2793354-1-kuba@kernel.org
Signed-off-by: Jakub Kicinski <kuba@kernel.org>
This commit is contained in:
Jakub Kicinski 2026-05-28 14:37:49 -07:00
commit 1f03da8aea
9 changed files with 172 additions and 93 deletions

View File

@ -19,17 +19,14 @@ The following technologies are described:
Things that should be documented here but aren't yet:
* RX Checksum Offload
* CHECKSUM_UNNECESSARY conversion
TX Checksum Offload
===================
The interface for offloading a transmit checksum to a device is explained in
detail in comments near the top of include/linux/skbuff.h.
In brief, it allows to request the device fill in a single ones-complement
In brief, Tx checksum offload allows to request the device fill in a single
ones-complement
checksum defined by the sk_buff fields skb->csum_start and skb->csum_offset.
The device should compute the 16-bit ones-complement checksum (i.e. the
'IP-style' checksum) from csum_start to the end of the packet, and fill in the
@ -45,9 +42,10 @@ encapsulation is used, the packet may have multiple checksum fields in
different header layers, and the rest will have to be handled by another
mechanism such as LCO or RCO.
CRC32c can also be offloaded using this interface, by means of filling
skb->csum_start and skb->csum_offset as described above, and setting
skb->csum_not_inet: see skbuff.h comment (section 'D') for more details.
SCTP CRC32c can also be offloaded using this interface, by means of filling
skb->csum_start and skb->csum_offset as described above, setting
skb->csum_not_inet, and advertising NETIF_F_SCTP_CRC. Drivers must not treat
ordinary IP checksum offload as SCTP CRC32c support.
No offloading of the IP header checksum is performed; it is always done in
software. This is OK because when we build the IP header, we obviously have it
@ -55,18 +53,15 @@ in cache, so summing it isn't expensive. It's also rather short.
The requirements for GSO are more complicated, because when segmenting an
encapsulated packet both the inner and outer checksums may need to be edited or
recomputed for each resulting segment. See the skbuff.h comment (section 'E')
for more details.
recomputed for each resulting segment.
A driver declares its offload capabilities in netdev->hw_features; see
Documentation/networking/netdev-features.rst for more. Note that a device
which only advertises NETIF_F_IP[V6]_CSUM must still obey the csum_start and
csum_offset given in the SKB; if it tries to deduce these itself in hardware
(as some NICs do) the driver should check that the values in the SKB match
those which the hardware will deduce, and if not, fall back to checksumming in
software instead (with skb_csum_hwoffload_help() or one of the
skb_checksum_help() / skb_crc32c_csum_help functions, as mentioned in
include/linux/skbuff.h).
Documentation/networking/netdev-features.rst for more. NETIF_F_IP_CSUM and
NETIF_F_IPV6_CSUM are restricted legacy features and are being deprecated in
favor of NETIF_F_HW_CSUM. New devices should use NETIF_F_HW_CSUM to advertise
generic checksum offload. The skb_csum_hwoffload_help() helper can resolve
CHECKSUM_PARTIAL according to the device's advertised checksum capabilities,
falling back to software when needed.
The stack should, for the most part, assume that checksum offload is supported
by the underlying device. The only place that should check is
@ -108,11 +103,9 @@ LCO is performed by the stack when constructing an outer UDP header for an
encapsulation such as VXLAN or GENEVE, in udp_set_csum(). Similarly for the
IPv6 equivalents, in udp6_set_csum().
It is also performed when constructing an IPv4 GRE header, in
net/ipv4/ip_gre.c:build_header(). It is *not* currently performed when
constructing an IPv6 GRE header; the GRE checksum is computed over the whole
packet in net/ipv6/ip6_gre.c:ip6gre_xmit2(), but it should be possible to use
LCO here as IPv6 GRE still uses an IP-style checksum.
It is also performed when constructing GRE headers with the shared
gre_build_header() helper in include/net/gre.h, which is used by both IPv4 and
IPv6 GRE.
All of the LCO implementations use a helper function lco_csum(), in
include/linux/skbuff.h.
@ -138,6 +131,28 @@ RCO is detailed in the following Internet-Drafts:
* https://tools.ietf.org/html/draft-herbert-vxlan-rco-00
In Linux, RCO is implemented individually in each encapsulation protocol, and
most tunnel types have flags controlling its use. For instance, VXLAN has the
flag VXLAN_F_REMCSUM_TX (per struct vxlan_rdst) to indicate that RCO should be
used when transmitting to a given remote destination.
most tunnel types have flags controlling its use. For instance, VXLAN has the
configuration flag VXLAN_F_REMCSUM_TX to indicate that RCO should be used when
transmitting.
RX Checksum Offload
===================
RX checksum offload is controlled via NETIF_F_RXCSUM. When disabled the driver
must not set skb->ip_summed on ingress packets. As mentioned, IPv4 checksum
is not offloaded, the RXCSUM feature controls the offload of verification of
transport layer checksums.
Note that packets with bad TCP/UDP checksums must still be passed
to the stack. skb->ip_summed of such packets can be set to ``CHECKSUM_COMPLETE``
or left at ``CHECKSUM_NONE``. Drivers **must not discard** packets with
bad TCP/UDP checksum and must not configure the device to drop them.
Checksum validation is relatively inexpensive and having bad packets reflected
in SNMP counters is crucial for network monitoring.
skb checksum documentation
==========================
.. kernel-doc:: include/linux/skbuff.h
:doc: skb checksums

View File

@ -103,24 +103,22 @@ The user must bind a dmabuf to any number of RX queues on a given NIC using
the netlink API::
/* Bind dmabuf to NIC RX queue 15 */
struct netdev_queue *queues;
queues = malloc(sizeof(*queues) * 1);
struct netdev_queue_id *queues;
queues[0]._present.type = 1;
queues[0]._present.idx = 1;
queues[0].type = NETDEV_RX_QUEUE_TYPE_RX;
queues[0].idx = 15;
queues = netdev_queue_id_alloc(1);
netdev_queue_id_set_type(&queues[0], NETDEV_QUEUE_TYPE_RX);
netdev_queue_id_set_id(&queues[0], 15);
*ys = ynl_sock_create(&ynl_netdev_family, &yerr);
req = netdev_bind_rx_req_alloc();
netdev_bind_rx_req_set_ifindex(req, 1 /* ifindex */);
netdev_bind_rx_req_set_dmabuf_fd(req, dmabuf_fd);
__netdev_bind_rx_req_set_queues(req, queues, n_queue_index);
netdev_bind_rx_req_set_fd(req, dmabuf_fd);
__netdev_bind_rx_req_set_queues(req, queues, 1);
rsp = netdev_bind_rx(*ys, req);
dmabuf_id = rsp->dmabuf_id;
dmabuf_id = rsp->id;
The netlink API returns a dmabuf_id: a unique ID that refers to this dmabuf
@ -302,13 +300,12 @@ The user should create a msghdr where,
* iov_base is set to the offset into the dmabuf to start sending from
* iov_len is set to the number of bytes to be sent from the dmabuf
The user passes the dma-buf id to send from via the dmabuf_tx_cmsg.dmabuf_id.
The user passes the dma-buf id to send from as a u32 cmsg payload.
The example below sends 1024 bytes from offset 100 into the dmabuf, and 2048
from offset 2000 into the dmabuf. The dmabuf to send from is tx_dmabuf_id::
char ctrl_data[CMSG_SPACE(sizeof(struct dmabuf_tx_cmsg))];
struct dmabuf_tx_cmsg ddmabuf;
char ctrl_data[CMSG_SPACE(sizeof(__u32))];
struct msghdr msg = {};
struct cmsghdr *cmsg;
struct iovec iov[2];
@ -327,11 +324,9 @@ from offset 2000 into the dmabuf. The dmabuf to send from is tx_dmabuf_id::
cmsg = CMSG_FIRSTHDR(&msg);
cmsg->cmsg_level = SOL_SOCKET;
cmsg->cmsg_type = SCM_DEVMEM_DMABUF;
cmsg->cmsg_len = CMSG_LEN(sizeof(struct dmabuf_tx_cmsg));
cmsg->cmsg_len = CMSG_LEN(sizeof(__u32));
ddmabuf.dmabuf_id = tx_dmabuf_id;
*((struct dmabuf_tx_cmsg *)CMSG_DATA(cmsg)) = ddmabuf;
*((__u32 *)CMSG_DATA(cmsg)) = tx_dmabuf_id;
sendmsg(socket_fd, &msg, MSG_ZEROCOPY);

View File

@ -51,7 +51,7 @@ for a driver implementing scatter-gather this means:
{
u32 used = READ_ONCE(dr->prod) - READ_ONCE(dr->cons);
return dr->tx_ring_size - (used & bp->tx_ring_mask);
return dr->tx_ring_size - (used & dr->tx_ring_mask);
}
static netdev_tx_t drv_hard_start_xmit(struct sk_buff *skb,
@ -69,7 +69,7 @@ for a driver implementing scatter-gather this means:
//...
/* This should be a very rare race - log it. */
if (drv_tx_avail(dr) <= skb_shinfo(skb)->nr_frags + 1) {
netif_stop_queue(dev);
netif_tx_stop_queue(txq);
netdev_warn(dev, "Tx Ring full when queue awake!\n");
return NETDEV_TX_BUSY;
}
@ -103,6 +103,9 @@ Lockless queue stop / wake helper macros
.. kernel-doc:: include/net/netdev_queues.h
:doc: Lockless queue stopping / waking helpers.
The standard macros like netif_txq_maybe_stop(), netif_txq_try_stop() etc.
are well tested, prefer them over local synchronization schemes.
No exclusive ownership
----------------------

View File

@ -49,7 +49,11 @@ instance to be released.
The control APIs are not idempotent. Control API calls are safe against
concurrent use of datapath APIs but an incorrect sequence of control API
calls may result in crashes, deadlocks, or race conditions. For example,
calling napi_disable() multiple times in a row will deadlock.
calling napi_disable() multiple times in a row will hang waiting for
ownership of the NAPI instance to be released.
Drivers using the netdev instance lock may need to use the ``_locked()``
variants of the control APIs when that lock is already held.
Datapath API
------------
@ -190,7 +194,8 @@ User API
========
User interactions with NAPI depend on NAPI instance ID. The instance IDs
are only visible to the user thru the ``SO_INCOMING_NAPI_ID`` socket option.
are visible to the user through the ``SO_INCOMING_NAPI_ID`` socket option
and the netdev Netlink API.
Users can query NAPI IDs for a device or device queue using netlink. This can
be done programmatically in a user application or by using a script included in
@ -371,7 +376,7 @@ To use this mechanism:
the application has stalled. This value should be chosen so that it covers
the amount of time the user application needs to process data from its
call to epoll_wait, noting that applications can control how much data
they retrieve by setting ``max_events`` when calling epoll_wait.
they retrieve by setting ``maxevents`` when calling epoll_wait.
2. The sysfs parameter or per-NAPI config parameters ``gro_flush_timeout``
and ``napi_defer_hard_irqs`` can be set to low values. They will be used

View File

@ -18,29 +18,38 @@ that relieve an OS of various tasks like generating and checking checksums,
splitting packets, classifying them. Those capabilities and their state
are commonly referred to as netdev features in Linux kernel world.
There are currently three sets of features relevant to the driver, and
one used internally by network core:
There are currently three main sets of features on each netdevice,
first and second are initialized by the driver:
1. netdev->hw_features set contains features whose state may possibly
be changed (enabled or disabled) for a particular device by user's
request. This set should be initialized in ndo_init callback and not
changed later.
request. Drivers normally initialize this set before registration or
in the ndo_init callback. Changes after registration should be made
very carefully as other parts of the code may assume hw_features are
static. At the very least changes must be made under rtnl_lock and
the netdev instance lock, and followed by netdev_update_features().
2. netdev->features set contains features which are currently enabled
for a device. This should be changed only by network core or in
error paths of ndo_set_features callback.
3. netdev->vlan_features set contains features whose state is inherited
by child VLAN devices (limits netdev->features set). This is currently
used for all VLAN devices whether tags are stripped or inserted in
hardware or software.
4. netdev->wanted_features set contains feature set requested by user.
3. netdev->wanted_features set contains feature set requested by user.
This set is filtered by ndo_fix_features callback whenever it or
some device-specific conditions change. This set is internal to
networking core and should not be referenced in drivers.
On top of those three main sets, each netdev has:
1. Sets which control features inherited by child devices (VLAN, MPLS,
hw_enc for L3/L4 tunnels). These sets allow the driver to limit which
netdev->features are propagated, in case HW cannot perform the offloads
with the extra headers present.
2. netdev->mangleid_features, TSO features which are supported only when
IP ID field can be mangled (constant instead of incrementing) during TSO.
3. netdev->gso_partial_features, additional TSO features which HW can
support via NETIF_F_GSO_PARTIAL.
Part II: Controlling enabled features
=====================================
@ -62,11 +71,15 @@ ndo_*_features callbacks are called with rtnl_lock held. Missing callbacks
are treated as always returning success.
A driver that wants to trigger recalculation must do so by calling
netdev_update_features() while holding rtnl_lock. This should not be done
from ndo_*_features callbacks. netdev->features should not be modified by
driver except by means of ndo_fix_features callback.
netdev_update_features() while holding rtnl_lock. If the device uses the
netdev instance lock, that lock must be held as well. This should not be
done from ndo_*_features callbacks. netdev->features should not be modified
by driver except by means of ndo_fix_features callback.
ndo_features_check is called for each skb before that skb is passed to
ndo_start_xmit. Driver may perform any non-trivial checks (e.g. exact
header geometry / length) and withdraw features like HW_CSUM or TSO,
requesting the networking stack to fall back to the software implementation.
Part III: Implementation hints
==============================
@ -83,8 +96,9 @@ stateless). It can be called multiple times between successive
ndo_set_features calls.
Callback must not alter features contained in NETIF_F_SOFT_FEATURES or
NETIF_F_NEVER_CHANGE sets. The exception is NETIF_F_VLAN_CHALLENGED but
care must be taken as the change won't affect already configured VLANs.
NETIF_F_NEVER_CHANGE, except that NETIF_F_VLAN_CHALLENGED may be changed.
Care must be taken as changes to NETIF_F_VLAN_CHALLENGED won't affect already
configured VLANs.
* ndo_set_features:
@ -186,10 +200,14 @@ Redundancy) frames from one port to another in hardware.
* hsr-dup-offload
This should be set for devices which duplicate outgoing HSR (High-availability
Seamless Redundancy) or PRP (Parallel Redundancy Protocol) tags automatically
frames in hardware.
Seamless Redundancy) or PRP (Parallel Redundancy Protocol) frames
automatically in hardware.
* netmem-tx
Part V: Related device flags
============================
This should be set for devices which support netmem TX. See
Documentation/networking/netmem.rst
* netdev->netmem_tx
This is not a netdev feature bit. Drivers support netmem TX by setting
netdev->netmem_tx to one of the values in enum netmem_tx_mode.
See Documentation/networking/netmem.rst.

View File

@ -21,13 +21,14 @@ by free_netdev(). This is required to handle the pathological case cleanly
alloc_netdev_mqs() / alloc_netdev() reserve extra space for driver
private data which gets freed when the network device is freed. If
separately allocated data is attached to the network device
(netdev_priv()) then it is up to the module exit handler to free that.
(extra pointers stored in the device private struct) then it is up
to the module exit handler to free that.
There are two groups of APIs for registering struct net_device.
First group can be used in normal contexts where ``rtnl_lock`` is not already
held: register_netdev(), unregister_netdev().
Second group can be used when ``rtnl_lock`` is already held:
register_netdevice(), unregister_netdevice(), free_netdevice().
register_netdevice(), unregister_netdevice(), free_netdev().
Simple drivers
--------------
@ -58,6 +59,7 @@ the register_netdev(), and unregister_netdev() functions:
goto err_undo;
/* net_device is visible to the user! */
return 0;
err_undo:
/* ... undo the device setup ... */
@ -73,7 +75,7 @@ the register_netdev(), and unregister_netdev() functions:
Note that after calling register_netdev() the device is visible in the system.
Users can open it and start sending / receiving traffic immediately,
or run any other callback, so all initialization must be done prior to
or run any other callback, so all initialization must be **complete** prior to
registration.
unregister_netdev() closes the device and waits for all users to be done
@ -157,7 +159,7 @@ register_netdevice() fails. The callback may be invoked with or without
There is no explicit constructor callback, driver "constructs" the private
netdev state after allocating it and before registration.
Setting struct net_device.needs_free_netdev makes core call free_netdevice()
Setting struct net_device.needs_free_netdev makes core call free_netdev()
automatically after unregister_netdevice() when all references to the device
are gone. It only takes effect after a successful call to register_netdevice()
so if register_netdevice() fails driver is responsible for calling
@ -256,7 +258,7 @@ ndo_eth_ioctl:
lock if the driver implements queue management or shaper API.
Context: process
ndo_get_stats:
ndo_get_stats / ndo_get_stats64:
Synchronization: RCU (can be called concurrently with the stats
update path).
Context: atomic (can't sleep under RCU)
@ -264,12 +266,9 @@ ndo_get_stats:
ndo_start_xmit:
Synchronization: __netif_tx_lock spinlock.
When the driver sets dev->lltx this will be
called without holding netif_tx_lock. In this case the driver
has to lock by itself when needed.
The locking there should also properly protect against
set_rx_mode. WARNING: use of dev->lltx is deprecated.
Don't use it for new drivers.
When the driver sets dev->lltx this will be called without holding
netif_tx_lock. dev->lltx is meant for software drivers only, since
they often have no per-queue state.
Context: Process with BHs disabled or BH (timer),
will be called with interrupts disabled by netconsole.
@ -304,11 +303,15 @@ ndo_change_rx_flags:
lock if the driver implements queue management or shaper API.
ndo_setup_tc:
``TC_SETUP_BLOCK`` and ``TC_SETUP_FT`` are running under NFT locks
(i.e. no ``rtnl_lock`` and no device instance lock). The rest of
``tc_setup_type`` types run under netdev instance lock if the driver
Locking depends on ``tc_setup_type``. For most types the callback
is invoked under ``rtnl_lock`` and netdev instance lock if the driver
implements queue management or shaper API.
For ``TC_SETUP_BLOCK`` and ``TC_SETUP_FT`` ``rtnl_lock`` may or
may not be held, and the netdev instance lock is not held.
``TC_SETUP_BLOCK`` runs under ``block->cb_lock`` and ``TC_SETUP_FT``
runs under ``flowtable->flow_block_lock``.
Most ndo callbacks not specified in the list above are running
under ``rtnl_lock``. In addition, netdev instance lock is taken as well if
the driver implements queue management or shaper API.

View File

@ -14,10 +14,13 @@ to take advantage of segmentation offload capabilities of various NICs.
The following technologies are described:
* TCP Segmentation Offload - TSO
* UDP Fragmentation Offload - UFO
* UDP Segmentation Offload - USO
* IPIP, SIT, GRE, and UDP Tunnel Offloads
* Generic Segmentation Offload - GSO
* Generic Receive Offload - GRO
* Partial Generic Segmentation Offload - GSO_PARTIAL
* ESP Segmentation Offload
* Fraglist Generic Segmentation Offload - GSO_FRAGLIST
* SCTP acceleration with GSO - GSO_BY_FRAGS
@ -38,7 +41,8 @@ In order to support TCP segmentation offload it is necessary to populate
the network and transport header offsets of the skbuff so that the device
drivers will be able determine the offsets of the IP or IPv6 header and the
TCP header. In addition as CHECKSUM_PARTIAL is required csum_start should
also point to the TCP header of the packet.
also point to the TCP header of the packet, or to the inner transport header
for encapsulated TSO.
For IPv4 segmentation we support one of two types in terms of the IP ID.
The default behavior is to increment the IP ID with every segment. If the
@ -57,6 +61,10 @@ DF bit is not set on the outer header, in which case the device driver must
guarantee that the IP ID field is incremented in the outer header with every
segment.
SKB_GSO_TCP_ACCECN is a modifier used with TCP segmentation offload for
AccECN packets where the CWR bit must not be cleared during segmentation.
Devices advertise support for this using NETIF_F_GSO_ACCECN.
UDP Fragmentation Offload
=========================
@ -71,6 +79,16 @@ still receive them from tuntap and similar devices. Offload of UDP-based
tunnel protocols is still supported.
UDP Segmentation Offload
========================
UDP segmentation offload allows a device to segment a large UDP packet into
multiple UDP datagrams. Unlike UFO, these are not IP fragments. The payload
size of each datagram is specified in skb_shinfo()->gso_size and the GSO type
is SKB_GSO_UDP_L4. Devices advertise support for this using
NETIF_F_GSO_UDP_L4.
IPIP, SIT, GRE, UDP Tunnel, and Remote Checksum Offloads
========================================================
@ -154,6 +172,23 @@ that the IPv4 ID field is incremented in the case that a given header does
not have the DF bit set.
ESP Segmentation Offload
========================
ESP segmentation offload uses SKB_GSO_ESP to mark packets that require
IPsec ESP segmentation. This type is set by the XFRM output path for GSO
packets handled by ESP hardware offload.
Fraglist Generic Segmentation Offload
=====================================
Fraglist GSO uses SKB_GSO_FRAGLIST to mark packets whose segments are
already arranged as a list of skbs. The segmentation path splits the skb
based on that list rather than by creating segments of skb_shinfo()->gso_size
bytes from the linear and page-fragment data.
SCTP acceleration with GSO
===========================

View File

@ -29,9 +29,3 @@ dataref and headerless skbs
.. kernel-doc:: include/linux/skbuff.h
:doc: dataref and headerless skbs
Checksum information
--------------------
.. kernel-doc:: include/linux/skbuff.h
:doc: skb checksums

View File

@ -231,8 +231,19 @@ Kernel-internal data structures
-------------------------------
The following structures are internal to the kernel, their members are
translated to netlink attributes when dumped. Drivers must not overwrite
the statistics they don't report with 0.
translated to netlink attributes when dumped. Fields are pre-initialized
to ``ETHTOOL_STAT_NOT_SET`` (by ``ethtool_stats_init()``); drivers must
leave fields they do not report at that value rather than overwriting
them with 0.
- ethtool_pause_stats()
- ethtool_fec_stats()
- ``struct ethtool_eth_ctrl_stats``
- ``struct ethtool_eth_mac_stats``
- ``struct ethtool_eth_phy_stats``
- ``struct ethtool_fec_hist``
- ``struct ethtool_fec_stats``
- ``struct ethtool_link_ext_stats``
- ``struct ethtool_mm_stats``
- ``struct ethtool_pause_stats``
- ``struct ethtool_phy_stats``
- ``struct ethtool_rmon_stats``
- ``struct ethtool_ts_stats``