github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-06-08 06:25:52 +02:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph
Linux kernel source tree
1,140,264 Commits 1 Branch 934 Tags 20 GiB
C 97%
Assembly 1%
Shell 0.6%
Rust 0.5%
Python 0.4%
Other 0.3%
6003a74bd4
Go to file
Shawn Bohrer 6003a74bd4 veth: Fix race with AF_XDP exposing old or uninitialized descriptors 
[ Upstream commit fa349e396e ]

When AF_XDP is used on on a veth interface the RX ring is updated in two
steps.  veth_xdp_rcv() removes packet descriptors from the FILL ring
fills them and places them in the RX ring updating the cached_prod
pointer.  Later xdp_do_flush() syncs the RX ring prod pointer with the
cached_prod pointer allowing user-space to see the recently filled in
descriptors.  The rings are intended to be SPSC, however the existing
order in veth_poll allows the xdp_do_flush() to run concurrently with
another CPU creating a race condition that allows user-space to see old
or uninitialized descriptors in the RX ring.  This bug has been observed
in production systems.

To summarize, we are expecting this ordering:

CPU 0 __xsk_rcv_zc()
CPU 0 __xsk_map_flush()
CPU 2 __xsk_rcv_zc()
CPU 2 __xsk_map_flush()

But we are seeing this order:

CPU 0 __xsk_rcv_zc()
CPU 2 __xsk_rcv_zc()
CPU 0 __xsk_map_flush()
CPU 2 __xsk_map_flush()

This occurs because we rely on NAPI to ensure that only one napi_poll
handler is running at a time for the given veth receive queue.
napi_schedule_prep() will prevent multiple instances from getting
scheduled. However calling napi_complete_done() signals that this
napi_poll is complete and allows subsequent calls to
napi_schedule_prep() and __napi_schedule() to succeed in scheduling a
concurrent napi_poll before the xdp_do_flush() has been called.  For the
veth driver a concurrent call to napi_schedule_prep() and
__napi_schedule() can occur on a different CPU because the veth xmit
path can additionally schedule a napi_poll creating the race.

The fix as suggested by Magnus Karlsson, is to simply move the
xdp_do_flush() call before napi_complete_done().  This syncs the
producer ring pointers before another instance of napi_poll can be
scheduled on another CPU.  It will also slightly improve performance by
moving the flush closer to when the descriptors were placed in the
RX ring.

Fixes: d1396004dd ("veth: Add XDP TX and REDIRECT")
Suggested-by: Magnus Karlsson <magnus.karlsson@gmail.com>
Signed-off-by: Shawn Bohrer <sbohrer@cloudflare.com>
Link: https://lore.kernel.org/r/20221220185903.1105011-1-sbohrer@cloudflare.com
Signed-off-by: Paolo Abeni <pabeni@redhat.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
2023-01-12 12:01:59 +01:00
arch ARM: renumber bits related to _TIF_WORK_MASK 2023-01-12 12:01:55 +01:00
block block: mq-deadline: Do not break sequential write streams to zoned HDDs 2023-01-07 11:11:51 +01:00
certs certs: make system keyring depend on built-in x509 parser 2022-09-24 04:31:18 +09:00
crypto crypto: tcrypt - Fix multibuffer skcipher speed test mem leak 2022-12-31 13:32:34 +01:00
Documentation ext4: journal_path mount options should follow links 2023-01-07 11:11:59 +01:00
drivers veth: Fix race with AF_XDP exposing old or uninitialized descriptors 2023-01-12 12:01:59 +01:00
fs btrfs: fix an error handling path in btrfs_defrag_leaves() 2023-01-12 12:01:56 +01:00
include netfilter: nf_tables: honor set timeout and garbage collection updates 2023-01-12 12:01:58 +01:00
init init/Kconfig: fix CC_HAS_ASM_GOTO_TIED_OUTPUT test with dash 2022-11-22 22:42:38 +09:00
io_uring io_uring: pass in EPOLL_URING_WAKE for eventfd signaling and wakeups 2023-01-04 11:28:48 +01:00
ipc ipc: fix memory leak in init_mqueue_fs() 2022-12-31 13:32:01 +01:00
kernel tracing: Fix infinite loop in tracing_read_pipe on overflowed print_trace_line 2023-01-07 11:11:49 +01:00
lib test_kprobes: Fix implicit declaration error of test_kprobes 2023-01-07 11:11:55 +01:00
LICENSES LICENSES/LGPL-2.1: Add LGPL-2.1-or-later as valid identifiers 2021-12-16 14:33:10 +01:00
mm hugetlb: really allocate vma lock for all sharable vmas 2023-01-07 11:11:55 +01:00
net netfilter: nf_tables: honor set timeout and garbage collection updates 2023-01-12 12:01:58 +01:00
rust Kbuild: add Rust support 2022-09-28 09:02:20 +02:00
samples samples: vfio-mdev: Fix missing pci_disable_device() in mdpy_fb_probe() 2022-12-31 13:32:42 +01:00
scripts - Handle different output of readelf on different distros running 2022-11-27 12:08:17 -08:00
security device_cgroup: Roll back to original exceptions after copy failure 2023-01-07 11:11:56 +01:00
sound ASoC: jz4740-i2s: Handle independent FIFO flush bits 2023-01-07 11:11:51 +01:00
tools perf probe: Fix to get the DW_AT_decl_file and DW_AT_call_file as unsinged data 2023-01-12 12:01:56 +01:00
usr usr/gen_init_cpio.c: remove unnecessary -1 values from int file 2022-10-03 14:21:44 -07:00
virt Merge branch 'kvm-dwmw2-fixes' into HEAD 2022-11-23 18:59:45 -05:00
.clang-format inet: ping: use hlist_nulls rcu iterator during lookup 2022-12-01 12:42:46 +01:00
.cocciconfig scripts: add Linux .cocciconfig for coccinelle 2016-07-22 12:13:39 +02:00
.get_maintainer.ignore get_maintainer: add Alan to .get_maintainer.ignore 2022-08-20 15:17:44 -07:00
.gitattributes .gitattributes: use 'dts' diff driver for dts files 2019-12-04 19:44:11 -08:00
.gitignore Kbuild: add Rust support 2022-09-28 09:02:20 +02:00
.mailmap 9 hotfixes. 6 for MM, 3 for other areas. Four of these patches address 2022-12-10 17:10:52 -08:00
.rustfmt.toml rust: add .rustfmt.toml 2022-09-28 09:02:20 +02:00
COPYING COPYING: state that all contributions really are covered by this file 2020-02-10 13:32:20 -08:00
CREDITS MAINTAINERS: Remove Michal Marek from Kbuild maintainers 2022-11-16 14:53:00 +09:00
Kbuild Kbuild updates for v6.1 2022-10-10 12:00:45 -07:00
Kconfig kbuild: ensure full rebuild when the compiler is updated 2020-05-12 13:28:33 +09:00
MAINTAINERS 9 hotfixes. 6 for MM, 3 for other areas. Four of these patches address 2022-12-10 17:10:52 -08:00
Makefile Linux 6.1.4 2023-01-07 11:12:04 +01:00
README Drop all 00-INDEX files from Documentation/ 2018-09-09 15:08:58 -06:00

README 

Linux kernel
============

There are several guides for kernel developers and users. These guides can
be rendered in a number of formats, like HTML and PDF. Please read
Documentation/admin-guide/README.rst first.

In order to build the documentation, use ``make htmldocs`` or
``make pdfdocs``.  The formatted documentation can also be read online at:

    https://www.kernel.org/doc/html/latest/

There are various text files in the Documentation/ subdirectory,
several of them using the Restructured Text markup notation.

Please read the Documentation/process/changes.rst file, as it contains the
requirements for building and running the kernel, and information about
the problems which may result by upgrading your kernel.