diff --git a/.mailmap b/.mailmap index 01453c5e1ab4..8079e9560748 100644 --- a/.mailmap +++ b/.mailmap @@ -171,6 +171,7 @@ Boris Brezillon Boris Brezillon Boris Brezillon Brendan Higgins +Brendan Jackman Brian Avery Brian Cain Brian Cain @@ -233,6 +234,8 @@ Daniel Lezcano Daniel Lezcano Daniel Thompson Daniele Alessandrelli +Danila Tikhonov +Danila Tikhonov Danilo Krummrich David Brownell David Collins diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst index 14b8c571c0d1..aed195a71cbf 100644 --- a/Documentation/admin-guide/cgroup-v2.rst +++ b/Documentation/admin-guide/cgroup-v2.rst @@ -2239,9 +2239,12 @@ IO Latency ~~~~~~~~~~ This is a cgroup v2 controller for IO workload protection. You provide a group -with a latency target, and if the average latency exceeds that target the -controller will throttle any peers that have a lower latency target than the -protected workload. +with a latency target, and if the group misses its target the controller will +throttle any peers that have a lower latency target than the protected +workload. How a miss is detected depends on the device: on rotational devices +the average latency over the window must exceed the target, while on +non-rotational devices a miss is counted once enough of the IOs in the window +individually exceed the target. The limits are only applied at the peer level in the hierarchy. This means that in the diagram below, only groups A, B, and C will influence each other, and @@ -2258,10 +2261,12 @@ So the ideal way to configure this is to set io.latency in groups A, B, and C. Generally you do not want to set a value lower than the latency your device supports. Experiment to find the value that works best for your workload. Start at higher than the expected latency for your device and, with -blkcg_debug_stats enabled, watch the avg_lat value in io.stat for your -workload group to get an idea of the latency you see during normal operation. -Use the avg_lat value as a basis for your real setting, setting at 10-15% -higher than the value in io.stat. +blkcg_debug_stats enabled, observe io.stat for your workload group to get an +idea of the latency you see during normal operation. On rotational devices, +use the avg_lat value as a basis for your real setting, setting it 10-15% +higher. On non-rotational devices io.stat reports no average latency; set +the target based on your device and use the missed/total fields to verify it +is being met. How IO Latency Throttling Works ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -2303,19 +2308,36 @@ IO Latency Interface Files the blkcg_debug_stats module parameter is enabled (it is disabled by default). + The reported latency fields depend on the device. Rotational devices + report avg_lat and win; non-rotational devices report missed and total + instead. missed and total are live counters for the current window and + may change between reads. + depth This is the current queue depth for the group. avg_lat - This is an exponential moving average with a decay rate of 1/exp - bound by the sampling interval. The decay rate interval can be - calculated by multiplying the win value in io.stat by the - corresponding number of samples based on the win value. + (Rotational devices only.) This is an exponential moving + average with a decay rate of 1/exp bound by the sampling + interval. The decay rate interval can be calculated by + multiplying the win value in io.stat by the corresponding number + of samples based on the win value. win - The sampling window size in milliseconds. This is the minimum - duration of time between evaluation events. Windows only elapse - with IO activity. Idle periods extend the most recent window. + (Rotational devices only.) The sampling window size in + milliseconds. This is the minimum duration of time between + evaluation events. Windows only elapse with IO activity. Idle + periods extend the most recent window. + + missed + (Non-rotational devices only.) The number of IOs in the + current window whose latency exceeded the target. A group is + considered to be missing its target once missed reaches a + certain ratio of total. + + total + (Non-rotational devices only.) The total number of IOs + accounted in the current window. IO Priority ~~~~~~~~~~~ diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index b5493a7f8f22..3d35270dddef 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -8169,6 +8169,11 @@ Kernel parameters q = USB_QUIRK_FORCE_ONE_CONFIG (Device claims zero configurations, forcing to 1); + r = USB_QUIRK_WINDOWS_CONFIG_REQ_SIZE (Device + fails during initialization when asked for + 9-bytes configuration descriptor request. + Ask for 255-bytes request instead to mirror + Windows' behavior); Example: quirks=0781:5580:bk,0a5c:5834:gij usbhid.mousepoll= diff --git a/Documentation/netlink/specs/ovpn.yaml b/Documentation/netlink/specs/ovpn.yaml index b0c782e59a32..ac50d1d7c00a 100644 --- a/Documentation/netlink/specs/ovpn.yaml +++ b/Documentation/netlink/specs/ovpn.yaml @@ -118,12 +118,16 @@ attribute-sets: doc: >- The number of seconds after which a keep alive message is sent to the peer + checks: + max: 86400 - name: keepalive-timeout type: u32 doc: >- The number of seconds from the last activity after which the peer is assumed dead + checks: + max: 86400 - name: del-reason type: u32 diff --git a/Documentation/process/coding-assistants.rst b/Documentation/process/coding-assistants.rst index 899f4459c52d..e71df7d28467 100644 --- a/Documentation/process/coding-assistants.rst +++ b/Documentation/process/coding-assistants.rst @@ -57,3 +57,40 @@ Basic development tools (git, gcc, make, editors) should not be listed. Example:: Assisted-by: Claude:claude-3-opus coccinelle sparse + +Procedure for finding and fixing bugs +===================================== + +When an AI assistant is used to find and fix bugs, it **MUST** follow at least +these steps: + +1. Before starting, read the whole process documentation listed above, as well + as any other document mentioned in the request. Do not rely on isolated + parts found by keyword search. +2. Note the commit ID and Locate a bug as instructed. +3. For any bug found that is not trivial, verify that it looks real by + attempting to create a reproducer to demonstrate it. Lacking it may cause + the report to be ignored, as many unverified bug reports sent to maintainers + happen to be invalid. Stop here if it finally looks wrong. +4. Write a fix for the bug. This part is not optional: except in a few very + rare cases, an AI assistant able to find a bug is able to fix it. Note that + fixes written in the same session as used to find the bug will generally + lead to better and more accurate fixes as the LLM's reasoning context + remains present. +5. Build and verify that the fix works either using the reproducer or by + re-running a complete analysis; drop any fix that doesn't work and try + another one. The fix must not add build warnings and must pass the + checkpatch.pl checks (see submitting-patches.rst). +6. Commit the working fix with a detailed message describing the problem, the + solution and a Fixes tag. Do not add a Signed-off-by tag, and add an + Assisted-by tag, as described above. +7. Identify the maintainers and lists using scripts/get_maintainer.pl. + Documentation/process/security-bugs.rst shows how to do that. +8. Indicate what could not be done. If the fix could not be built or tested, or + if no reproducer could be produced, say so explicitly: maintainers currently + waste too much time analyzing unverified reports and untested fixes. +9. Read Documentation/process/threat-model.rst to determine whether the bug is + a vulnerability or a regular bug, and leave the result to the reporter for + review (the assistant must never send anything itself). Regular bugs are + submitted as described in Documentation/process/submitting-patches.rst, + vulnerabilities as described in Documentation/process/security-bugs.rst. diff --git a/Documentation/process/security-bugs.rst b/Documentation/process/security-bugs.rst index 3c51ddde31dd..c7dfecc87764 100644 --- a/Documentation/process/security-bugs.rst +++ b/Documentation/process/security-bugs.rst @@ -18,6 +18,10 @@ information is helpful. The following information are absolutely necessary in **any** security bug report: * **affected kernel version range**: with no version indication, your report + will not be processed. Note that a time-dependent version such as "latest + mainline" is not acceptable. A stable identifier such as a commit ID or an + exact version is required. Versions designating kernels not coming from + kernel.org (such as distro kernels) are meaningless to maintainers and will not be processed. A significant part of reports are for bugs that have already been fixed, so it is extremely important that vulnerabilities are verified on recent versions (development tree or latest stable @@ -225,6 +229,28 @@ there is no need to consume a maintainer's time with an unimportant report. If the issue is clearly trivial and publicly discoverable, you should report it directly to the public mailing lists. +At the very least, when using an AI assistant to find and report bugs, make the +assistant read this file and threat-model.rst before proceeding, and have it +follow the procedure described in coding-assistants.rst: work on an up-to-date +mainline tree and note the commit ID, verify the bug is real, write a fix, +build it warning-free and checkpatch-clean, commit it with a Fixes tag, and +identify the maintainers with get_maintainer.pl. + +On top of that procedure, the AI assistant **MUST**: + +1. Prepare a plain-text report explaining the problem. It must contain the + four items listed at the top of this file as absolutely necessary: the + affected version or commit ID noted while following the procedure above, + the description of the problem, the reproducer or its status, and the + triggering conditions. +2. Start the report with a temporary section listing the recipients' addresses + (maintainers+list for the patch, maintainers only for the report and + reproducer), and with instructions reminding the reporter to check that + their email client is properly setup (see email-clients.rst), and leave it + to the reporter to remove that temporary section. + +A more detailed process is covered at https://github.com/masoncl/kres.git. + Sending the report ------------------ diff --git a/Documentation/process/threat-model.rst b/Documentation/process/threat-model.rst index 9dd8011dde82..a68be888ce8e 100644 --- a/Documentation/process/threat-model.rst +++ b/Documentation/process/threat-model.rst @@ -98,11 +98,11 @@ measures whose purpose is to avoid crossing a security boundary when certain classes of bugs are found, but a failure of these extra protections do not constitute a vulnerability alone. -What does not constitute a security bug ---------------------------------------- +What classes of problems are not considered vulnerabilities +----------------------------------------------------------- In the Linux kernel's threat model, the following classes of problems are -**NOT** considered as Linux Kernel security bugs. However, when it is believed +**NOT** considered Linux Kernel vulnerabilities. However, when it is believed that the kernel could do better, they should be reported, so that they can be reviewed and fixed where reasonably possible, but they will be handled as any regular bug: @@ -111,8 +111,8 @@ regular bug: * outdated kernels and particularly end-of-life branches are out of the scope of the kernel's threat model: administrators are responsible for keeping - their system up to date. For a bug to qualify as a security bug, it must be - demonstrated that it affects actively maintained versions. + their system up to date. For a bug to qualify as a vulnerability, it must + be demonstrated that it affects actively maintained versions. * build-level: changes to the kernel configuration that are explicitly documented as lowering the security level (e.g. ``CONFIG_NOMMU``), or @@ -178,9 +178,6 @@ regular bug: involving tens of millions of threads, tens of thousands of CPUs, unrealistic CPU frequencies, RAM sizes or disk capacities, network speeds). - * issues whose reproduction requires hardware modification or emulation, - including fake USB devices that pretend to be another one. - * as well as issues that can be triggered at a cost that is orders of magnitude higher than the expected benefits (e.g. fully functional keyboard emulator only to retrieve 7 uninitialized bytes in a structure, or @@ -208,19 +205,27 @@ regular bug: messages. * Leaks of kernel memory addresses/pointers do not constitute an immediately - exploitable vector and are not security bugs, though they must be reported - and fixed. + exploitable vector and are not vulnerabilities, though they must be + reported and fixed. -* **Crafted file system images**: +* **Non-conforming devices and media**: + + Drivers are implemented against a specification. When a device or a storage + medium violates the specification its driver was written against, the + resulting misbehaviour is a regular bug to be fixed, not a vulnerability, + unless the driver is specifically documented as being hardened against + hostile inputs. The following are therefore not considered vulnerabilities: * bugs triggered by mounting a corrupted or maliciously crafted file system - image are generally not security bugs, as the kernel assumes the underlying - storage media is under the administrator's control, unless the filesystem - driver is specifically documented as being hardened against untrusted media. - - * issues that are resolved, mitigated, or detected by running a filesystem + image: mounting a block device is a privileged operation (see above), and + the administrator is responsible for the media they mount. This includes + issues that are resolved, mitigated, or detected by running a filesystem consistency check (fsck) on the image prior to mounting. + * bugs whose reproduction requires hardware modification or emulation, + including fake USB devices that pretend to be another one, or devices + reporting values outside their documented ranges. + * **Physical access**: Issues that require physical access to the machine, hardware modification, or @@ -232,4 +237,4 @@ regular bug: * **Functional and performance regressions**: Any issue that can be mitigated by setting proper permissions and limits - doesn't qualify as a security bug. + doesn't qualify as a vulnerability. diff --git a/MAINTAINERS b/MAINTAINERS index a21c79f3e436..8014b9f8253e 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -2676,6 +2676,8 @@ F: drivers/irqchip/irq-aspeed-i2c-ic.c ARM/ASPEED MACHINE SUPPORT M: Joel Stanley M: Andrew Jeffery +R: Ryan Chen +R: Billy Tsai L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) L: linux-aspeed@lists.ozlabs.org (moderated for non-subscribers) S: Supported @@ -2881,7 +2883,7 @@ W: http://www.armlinux.org.uk/ F: arch/arm/include/asm/hardware/dec21285.h F: arch/arm/mach-footbridge/ -ARM/FREESCALE IMX / MXC ARM ARCHITECTURE +ARM/FREESCALE IMX / MXC / LAYERSCAPE ARM ARCHITECTURE M: Frank Li M: Sascha Hauer R: Pengutronix Kernel Team @@ -2895,22 +2897,11 @@ F: Documentation/devicetree/bindings/firmware/nxp* F: arch/arm/boot/dts/nxp/ F: arch/arm64/boot/dts/freescale/ X: Documentation/devicetree/bindings/media/i2c/ -X: arch/arm64/boot/dts/freescale/fsl-* -X: arch/arm64/boot/dts/freescale/qoriq-* X: drivers/media/i2c/ N: imx N: mxs N: \bmxc[^\d] -ARM/FREESCALE LAYERSCAPE ARM ARCHITECTURE -M: Frank Li -L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers) -S: Maintained -T: git git://git.kernel.org/pub/scm/linux/kernel/git/frank.li/linux.git -F: arch/arm/boot/dts/nxp/ls/ -F: arch/arm64/boot/dts/freescale/fsl-* -F: arch/arm64/boot/dts/freescale/qoriq-* - ARM/FREESCALE VYBRID ARM ARCHITECTURE M: Frank Li M: Sascha Hauer @@ -10134,6 +10125,11 @@ F: drivers/base/firmware_loader/ F: rust/kernel/firmware.rs F: include/linux/firmware.h +FIXED-LAYOUT NVMEM LAYOUT DRIVER +M: Mathieu Dubois-Briand +S: Maintained +F: drivers/nvmem/layouts/fixed-layout.c + FLEXTIMER FTM-QUADDEC DRIVER M: Patrick Havelange L: linux-iio@vger.kernel.org @@ -11743,6 +11739,7 @@ F: drivers/net/ethernet/hisilicon/hibmcge/ HISILICON NETWORK SUBSYSTEM DRIVER M: Jian Shen +M: Jijie Shao L: netdev@vger.kernel.org S: Maintained W: http://www.hisilicon.com @@ -17160,7 +17157,7 @@ M: Andrew Morton M: Vlastimil Babka R: Suren Baghdasaryan R: Michal Hocko -R: Brendan Jackman +R: Brendan Jackman R: Johannes Weiner R: Zi Yan L: linux-mm@kvack.org @@ -17846,7 +17843,7 @@ F: drivers/net/wireless/microchip/ MICROCHIP ZL3073X DRIVER M: Ivan Vecera -M: Prathosh Satish +M: Min Li L: netdev@vger.kernel.org S: Supported F: Documentation/devicetree/bindings/dpll/microchip,zl30731.yaml diff --git a/Makefile b/Makefile index 902f3f3d54b7..0e6fc0803775 100644 --- a/Makefile +++ b/Makefile @@ -2,7 +2,7 @@ VERSION = 7 PATCHLEVEL = 2 SUBLEVEL = 0 -EXTRAVERSION = -rc6 +EXTRAVERSION = -rc7 NAME = Baby Opossum Posse # *DOCUMENTATION* diff --git a/arch/arm/boot/dts/broadcom/bcm-ns.dtsi b/arch/arm/boot/dts/broadcom/bcm-ns.dtsi index 392a25713669..bd52de0faa3e 100644 --- a/arch/arm/boot/dts/broadcom/bcm-ns.dtsi +++ b/arch/arm/boot/dts/broadcom/bcm-ns.dtsi @@ -141,7 +141,7 @@ axi@18000000 { /* PCIe Controller 2 */ <0x00014000 0 &gic GIC_SPI 138 IRQ_TYPE_LEVEL_HIGH>, - <0x00014000 1 &gic GIC_SPI 138 IRQ_TYPE_LEVEL_HIGH>, + <0x00014000 1 &gic GIC_SPI 139 IRQ_TYPE_LEVEL_HIGH>, <0x00014000 2 &gic GIC_SPI 140 IRQ_TYPE_LEVEL_HIGH>, <0x00014000 3 &gic GIC_SPI 141 IRQ_TYPE_LEVEL_HIGH>, <0x00014000 4 &gic GIC_SPI 142 IRQ_TYPE_LEVEL_HIGH>, diff --git a/arch/arm/boot/dts/broadcom/bcm4709-linksys-ea9200.dts b/arch/arm/boot/dts/broadcom/bcm4709-linksys-ea9200.dts index 5bbc2ba0f959..87569408bb69 100644 --- a/arch/arm/boot/dts/broadcom/bcm4709-linksys-ea9200.dts +++ b/arch/arm/boot/dts/broadcom/bcm4709-linksys-ea9200.dts @@ -26,7 +26,7 @@ memory@0 { nvram@1c080000 { compatible = "brcm,nvram"; - reg = <0x1c080000 0x180000>; + reg = <0x1c080000 0x100000>; et2macaddr: et2macaddr { #nvmem-cell-cells = <1>; diff --git a/arch/arm/mach-npcm/platsmp.c b/arch/arm/mach-npcm/platsmp.c index 41891d3aa124..4c1fc9983746 100644 --- a/arch/arm/mach-npcm/platsmp.c +++ b/arch/arm/mach-npcm/platsmp.c @@ -32,6 +32,7 @@ static int npcm7xx_smp_boot_secondary(unsigned int cpu, goto out; } gcr_base = of_iomap(gcr_np, 0); + of_node_put(gcr_np); if (!gcr_base) { pr_err("could not iomap gcr"); ret = -ENOMEM; @@ -60,6 +61,7 @@ static void __init npcm7xx_smp_prepare_cpus(unsigned int max_cpus) return; } scu_base = of_iomap(scu_np, 0); + of_node_put(scu_np); if (!scu_base) { pr_err("could not iomap scu"); return; diff --git a/arch/arm64/boot/dts/broadcom/bcm2712.dtsi b/arch/arm64/boot/dts/broadcom/bcm2712.dtsi index 761c59d90ffc..09ff5e9959d3 100644 --- a/arch/arm64/boot/dts/broadcom/bcm2712.dtsi +++ b/arch/arm64/boot/dts/broadcom/bcm2712.dtsi @@ -678,8 +678,6 @@ IRQ_TYPE_LEVEL_LOW)>, , , - ; }; diff --git a/arch/arm64/boot/dts/qcom/eliza.dtsi b/arch/arm64/boot/dts/qcom/eliza.dtsi index 977de44b816e..72b786fec195 100644 --- a/arch/arm64/boot/dts/qcom/eliza.dtsi +++ b/arch/arm64/boot/dts/qcom/eliza.dtsi @@ -2559,7 +2559,7 @@ mdss_dsi1_phy: phy@ae97000 { "dsi_pll"; clocks = <&dispcc DISP_CC_MDSS_AHB_CLK>, - <&rpmhcc RPMH_CXO_CLK>; + <&bi_tcxo_div2>; clock-names = "iface", "ref"; diff --git a/arch/arm64/boot/dts/qcom/glymur.dtsi b/arch/arm64/boot/dts/qcom/glymur.dtsi index 20b49af7298e..129be417ac42 100644 --- a/arch/arm64/boot/dts/qcom/glymur.dtsi +++ b/arch/arm64/boot/dts/qcom/glymur.dtsi @@ -1876,7 +1876,7 @@ &config_noc SLAVE_QUP_0 QCOM_ICC_TAG_ALWAYS>, spi0: spi@b80000 { compatible = "qcom,geni-spi"; reg = <0x0 0x00b80000 0x0 0x4000>; - interrupts = ; + interrupts = ; clocks = <&gcc GCC_QUPV3_WRAP0_S0_CLK>; clock-names = "se"; interconnects = <&clk_virt MASTER_QUP_CORE_0 QCOM_ICC_TAG_ALWAYS @@ -1903,7 +1903,7 @@ &config_noc SLAVE_QUP_0 QCOM_ICC_TAG_ALWAYS>, i2c1: i2c@b84000 { compatible = "qcom,geni-i2c"; reg = <0x0 0x00b84000 0x0 0x4000>; - interrupts = ; + interrupts = ; clocks = <&gcc GCC_QUPV3_WRAP0_S1_CLK>; clock-names = "se"; interconnects = <&clk_virt MASTER_QUP_CORE_0 QCOM_ICC_TAG_ALWAYS @@ -1930,7 +1930,7 @@ &config_noc SLAVE_QUP_0 QCOM_ICC_TAG_ALWAYS>, spi1: spi@b84000 { compatible = "qcom,geni-spi"; reg = <0x0 0x00b84000 0x0 0x4000>; - interrupts = ; + interrupts = ; clocks = <&gcc GCC_QUPV3_WRAP0_S1_CLK>; clock-names = "se"; interconnects = <&clk_virt MASTER_QUP_CORE_0 QCOM_ICC_TAG_ALWAYS @@ -1957,7 +1957,7 @@ &config_noc SLAVE_QUP_0 QCOM_ICC_TAG_ALWAYS>, i2c2: i2c@b88000 { compatible = "qcom,geni-i2c"; reg = <0x0 0x00b88000 0x0 0x4000>; - interrupts = ; + interrupts = ; clocks = <&gcc GCC_QUPV3_WRAP0_S2_CLK>; clock-names = "se"; interconnects = <&clk_virt MASTER_QUP_CORE_0 QCOM_ICC_TAG_ALWAYS @@ -1984,7 +1984,7 @@ &config_noc SLAVE_QUP_0 QCOM_ICC_TAG_ALWAYS>, spi2: spi@b88000 { compatible = "qcom,geni-spi"; reg = <0x0 0x00b88000 0x0 0x4000>; - interrupts = ; + interrupts = ; clocks = <&gcc GCC_QUPV3_WRAP0_S2_CLK>; clock-names = "se"; interconnects = <&clk_virt MASTER_QUP_CORE_0 QCOM_ICC_TAG_ALWAYS @@ -2011,7 +2011,7 @@ &config_noc SLAVE_QUP_0 QCOM_ICC_TAG_ALWAYS>, uart2: serial@b88000 { compatible = "qcom,geni-uart"; reg = <0x0 0x00b88000 0x0 0x4000>; - interrupts = ; + interrupts = ; clocks = <&gcc GCC_QUPV3_WRAP0_S2_CLK>; clock-names = "se"; interconnects = <&clk_virt MASTER_QUP_CORE_0 QCOM_ICC_TAG_ALWAYS @@ -2056,7 +2056,7 @@ &config_noc SLAVE_QUP_0 QCOM_ICC_TAG_ALWAYS>, spi3: spi@b8c000 { compatible = "qcom,geni-spi"; reg = <0x0 0x00b8c000 0x0 0x4000>; - interrupts = ; + interrupts = ; clocks = <&gcc GCC_QUPV3_WRAP0_S3_CLK>; clock-names = "se"; interconnects = <&clk_virt MASTER_QUP_CORE_0 QCOM_ICC_TAG_ALWAYS @@ -2110,7 +2110,7 @@ &config_noc SLAVE_QUP_0 QCOM_ICC_TAG_ALWAYS>, spi4: spi@b90000 { compatible = "qcom,geni-spi"; reg = <0x0 0x00b90000 0x0 0x4000>; - interrupts = ; + interrupts = ; clocks = <&gcc GCC_QUPV3_WRAP0_S4_CLK>; clock-names = "se"; interconnects = <&clk_virt MASTER_QUP_CORE_0 QCOM_ICC_TAG_ALWAYS @@ -2164,7 +2164,7 @@ &config_noc SLAVE_QUP_0 QCOM_ICC_TAG_ALWAYS>, spi5: spi@b94000 { compatible = "qcom,geni-spi"; reg = <0x0 0x00b94000 0x0 0x4000>; - interrupts = ; + interrupts = ; clocks = <&gcc GCC_QUPV3_WRAP0_S5_CLK>; clock-names = "se"; interconnects = <&clk_virt MASTER_QUP_CORE_0 QCOM_ICC_TAG_ALWAYS @@ -2191,7 +2191,7 @@ &config_noc SLAVE_QUP_0 QCOM_ICC_TAG_ALWAYS>, i2c6: i2c@b98000 { compatible = "qcom,geni-i2c"; reg = <0x0 0x00b98000 0x0 0x4000>; - interrupts = ; + interrupts = ; clocks = <&gcc GCC_QUPV3_WRAP0_S6_CLK>; clock-names = "se"; interconnects = <&clk_virt MASTER_QUP_CORE_0 QCOM_ICC_TAG_ALWAYS @@ -2218,7 +2218,7 @@ &config_noc SLAVE_QUP_0 QCOM_ICC_TAG_ALWAYS>, spi6: spi@b98000 { compatible = "qcom,geni-spi"; reg = <0x0 0x00b98000 0x0 0x4000>; - interrupts = ; + interrupts = ; clocks = <&gcc GCC_QUPV3_WRAP0_S6_CLK>; clock-names = "se"; interconnects = <&clk_virt MASTER_QUP_CORE_0 QCOM_ICC_TAG_ALWAYS @@ -2245,7 +2245,7 @@ &config_noc SLAVE_QUP_0 QCOM_ICC_TAG_ALWAYS>, i2c7: i2c@b9c000 { compatible = "qcom,geni-i2c"; reg = <0x0 0x00b9c000 0x0 0x4000>; - interrupts = ; + interrupts = ; clocks = <&gcc GCC_QUPV3_WRAP0_S7_CLK>; clock-names = "se"; interconnects = <&clk_virt MASTER_QUP_CORE_0 QCOM_ICC_TAG_ALWAYS @@ -2272,7 +2272,7 @@ &config_noc SLAVE_QUP_0 QCOM_ICC_TAG_ALWAYS>, spi7: spi@b9c000 { compatible = "qcom,geni-spi"; reg = <0x0 0x00b9c000 0x0 0x4000>; - interrupts = ; + interrupts = ; clocks = <&gcc GCC_QUPV3_WRAP0_S7_CLK>; clock-names = "se"; interconnects = <&clk_virt MASTER_QUP_CORE_0 QCOM_ICC_TAG_ALWAYS @@ -6874,9 +6874,9 @@ apps_smmu: iommu@15000000 { pcie_smmu: iommu@15480000 { compatible = "arm,smmu-v3"; reg = <0x0 0x15480000 0x0 0x20000>; - interrupts = , - , - ; + interrupts = , + , + ; interrupt-names = "eventq", "cmdq-sync", "gerror"; dma-coherent; #iommu-cells = <1>; diff --git a/arch/arm64/boot/dts/qcom/monaco.dtsi b/arch/arm64/boot/dts/qcom/monaco.dtsi index e4c8466f941b..a350a8ba48fa 100644 --- a/arch/arm64/boot/dts/qcom/monaco.dtsi +++ b/arch/arm64/boot/dts/qcom/monaco.dtsi @@ -7384,6 +7384,7 @@ intc: interrupt-controller@17a00000 { interrupt-controller; #redistributor-regions = <1>; redistributor-stride = <0x0 0x20000>; + #address-cells = <0>; }; watchdog@17c10000 { diff --git a/arch/arm64/boot/dts/qcom/purwa.dtsi b/arch/arm64/boot/dts/qcom/purwa.dtsi index 9ab4f26b35f2..5b17840fb62f 100644 --- a/arch/arm64/boot/dts/qcom/purwa.dtsi +++ b/arch/arm64/boot/dts/qcom/purwa.dtsi @@ -47,6 +47,8 @@ &gmu { &gpu { compatible = "qcom,adreno-43030c00", "qcom,adreno"; + iommus = <&adreno_smmu 0 0x0>; + nvmem-cells = <&gpu_speed_bin>; nvmem-cell-names = "speed_bin"; diff --git a/arch/arm64/boot/dts/qcom/sc8280xp-huawei-gaokun3.dts b/arch/arm64/boot/dts/qcom/sc8280xp-huawei-gaokun3.dts index f3c00be67081..cd575e20d90a 100644 --- a/arch/arm64/boot/dts/qcom/sc8280xp-huawei-gaokun3.dts +++ b/arch/arm64/boot/dts/qcom/sc8280xp-huawei-gaokun3.dts @@ -637,7 +637,7 @@ embedded-controller@38 { compatible = "huawei,gaokun3-ec"; reg = <0x38>; - interrupts-extended = <&tlmm 107 IRQ_TYPE_LEVEL_LOW>; + interrupts-extended = <&tlmm 103 IRQ_TYPE_LEVEL_LOW>; #address-cells = <1>; #size-cells = <0>; diff --git a/arch/arm64/boot/dts/qcom/sc8280xp.dtsi b/arch/arm64/boot/dts/qcom/sc8280xp.dtsi index a2bd6b10e475..b0de9e262f29 100644 --- a/arch/arm64/boot/dts/qcom/sc8280xp.dtsi +++ b/arch/arm64/boot/dts/qcom/sc8280xp.dtsi @@ -5356,7 +5356,7 @@ pdc: interrupt-controller@b220000 { <211 699 1>, <212 705 1>, <213 450 1>, - <214 643 1>, + <214 643 2>, <216 646 5>, <221 390 5>, <226 700 3>, @@ -5379,7 +5379,7 @@ pdc: interrupt-controller@b220000 { <252 798 1>, <253 765 1>, <254 763 1>, - <255 454 1>, + <255 454 3>, <258 139 1>, <259 786 2>, <261 370 2>, diff --git a/arch/arm64/boot/dts/qcom/sdm850-lenovo-yoga-c630.dts b/arch/arm64/boot/dts/qcom/sdm850-lenovo-yoga-c630.dts index e41200839dbe..8cdaef5360d1 100644 --- a/arch/arm64/boot/dts/qcom/sdm850-lenovo-yoga-c630.dts +++ b/arch/arm64/boot/dts/qcom/sdm850-lenovo-yoga-c630.dts @@ -347,6 +347,10 @@ &cdsp_pas { status = "okay"; }; +&cluster_sleep_0 { + arm,psci-suspend-param = <0x41008244>; +}; + &gcc { protected-clocks = , , diff --git a/arch/arm64/boot/dts/qcom/sm8650.dtsi b/arch/arm64/boot/dts/qcom/sm8650.dtsi index 160ead25ecf7..65c4a2b46d21 100644 --- a/arch/arm64/boot/dts/qcom/sm8650.dtsi +++ b/arch/arm64/boot/dts/qcom/sm8650.dtsi @@ -7091,8 +7091,8 @@ sram@14680000 { ranges = <0 0 0x14680000 0x2c000>; - ipa_modem_tables: modem-tables@8000 { - reg = <0x8000 0x2000>; + ipa_modem_tables: modem-tables@3000 { + reg = <0x3000 0x2000>; }; }; diff --git a/arch/arm64/include/asm/ptdump.h b/arch/arm64/include/asm/ptdump.h index 5b374a6ab34a..50a195eda8ed 100644 --- a/arch/arm64/include/asm/ptdump.h +++ b/arch/arm64/include/asm/ptdump.h @@ -7,8 +7,6 @@ #include -DECLARE_STATIC_KEY_FALSE(arm64_ptdump_lock_key); - #ifdef CONFIG_PTDUMP #include diff --git a/arch/arm64/mm/mmu.c b/arch/arm64/mm/mmu.c index 18a8b0d3714e..d4de88770ecf 100644 --- a/arch/arm64/mm/mmu.c +++ b/arch/arm64/mm/mmu.c @@ -49,8 +49,6 @@ #define NO_CONT_MAPPINGS BIT(1) #define NO_EXEC_MAPPINGS BIT(2) /* assumes FEAT_HPDS is not used */ -DEFINE_STATIC_KEY_FALSE(arm64_ptdump_lock_key); - u64 kimage_voffset __ro_after_init; EXPORT_SYMBOL(kimage_voffset); @@ -1864,8 +1862,7 @@ int pmd_clear_huge(pmd_t *pmdp) return 1; } -static int __pmd_free_pte_page(pmd_t *pmdp, unsigned long addr, - bool acquire_mmap_lock) +int pmd_free_pte_page(pmd_t *pmdp, unsigned long addr) { pte_t *table; pmd_t pmd; @@ -1877,25 +1874,13 @@ static int __pmd_free_pte_page(pmd_t *pmdp, unsigned long addr, return 1; } - /* See comment in pud_free_pmd_page for static key logic */ table = pte_offset_kernel(pmdp, addr); pmd_clear(pmdp); __flush_tlb_kernel_pgtable(addr); - if (static_branch_unlikely(&arm64_ptdump_lock_key) && acquire_mmap_lock) { - mmap_read_lock(&init_mm); - mmap_read_unlock(&init_mm); - } - pte_free_kernel(NULL, table); return 1; } -int pmd_free_pte_page(pmd_t *pmdp, unsigned long addr) -{ - /* If ptdump is walking the pagetables, acquire init_mm.mmap_lock */ - return __pmd_free_pte_page(pmdp, addr, /* acquire_mmap_lock = */ true); -} - int pud_free_pmd_page(pud_t *pudp, unsigned long addr) { pmd_t *table; @@ -1911,36 +1896,16 @@ int pud_free_pmd_page(pud_t *pudp, unsigned long addr) } table = pmd_offset(pudp, addr); - - /* - * Our objective is to prevent ptdump from reading a PMD table which has - * been freed. In this race, if pud_free_pmd_page observes the key on - * (which got flipped by ptdump) then the mmap lock sequence here will, - * as a result of the mmap write lock/unlock sequence in ptdump, give - * us the correct synchronization. If not, this means that ptdump has - * yet not started walking the pagetables - the sequence of barriers - * issued by __flush_tlb_kernel_pgtable() guarantees that ptdump will - * observe an empty PUD. - */ - pud_clear(pudp); - __flush_tlb_kernel_pgtable(addr); - if (static_branch_unlikely(&arm64_ptdump_lock_key)) { - mmap_read_lock(&init_mm); - mmap_read_unlock(&init_mm); - } - pmdp = table; next = addr; end = addr + PUD_SIZE; do { if (pmd_present(pmdp_get(pmdp))) - /* - * PMD has been isolated, so ptdump won't see it. No - * need to acquire init_mm.mmap_lock. - */ - __pmd_free_pte_page(pmdp, next, /* acquire_mmap_lock = */ false); + pmd_free_pte_page(pmdp, next); } while (pmdp++, next += PMD_SIZE, next != end); + pud_clear(pudp); + __flush_tlb_kernel_pgtable(addr); pmd_free(NULL, table); return 1; } diff --git a/arch/arm64/mm/ptdump.c b/arch/arm64/mm/ptdump.c index 1c20144700d7..5a76c59b5ada 100644 --- a/arch/arm64/mm/ptdump.c +++ b/arch/arm64/mm/ptdump.c @@ -283,13 +283,6 @@ void note_page_flush(struct ptdump_state *pt_st) note_page(pt_st, 0, -1, pte_val(pte_zero)); } -static void arm64_ptdump_walk_pgd(struct ptdump_state *st, struct mm_struct *mm) -{ - static_branch_inc(&arm64_ptdump_lock_key); - ptdump_walk_pgd(st, mm, NULL); - static_branch_dec(&arm64_ptdump_lock_key); -} - void ptdump_walk(struct seq_file *s, struct ptdump_info *info) { unsigned long end = ~0UL; @@ -318,7 +311,7 @@ void ptdump_walk(struct seq_file *s, struct ptdump_info *info) } }; - arm64_ptdump_walk_pgd(&st.ptdump, info->mm); + ptdump_walk_pgd(&st.ptdump, info->mm, NULL); } static void __init ptdump_initialize(void) @@ -360,7 +353,7 @@ bool ptdump_check_wx(void) } }; - arm64_ptdump_walk_pgd(&st.ptdump, &init_mm); + ptdump_walk_pgd(&st.ptdump, &init_mm, NULL); if (st.wx_pages || st.uxn_pages) { pr_warn("Checked W+X mappings: FAILED, %lu W+X pages found, %lu non-UXN pages found\n", diff --git a/arch/microblaze/kernel/head.S b/arch/microblaze/kernel/head.S index 808019c3b7ac..9bd3e513c89b 100644 --- a/arch/microblaze/kernel/head.S +++ b/arch/microblaze/kernel/head.S @@ -39,6 +39,8 @@ #include .section .data +/* The MMU requires a page aligned page directory. */ +.align 12 .global swapper_pg_dir swapper_pg_dir: .space PAGE_SIZE diff --git a/arch/powerpc/platforms/pseries/lparcfg.c b/arch/powerpc/platforms/pseries/lparcfg.c index 54b7ecf375b5..3280b7968cab 100644 --- a/arch/powerpc/platforms/pseries/lparcfg.c +++ b/arch/powerpc/platforms/pseries/lparcfg.c @@ -699,7 +699,7 @@ static ssize_t lparcfg_write(struct file *file, const char __user * buf, if (!firmware_has_feature(FW_FEATURE_SPLPAR)) return -EINVAL; - if (count > sizeof(kbuf)) + if (count == 0 || count > sizeof(kbuf)) return -EINVAL; if (copy_from_user(kbuf, buf, count)) diff --git a/arch/powerpc/platforms/pseries/papr-phy-attest.c b/arch/powerpc/platforms/pseries/papr-phy-attest.c index 20a0e1581302..350ba26e5962 100644 --- a/arch/powerpc/platforms/pseries/papr-phy-attest.c +++ b/arch/powerpc/platforms/pseries/papr-phy-attest.c @@ -230,10 +230,17 @@ static long papr_phy_attest_create_handle(struct papr_phy_attest_io_block __user return -ENOMEM; if (copy_from_user(¶ms->cmd, ulc, - sizeof(struct papr_phy_attest_io_block))) + sizeof(struct papr_phy_attest_io_block))) { + kfree(params); return -EFAULT; + } params->cmd_len = be32_to_cpu(params->cmd.length); + if (params->cmd_len == 0 || params->cmd_len > sizeof(params->cmd)) { + kfree(params); + return -EINVAL; + } + seq = (struct papr_rtas_sequence) { .begin = phy_attest_sequence_begin, .end = phy_attest_sequence_end, @@ -246,6 +253,9 @@ static long papr_phy_attest_create_handle(struct papr_phy_attest_io_block __user &papr_phy_attest_handle_ops, "[papr-physical-attestation]"); + if (fd < 0) + kfree(params); + return fd; } diff --git a/arch/powerpc/platforms/pseries/pci.c b/arch/powerpc/platforms/pseries/pci.c index 84e4ffe957a8..d11a64a086c1 100644 --- a/arch/powerpc/platforms/pseries/pci.c +++ b/arch/powerpc/platforms/pseries/pci.c @@ -132,7 +132,7 @@ static int pseries_pci_sriov_enable(struct pci_dev *pdev, u16 num_vfs) /* First integer stores max config */ max_config_vfs = of_read_number(&max_vfs[0], 1); - if (max_config_vfs < num_vfs && num_vfs > MAX_VFS_FOR_MAP_PE) { + if (max_config_vfs < num_vfs || num_vfs > MAX_VFS_FOR_MAP_PE) { dev_err(&pdev->dev, "Num VFs %x > %x Configurable VFs\n", num_vfs, (num_vfs > MAX_VFS_FOR_MAP_PE) ? diff --git a/arch/s390/include/asm/kvm_host.h b/arch/s390/include/asm/kvm_host.h index c172f9b212d1..b4182ca4435f 100644 --- a/arch/s390/include/asm/kvm_host.h +++ b/arch/s390/include/asm/kvm_host.h @@ -440,6 +440,7 @@ struct kvm_vcpu_arch { bool skey_enabled; /* Indicator if the access registers have been loaded from guest */ bool acrs_loaded; + bool initialized; struct kvm_s390_pv_vcpu pv; union diag318_info diag318_info; struct kvm_s390_mmu_cache *mc; diff --git a/arch/s390/kvm/dat.c b/arch/s390/kvm/dat.c index ed4259d17629..3f2d6e8902d7 100644 --- a/arch/s390/kvm/dat.c +++ b/arch/s390/kvm/dat.c @@ -755,13 +755,15 @@ int dat_cond_set_storage_key(struct kvm_s390_mmu_cache *mmc, union asce asce, gf return rc; } -int dat_reset_reference_bit(union asce asce, gfn_t gfn) +int dat_reset_reference_bit(union asce asce, gfn_t gfn, union skey *skey) { union pgste pgste, old; union crste *crstep; union pte *ptep; int rc; + skey->skey = 0; + rc = dat_entry_walk(NULL, gfn, asce, DAT_WALK_ANY, TABLE_TYPE_PAGE_TABLE, &crstep, &ptep); if (rc) return rc; @@ -771,21 +773,23 @@ int dat_reset_reference_bit(union asce asce, gfn_t gfn) if (!crste.h.fc || !crste.s.fc1.pr) return 0; - return page_reset_referenced(large_crste_to_phys(*crstep, gfn)); + skey->skey = page_reset_referenced(large_crste_to_phys(*crstep, gfn)) << 1; + return 0; } old = pgste_get_lock(ptep); pgste = old; if (!ptep->h.i) { - rc = page_reset_referenced(pte_origin(*ptep)); - pgste.hr = rc >> 1; + skey->skey = page_reset_referenced(pte_origin(*ptep)) << 1; + pgste.hr = skey->r; } - rc |= (pgste.gr << 1) | pgste.gc; + skey->r |= pgste.gr; + skey->c |= pgste.gc; pgste.gr = 0; dat_update_ptep_sd(old, pgste, ptep); pgste_set_unlock(ptep, pgste); - return rc; + return 0; } static long dat_reset_skeys_pte(union pte *ptep, gfn_t gfn, gfn_t next, struct dat_walk *walk) @@ -846,6 +850,7 @@ static long _dat_slot_pte(union pte *ptep, gfn_t gfn, gfn_t next, struct dat_wal struct slot_priv *p = walk->priv; union crste dummy = { .val = p->token }; union pte new_pte, pte = READ_ONCE(*ptep); + union pgste pgste; new_pte = _PTE_TOK(dummy.tok.type, dummy.tok.par); @@ -853,7 +858,11 @@ static long _dat_slot_pte(union pte *ptep, gfn_t gfn, gfn_t next, struct dat_wal if (pte.val == new_pte.val) return 0; - dat_ptep_xchg(ptep, new_pte, gfn, walk->asce, false); + pgste = pgste_get_lock(ptep); + pgste = __dat_ptep_xchg(ptep, pgste, new_pte, gfn, walk->asce, false); + pgste.cmma_d = 0; + pgste_set_unlock(ptep, pgste); + return 0; } diff --git a/arch/s390/kvm/dat.h b/arch/s390/kvm/dat.h index fad605305e05..141ee7b9f019 100644 --- a/arch/s390/kvm/dat.h +++ b/arch/s390/kvm/dat.h @@ -537,7 +537,7 @@ int dat_set_storage_key(struct kvm_s390_mmu_cache *mc, union asce asce, gfn_t gf union skey skey, bool nq); int dat_cond_set_storage_key(struct kvm_s390_mmu_cache *mmc, union asce asce, gfn_t gfn, union skey skey, union skey *oldkey, bool nq, bool mr, bool mc); -int dat_reset_reference_bit(union asce asce, gfn_t gfn); +int dat_reset_reference_bit(union asce asce, gfn_t gfn, union skey *skey); long dat_reset_skeys(union asce asce, gfn_t start); unsigned long dat_get_ptval(struct page_table *table, struct ptval_param param); diff --git a/arch/s390/kvm/faultin.c b/arch/s390/kvm/faultin.c index fee80047bd94..3cc45f7f5b2d 100644 --- a/arch/s390/kvm/faultin.c +++ b/arch/s390/kvm/faultin.c @@ -91,9 +91,9 @@ int kvm_s390_faultin_gfn(struct kvm_vcpu *vcpu, struct kvm *kvm, struct guest_fa /* Access outside memory, addressing exception. */ if (is_noslot_pfn(f->pfn)) return PGM_ADDRESSING; - /* Signal pending: try again. */ - if (f->pfn == KVM_PFN_ERR_SIGPENDING) - return -EAGAIN; + /* Fatal signal pending: bail out. */ + if (is_sigpending_pfn(f->pfn)) + return -EINTR; /* Check if it's read-only memory; don't try to actually handle that case. */ if (f->pfn == KVM_PFN_ERR_RO_FAULT) return -EOPNOTSUPP; diff --git a/arch/s390/kvm/interrupt.c b/arch/s390/kvm/interrupt.c index 7514d9e2403c..8f24bcd1a6d3 100644 --- a/arch/s390/kvm/interrupt.c +++ b/arch/s390/kvm/interrupt.c @@ -45,13 +45,16 @@ static struct kvm_s390_gib *gib; static int sca_ext_call_pending(struct kvm_vcpu *vcpu, int *src_id) { struct esca_block *sca = vcpu->kvm->arch.sca; - union esca_sigp_ctrl sigp_ctrl = sca->cpu[vcpu->vcpu_id].sigp_ctrl; + union esca_sigp_ctrl sigp_ctrl; if (!kvm_s390_test_cpuflags(vcpu, CPUSTAT_ECALL_PEND)) return 0; + if (kvm_is_ucontrol(vcpu->kvm)) + return 0; BUG_ON(!kvm_s390_use_sca_entries()); + sigp_ctrl = sca->cpu[vcpu->vcpu_id].sigp_ctrl; if (src_id) *src_id = sigp_ctrl.scn; @@ -60,13 +63,16 @@ static int sca_ext_call_pending(struct kvm_vcpu *vcpu, int *src_id) static int sca_inject_ext_call(struct kvm_vcpu *vcpu, int src_id) { - struct esca_block *sca = vcpu->kvm->arch.sca; - union esca_sigp_ctrl *sigp_ctrl = &sca->cpu[vcpu->vcpu_id].sigp_ctrl; union esca_sigp_ctrl old_val, new_val = {.scn = src_id, .c = 1}; + struct esca_block *sca = vcpu->kvm->arch.sca; + union esca_sigp_ctrl *sigp_ctrl; int expect, rc; BUG_ON(!kvm_s390_use_sca_entries()); + if (kvm_is_ucontrol(vcpu->kvm)) + return -EINVAL; + sigp_ctrl = &sca->cpu[vcpu->vcpu_id].sigp_ctrl; old_val = READ_ONCE(*sigp_ctrl); old_val.c = 0; @@ -84,10 +90,13 @@ static int sca_inject_ext_call(struct kvm_vcpu *vcpu, int src_id) static void sca_clear_ext_call(struct kvm_vcpu *vcpu) { struct esca_block *sca = vcpu->kvm->arch.sca; - union esca_sigp_ctrl *sigp_ctrl = &sca->cpu[vcpu->vcpu_id].sigp_ctrl; + union esca_sigp_ctrl *sigp_ctrl; - if (!kvm_s390_use_sca_entries()) + if (!kvm_s390_use_sca_entries() || !vcpu->arch.initialized || kvm_is_ucontrol(vcpu->kvm)) return; + + /* Initialize after the above check, to prevent going out of bounds */ + sigp_ctrl = &sca->cpu[vcpu->vcpu_id].sigp_ctrl; kvm_s390_clear_cpuflags(vcpu, CPUSTAT_ECALL_PEND); WRITE_ONCE(sigp_ctrl->value, 0); diff --git a/arch/s390/kvm/kvm-s390.c b/arch/s390/kvm/kvm-s390.c index 150b5dd2170e..518a69c55e85 100644 --- a/arch/s390/kvm/kvm-s390.c +++ b/arch/s390/kvm/kvm-s390.c @@ -571,7 +571,7 @@ static int kvm_s390_keyop(struct kvm_s390_mmu_cache *mc, struct kvm *kvm, int op switch (op) { case KVM_S390_KEYOP_SSKE: r = dat_cond_set_storage_key(mc, asce, gfn, skey, &skey, 0, 0, 0); - if (r >= 0) + if (r == 0 || r == 1) return skey.skey; break; case KVM_S390_KEYOP_ISKE: @@ -580,14 +580,14 @@ static int kvm_s390_keyop(struct kvm_s390_mmu_cache *mc, struct kvm *kvm, int op return skey.skey; break; case KVM_S390_KEYOP_RRBE: - r = dat_reset_reference_bit(asce, gfn); - if (r > 0) - return r << 1; + r = dat_reset_reference_bit(asce, gfn, &skey); + if (!r) + return skey.skey; break; default: return -EINVAL; } - return r; + return r > 0 ? -EFAULT : r; } /* Section: device related */ @@ -1219,8 +1219,8 @@ static void kvm_s390_sync_request_broadcast(struct kvm *kvm, int req) /* * Must be called with kvm->srcu held to avoid races on memslots, and with - * kvm->slots_lock to avoid races with ourselves, kvm_s390_vm_stop_migration(), - * and kvm_s390_get_cmma_bits(). + * kvm->slots_arch_lock to avoid races with ourselves, + * kvm_s390_vm_stop_migration(), and kvm_s390_get_cmma_bits(). */ static int kvm_s390_vm_start_migration(struct kvm *kvm) { @@ -1265,7 +1265,7 @@ static int kvm_s390_vm_start_migration(struct kvm *kvm) } /* - * Must be called with kvm->slots_lock to avoid races with ourselves, + * Must be called with kvm->slots_arch_lock to avoid races with ourselves, * kvm_s390_vm_start_migration() and kvm_s390_get_cmma_bits(). */ static int kvm_s390_vm_stop_migration(struct kvm *kvm) @@ -1300,7 +1300,9 @@ static int kvm_s390_vm_set_migration(struct kvm *kvm, { int res = -ENXIO; - mutex_lock(&kvm->slots_lock); + guard(srcu)(&kvm->srcu); + guard(mutex)(&kvm->slots_arch_lock); + switch (attr->attr) { case KVM_S390_VM_MIGRATION_START: res = kvm_s390_vm_start_migration(kvm); @@ -1311,7 +1313,6 @@ static int kvm_s390_vm_set_migration(struct kvm *kvm, default: break; } - mutex_unlock(&kvm->slots_lock); return res; } @@ -2214,7 +2215,7 @@ static int kvm_s390_get_skeys(struct kvm *kvm, struct kvm_s390_skeys *args) } kvfree(keys); - return r; + return r <= 0 ? r : -EFAULT; } static int kvm_s390_set_skeys(struct kvm *kvm, struct kvm_s390_skeys *args) @@ -2276,7 +2277,7 @@ static int kvm_s390_set_skeys(struct kvm *kvm, struct kvm_s390_skeys *args) kvm_s390_free_mmu_cache(mc); out: kvfree(keys); - return r; + return r <= 0 ? r : -EFAULT; } /* @@ -2386,7 +2387,7 @@ static int kvm_s390_set_cmma_bits(struct kvm *kvm, set_bit(GMAP_FLAG_USES_CMM, &kvm->arch.gmap->flags); - return r; + return r <= 0 ? r : -EFAULT; } /** @@ -2934,6 +2935,9 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) case KVM_S390_INTERRUPT: { struct kvm_s390_interrupt s390int; + r = -EINVAL; + if (kvm_is_ucontrol(kvm)) + break; r = -EFAULT; if (copy_from_user(&s390int, argp, sizeof(s390int))) break; @@ -2998,9 +3002,8 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) r = -EFAULT; if (copy_from_user(&args, argp, sizeof(args))) break; - mutex_lock(&kvm->slots_lock); - r = kvm_s390_get_cmma_bits(kvm, &args); - mutex_unlock(&kvm->slots_lock); + scoped_guard(mutex, &kvm->slots_arch_lock) + r = kvm_s390_get_cmma_bits(kvm, &args); if (!r) { r = copy_to_user(argp, &args, sizeof(args)); if (r) @@ -3014,9 +3017,9 @@ int kvm_arch_vm_ioctl(struct file *filp, unsigned int ioctl, unsigned long arg) r = -EFAULT; if (copy_from_user(&args, argp, sizeof(args))) break; - mutex_lock(&kvm->slots_lock); + mutex_lock(&kvm->slots_arch_lock); r = kvm_s390_set_cmma_bits(kvm, &args); - mutex_unlock(&kvm->slots_lock); + mutex_unlock(&kvm->slots_arch_lock); break; } case KVM_S390_PV_COMMAND: { @@ -3247,7 +3250,8 @@ static void kvm_s390_crypto_init(struct kvm *kvm) static void sca_dispose(struct kvm *kvm) { - free_pages_exact(kvm->arch.sca, sizeof(*kvm->arch.sca)); + if (kvm->arch.sca) + free_pages_exact(kvm->arch.sca, sizeof(*kvm->arch.sca)); kvm->arch.sca = NULL; } @@ -3461,7 +3465,7 @@ static void sca_del_vcpu(struct kvm_vcpu *vcpu) { struct esca_block *sca = vcpu->kvm->arch.sca; - if (!kvm_s390_use_sca_entries()) + if (!kvm_s390_use_sca_entries() || !vcpu->arch.initialized) return; clear_bit_inv(vcpu->vcpu_id, (unsigned long *)sca->mcn); @@ -3481,8 +3485,8 @@ static void sca_add_vcpu(struct kvm_vcpu *vcpu) if (!kvm_s390_use_sca_entries()) return; + WRITE_ONCE(sca->cpu[vcpu->vcpu_id].sda, virt_to_phys(vcpu->arch.sie_block)); set_bit_inv(vcpu->vcpu_id, (unsigned long *)sca->mcn); - sca->cpu[vcpu->vcpu_id].sda = virt_to_phys(vcpu->arch.sie_block); } static int sca_can_add_vcpu(struct kvm *kvm, unsigned int id) @@ -3613,6 +3617,9 @@ void kvm_arch_vcpu_postcreate(struct kvm_vcpu *vcpu) if (test_kvm_facility(vcpu->kvm, 74) || vcpu->kvm->arch.user_instr0 || vcpu->kvm->arch.user_operexec) vcpu->arch.sie_block->ictl |= ICTL_OPEREXC; + + /* Pairs with smp_load_acquire() in kvm_arch_vcpu_ioctl_run() and kvm_arch_vcpu_ioctl() */ + smp_store_release(&vcpu->arch.initialized, true); } static bool kvm_has_pckmo_subfunc(struct kvm *kvm, unsigned long nr) @@ -3674,7 +3681,8 @@ static void kvm_s390_vcpu_crypto_setup(struct kvm_vcpu *vcpu) void kvm_s390_vcpu_unsetup_cmma(struct kvm_vcpu *vcpu) { - free_page((unsigned long)phys_to_virt(vcpu->arch.sie_block->cbrlo)); + if (vcpu->arch.sie_block->cbrlo) + free_page((unsigned long)phys_to_virt(vcpu->arch.sie_block->cbrlo)); vcpu->arch.sie_block->cbrlo = 0; } @@ -3792,21 +3800,21 @@ int kvm_arch_vcpu_precreate(struct kvm *kvm, unsigned int id) return 0; } +DEFINE_FREE(sie_page, struct sie_page *, if (_T) free_page((unsigned long)(_T))) + int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) { - struct sie_page *sie_page; + struct kvm_s390_mmu_cache *mc __free(kvm_s390_mmu_cache) = NULL; + struct sie_page *sie_page __free(sie_page) = NULL; int rc; BUILD_BUG_ON(sizeof(struct sie_page) != 4096); - vcpu->arch.mc = kvm_s390_new_mmu_cache(); - if (!vcpu->arch.mc) + mc = kvm_s390_new_mmu_cache(); + if (!mc) return -ENOMEM; sie_page = (struct sie_page *) get_zeroed_page(GFP_KERNEL_ACCOUNT); - if (!sie_page) { - kvm_s390_free_mmu_cache(vcpu->arch.mc); - vcpu->arch.mc = NULL; + if (!sie_page) return -ENOMEM; - } vcpu->arch.sie_block = &sie_page->sie_block; vcpu->arch.sie_block->itdba = virt_to_phys(&sie_page->itdb); @@ -3848,10 +3856,9 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) vcpu->run->kvm_valid_regs |= KVM_SYNC_FPRS; if (kvm_is_ucontrol(vcpu->kvm)) { - rc = -ENOMEM; vcpu->arch.gmap = gmap_new_child(vcpu->kvm->arch.gmap, -1UL); if (!vcpu->arch.gmap) - goto out_free_sie_block; + return -ENOMEM; } VM_EVENT(vcpu->kvm, 3, "create cpu %d at 0x%p, sie block at 0x%p", @@ -3859,20 +3866,19 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu) trace_kvm_s390_create_vcpu(vcpu->vcpu_id, vcpu, vcpu->arch.sie_block); rc = kvm_s390_vcpu_setup(vcpu); - if (rc) - goto out_ucontrol_uninit; + if (rc) { + if (kvm_is_ucontrol(vcpu->kvm)) { + scoped_guard(spinlock, &vcpu->kvm->arch.gmap->children_lock) + gmap_remove_child(vcpu->arch.gmap); + vcpu->arch.gmap = gmap_put(vcpu->arch.gmap); + } + return rc; + } + vcpu->arch.mc = no_free_ptr(mc); + sie_page = NULL; kvm_s390_update_topology_change_report(vcpu->kvm, 1); return 0; - -out_ucontrol_uninit: - if (kvm_is_ucontrol(vcpu->kvm)) { - gmap_remove_child(vcpu->arch.gmap); - vcpu->arch.gmap = gmap_put(vcpu->arch.gmap); - } -out_free_sie_block: - free_page((unsigned long)(vcpu->arch.sie_block)); - return rc; } int kvm_arch_vcpu_runnable(struct kvm_vcpu *vcpu) @@ -5039,6 +5045,10 @@ int kvm_arch_vcpu_ioctl_run(struct kvm_vcpu *vcpu) kvm_run->kvm_dirty_regs & ~KVM_SYNC_S390_VALID_FIELDS) return -EINVAL; + /* Pairs with smp_store_release() in kvm_arch_vcpu_postcreate() */ + if (!smp_load_acquire(&vcpu->arch.initialized)) + return -EINVAL; + vcpu_load(vcpu); if (guestdbg_exit_pending(vcpu)) { @@ -5447,6 +5457,8 @@ long kvm_arch_vcpu_unlocked_ioctl(struct file *filp, unsigned int ioctl, struct kvm_s390_interrupt s390int; struct kvm_s390_irq s390irq = {}; + if (kvm_is_ucontrol(vcpu->kvm)) + return -EINVAL; if (copy_from_user(&s390int, argp, sizeof(s390int))) return -EFAULT; if (s390int_to_s390irq(&s390int, &s390irq)) @@ -5523,6 +5535,10 @@ long kvm_arch_vcpu_ioctl(struct file *filp, long r; u16 rc, rrc; + /* Pairs with smp_store_release() in kvm_arch_vcpu_postcreate() */ + if (!smp_load_acquire(&vcpu->arch.initialized)) + return -EINVAL; + vcpu_load(vcpu); switch (ioctl) { @@ -5794,14 +5810,30 @@ int kvm_arch_prepare_memory_region(struct kvm *kvm, return 0; } +static long cmma_d_count_pte(union pte *ptep, gfn_t gfn, gfn_t next, struct dat_walk *walk) +{ + union pgste pgste; + + pgste = pgste_get_lock(ptep); + if (pgste.cmma_d) { + pgste.cmma_d = 0; + atomic64_dec(walk->priv); + } + pgste_set_unlock(ptep, pgste); + return 0; +} + void kvm_arch_commit_memory_region(struct kvm *kvm, struct kvm_memory_slot *old, const struct kvm_memory_slot *new, enum kvm_mr_change change) { - struct kvm_s390_mmu_cache *mc = NULL; + const struct dat_walk_ops ops = { .pte_entry = cmma_d_count_pte, }; + struct kvm_s390_mmu_cache *mc __free(kvm_s390_mmu_cache) = NULL; int rc = 0; + guard(mutex)(&kvm->slots_arch_lock); + if (change == KVM_MR_FLAGS_ONLY) return; @@ -5812,6 +5844,12 @@ void kvm_arch_commit_memory_region(struct kvm *kvm, } scoped_guard(write_lock, &kvm->mmu_lock) { + if (kvm->arch.migration_mode && kvm->arch.use_cmma && old) { + _dat_walk_gfn_range(old->base_gfn, old->base_gfn + old->npages, + kvm->arch.gmap->asce, &ops, DAT_WALK_IGN_HOLES, + &kvm->arch.cmma_dirty_pages); + } + switch (change) { case KVM_MR_DELETE: rc = dat_delete_slot(mc, kvm->arch.gmap->asce, old->base_gfn, old->npages); @@ -5833,7 +5871,6 @@ void kvm_arch_commit_memory_region(struct kvm *kvm, out: if (rc) pr_warn("failed to commit memory region\n"); - kvm_s390_free_mmu_cache(mc); return; } diff --git a/arch/s390/kvm/priv.c b/arch/s390/kvm/priv.c index ad0ddc433a73..b1ba24c346ef 100644 --- a/arch/s390/kvm/priv.c +++ b/arch/s390/kvm/priv.c @@ -289,6 +289,7 @@ static int handle_iske(struct kvm_vcpu *vcpu) static int handle_rrbe(struct kvm_vcpu *vcpu) { unsigned long gaddr; + union skey skey; int reg1, reg2; int rc; @@ -307,12 +308,12 @@ static int handle_rrbe(struct kvm_vcpu *vcpu) gaddr = kvm_s390_logical_to_effective(vcpu, gaddr); gaddr = kvm_s390_real_to_abs(vcpu, gaddr); scoped_guard(read_lock, &vcpu->kvm->mmu_lock) - rc = dat_reset_reference_bit(vcpu->arch.gmap->asce, gpa_to_gfn(gaddr)); + rc = dat_reset_reference_bit(vcpu->arch.gmap->asce, gpa_to_gfn(gaddr), &skey); if (rc > 0) return kvm_s390_inject_program_int(vcpu, rc); if (rc < 0) return rc; - kvm_s390_set_psw_cc(vcpu, rc); + kvm_s390_set_psw_cc(vcpu, (skey.skey >> 1) & 3); return 0; } @@ -1260,8 +1261,9 @@ static int handle_essa(struct kvm_vcpu *vcpu) /* Retry the ESSA instruction */ kvm_s390_retry_instr(vcpu); } else { - scoped_guard(read_lock, &vcpu->kvm->mmu_lock) - i = __do_essa(vcpu, orc); + scoped_guard(mutex, &vcpu->kvm->slots_arch_lock) + scoped_guard(read_lock, &vcpu->kvm->mmu_lock) + i = __do_essa(vcpu, orc); if (i < 0) return i; /* Account for the possible extra cbrl entry */ diff --git a/arch/s390/kvm/pv.c b/arch/s390/kvm/pv.c index 1beacc841ca8..b02e0159d3cd 100644 --- a/arch/s390/kvm/pv.c +++ b/arch/s390/kvm/pv.c @@ -244,6 +244,24 @@ static void kvm_s390_clear_pv_state(struct kvm *kvm) kvm->arch.pv.stor_var = NULL; } +static void kvm_s390_pv_dispose_cpu(struct kvm_vcpu *vcpu, bool free_stor_base) +{ + if (free_stor_base) + free_pages(vcpu->arch.pv.stor_base, get_order(uv_info.guest_cpu_stor_len)); + free_page((unsigned long)sida_addr(vcpu->arch.sie_block)); + vcpu->arch.sie_block->pv_handle_cpu = 0; + vcpu->arch.sie_block->pv_handle_config = 0; + memset(&vcpu->arch.pv, 0, sizeof(vcpu->arch.pv)); + vcpu->arch.sie_block->sdf = 0; + /* + * The sidad field (for sdf == 2) is now the gbea field (for sdf == 0). + * Use the reset value of gbea to avoid leaking the kernel pointer of + * the just freed sida. + */ + vcpu->arch.sie_block->gbea = 1; + kvm_make_request(KVM_REQ_TLB_FLUSH, vcpu); +} + int kvm_s390_pv_destroy_cpu(struct kvm_vcpu *vcpu, u16 *rc, u16 *rrc) { int cc; @@ -258,24 +276,9 @@ int kvm_s390_pv_destroy_cpu(struct kvm_vcpu *vcpu, u16 *rc, u16 *rrc) WARN_ONCE(cc, "protvirt destroy cpu failed rc %x rrc %x", *rc, *rrc); /* Intended memory leak for something that should never happen. */ - if (!cc) - free_pages(vcpu->arch.pv.stor_base, - get_order(uv_info.guest_cpu_stor_len)); + kvm_s390_pv_dispose_cpu(vcpu, !cc); - free_page((unsigned long)sida_addr(vcpu->arch.sie_block)); - vcpu->arch.sie_block->pv_handle_cpu = 0; - vcpu->arch.sie_block->pv_handle_config = 0; - memset(&vcpu->arch.pv, 0, sizeof(vcpu->arch.pv)); - vcpu->arch.sie_block->sdf = 0; - /* - * The sidad field (for sdf == 2) is now the gbea field (for sdf == 0). - * Use the reset value of gbea to avoid leaking the kernel pointer of - * the just freed sida. - */ - vcpu->arch.sie_block->gbea = 1; - kvm_make_request(KVM_REQ_TLB_FLUSH, vcpu); - - return cc ? EIO : 0; + return cc ? -EIO : 0; } int kvm_s390_pv_create_cpu(struct kvm_vcpu *vcpu, u16 *rc, u16 *rrc) @@ -319,9 +322,7 @@ int kvm_s390_pv_create_cpu(struct kvm_vcpu *vcpu, u16 *rc, u16 *rrc) uvcb.header.rrc); if (cc) { - u16 dummy; - - kvm_s390_pv_destroy_cpu(vcpu, &dummy, &dummy); + kvm_s390_pv_dispose_cpu(vcpu, true); return -EIO; } @@ -809,7 +810,7 @@ static int unpack_one(struct kvm *kvm, unsigned long addr, u64 tweak, return -EAGAIN; } - if (ret && ret != -EAGAIN) + if (ret && ret != -EAGAIN && ret != -EINTR) KVM_UV_EVENT(kvm, 3, "PROTVIRT VM UNPACK: failed addr %llx with rc %x rrc %x", uvcb.gaddr, *rc, *rrc); return ret; diff --git a/arch/x86/entry/entry_64.S b/arch/x86/entry/entry_64.S index c6d996593f32..253f0a585927 100644 --- a/arch/x86/entry/entry_64.S +++ b/arch/x86/entry/entry_64.S @@ -937,6 +937,8 @@ SYM_CODE_START(paranoid_entry) IBRS_ENTER save_reg=%r15 UNTRAIN_RET_FROM_CALL + HANDLE_INTR_SAFERET 8(%rsp) + RET SYM_CODE_END(paranoid_entry) @@ -1039,6 +1041,11 @@ SYM_CODE_START(error_entry) movl %ecx, %eax /* zero extend */ cmpq %rax, RIP+8(%rsp) je .Lbstep_iret + + VALIDATE_UNRET_END + + HANDLE_INTR_SAFERET 8(%rsp) + cmpq $.Lgs_change, RIP+8(%rsp) jne .Lerror_entry_done_lfence @@ -1057,7 +1064,6 @@ SYM_CODE_START(error_entry) FENCE_SWAPGS_KERNEL_ENTRY CALL_DEPTH_ACCOUNT leaq 8(%rsp), %rax /* return pt_regs pointer */ - VALIDATE_UNRET_END RET .Lbstep_iret: diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h index 5f6c1ce9673b..6db5b5f79df9 100644 --- a/arch/x86/include/asm/kvm_host.h +++ b/arch/x86/include/asm/kvm_host.h @@ -1858,7 +1858,7 @@ struct kvm_x86_ops { * Can potentially get non-canonical addresses through INVLPGs, which * the implementation may choose to ignore if appropriate. */ - void (*flush_tlb_gva)(struct kvm_vcpu *vcpu, gva_t addr); + void (*flush_tlb_gva)(struct kvm_vcpu *vcpu, gva_t addr, bool *full); /* * Flush any TLB entries created by the guest. Like tlb_flush_gva(), diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h index b68892e6d58c..2ea6591bc7b9 100644 --- a/arch/x86/include/asm/nospec-branch.h +++ b/arch/x86/include/asm/nospec-branch.h @@ -12,6 +12,7 @@ #include #include #include +#include /* * Call depth tracking for Intel SKL CPUs to address the RSB underflow @@ -176,6 +177,50 @@ add $(BITS_PER_LONG/8), %_ASM_SP; \ lfence; +/* + * Helper for detecting if an interrupt occurred at an unsafe location within + * Safe-RET. If Safe-RET is interrupted after the CALL or LEA the RSB may get + * poisoned by the interrupt handler. + * + * The Safe-RET sequence is: + * + * CALL + * LEA 8(%RSP), %RSP + * RET + * + * The two CMPs below check whether RIP points to after the CALL or after the + * LEA. + * + * The LFENCE below is to address this particular speculation case: + * + * 1. Userspace runs and poisons the BTB around the safe-RET routine + * + * 2. Userspace triggers some kind of exception + * + * 3. Kernel executes error_entry() and mis-speculates the branch into thinking + * it actually came from kernel space + * + * 4. The kernel then further mis-speculates that the exception occurred due + * to an interrupted safe-RET + * + * 5. The handle_interrupted_saferet() routine speculatively executes and + * speculatively does a safe-RET. But this is unsafe since it was never + * untrained. + * + * The LFENCE fixes this by ensuring step 5 is never reached speculatively. + * Note that this LFENCE only occurs if safe-RET was actually interrupted (so + * it's outside of the normal path). + */ +#define __HANDLE_INTR_SAFERET(name, pt_regs) \ + cmpq $(name), RIP+pt_regs; \ + jb 1f; \ + cmpq $(name)+5, RIP+pt_regs; \ + ja 1f; \ + lfence; \ + leaq pt_regs, %rdi; \ + call handle_interrupted_saferet; \ + 1: + #ifdef __ASSEMBLER__ /* @@ -293,6 +338,14 @@ #define UNTRAIN_RET_FROM_CALL \ __UNTRAIN_RET X86_FEATURE_ENTRY_IBPB, __stringify(RESET_CALL_DEPTH_FROM_CALL) +.macro HANDLE_INTR_SAFERET pt_regs +#ifdef CONFIG_MITIGATION_SRSO + ALTERNATIVE_2 "", \ + __stringify(__HANDLE_INTR_SAFERET(srso_safe_ret, \pt_regs)), X86_FEATURE_SRSO, \ + __stringify(__HANDLE_INTR_SAFERET(srso_alias_safe_ret, \pt_regs)), X86_FEATURE_SRSO_ALIAS + +#endif +.endm .macro CALL_DEPTH_ACCOUNT #ifdef CONFIG_MITIGATION_CALL_DEPTH_TRACKING @@ -625,6 +678,10 @@ static __always_inline void x86_idle_clear_cpu_buffers(void) x86_clear_cpu_buffers(); } +void srso_safe_ret(void); +void srso_alias_safe_ret(void); +void handle_interrupted_saferet(struct pt_regs *regs); + #endif /* __ASSEMBLER__ */ #endif /* _ASM_X86_NOSPEC_BRANCH_H_ */ diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c index d9af230c0512..82436b3534fa 100644 --- a/arch/x86/kernel/cpu/bugs.c +++ b/arch/x86/kernel/cpu/bugs.c @@ -3775,3 +3775,42 @@ void __warn_thunk(void) { WARN_ONCE(1, "Unpatched return thunk in use. This should not happen!\n"); } + +#ifdef CONFIG_MITIGATION_SRSO +/* + * Called during exception/interrupt entry if interrupted during the + * safe-RET sequence. The safe-RET sequence consists of 3 instructions: + * + * CALL + * LEA 8(%RSP), %RSP + * RET + * + * An interrupt after the CALL or after the LEA could potentially lead + * to branch predictor poisoning and results in the sequence not being + * able to be safely resumed. + * + * Therefore, modify the regs state as if the remaining part of the + * safe-RET sequence executed so the interrupt returns back to the + * desired return target, instead of the to the safe-RET sequence. + */ +void noinstr handle_interrupted_saferet(struct pt_regs *regs) +{ + unsigned long rip = regs->ip; + + if (rip == (unsigned long) srso_safe_ret || + rip == (unsigned long) srso_alias_safe_ret) { + /* Modify stack pointer as if LEA executed: */ + regs->sp += 8; + } + + /* + * Adjust registers as if RET executed: + * + * 1. Read the return address off the stack and into rIP: + */ + regs->ip = *(unsigned long *)(regs->sp); + + /* 2. Pop rIP off the stack: */ + regs->sp += 8; +} +#endif /* CONFIG_MITIGATION_SRSO */ diff --git a/arch/x86/kernel/cpu/mce/core.c b/arch/x86/kernel/cpu/mce/core.c index 9bba1e2f03af..cfb74be19994 100644 --- a/arch/x86/kernel/cpu/mce/core.c +++ b/arch/x86/kernel/cpu/mce/core.c @@ -2266,10 +2266,10 @@ void mcheck_cpu_init(struct cpuinfo_x86 *c) mca_cfg.initialized = 1; + __mcheck_cpu_setup_timer(); __mcheck_cpu_init_generic(); __mcheck_cpu_init_vendor(c); __mcheck_cpu_init_prepare_banks(); - __mcheck_cpu_setup_timer(); cr4_set_bits(X86_CR4_MCE); } diff --git a/arch/x86/kvm/hyperv.c b/arch/x86/kvm/hyperv.c index fd4eb1e561f7..d8736b5dfcd3 100644 --- a/arch/x86/kvm/hyperv.c +++ b/arch/x86/kvm/hyperv.c @@ -1974,6 +1974,7 @@ int kvm_hv_vcpu_flush_tlb(struct kvm_vcpu *vcpu) u64 entries[KVM_HV_TLB_FLUSH_FIFO_SIZE]; int i, j, count; gva_t gva; + bool full = false; if (!tdp_enabled || !hv_vcpu) return -EINVAL; @@ -1982,7 +1983,7 @@ int kvm_hv_vcpu_flush_tlb(struct kvm_vcpu *vcpu) count = kfifo_out(&tlb_flush_fifo->entries, entries, KVM_HV_TLB_FLUSH_FIFO_SIZE); - for (i = 0; i < count; i++) { + for (i = 0; i < count && !full; i++) { if (entries[i] == KVM_HV_TLB_FLUSHALL_ENTRY) goto out_flush_all; @@ -1991,11 +1992,11 @@ int kvm_hv_vcpu_flush_tlb(struct kvm_vcpu *vcpu) * pages to flush. */ gva = entries[i] & PAGE_MASK; - for (j = 0; j < (entries[i] & ~PAGE_MASK) + 1; j++) { + for (j = 0; j < (entries[i] & ~PAGE_MASK) + 1 && !full; j++) { if (is_noncanonical_invlpg_address(gva + j * PAGE_SIZE, vcpu)) continue; - kvm_x86_call(flush_tlb_gva)(vcpu, gva + j * PAGE_SIZE); + kvm_x86_call(flush_tlb_gva)(vcpu, gva + j * PAGE_SIZE, &full); } ++vcpu->stat.tlb_flush; diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c index 66e69d2a41b3..a61750f8e1e3 100644 --- a/arch/x86/kvm/mmu/mmu.c +++ b/arch/x86/kvm/mmu/mmu.c @@ -2442,6 +2442,9 @@ static union kvm_mmu_page_role kvm_mmu_child_role(u64 *sptep, bool direct, role.direct = direct; role.passthrough = 0; + WARN_ON_ONCE(role.invalid); + role.invalid = 0; + /* * If the guest has 4-byte PTEs then that means it's using 32-bit, * 2-level, non-PAE paging. KVM shadows such guests with PAE paging @@ -6652,7 +6655,7 @@ void kvm_mmu_invalidate_addr(struct kvm_vcpu *vcpu, struct kvm_mmu *mmu, if (is_noncanonical_invlpg_address(addr, vcpu)) return; - kvm_x86_call(flush_tlb_gva)(vcpu, addr); + kvm_x86_call(flush_tlb_gva)(vcpu, addr, NULL); } if (!mmu->sync_spte) diff --git a/arch/x86/kvm/svm/sev.c b/arch/x86/kvm/svm/sev.c index 944aaea6501f..0f0ea7896af5 100644 --- a/arch/x86/kvm/svm/sev.c +++ b/arch/x86/kvm/svm/sev.c @@ -97,6 +97,8 @@ static u64 sev_supported_vmsa_features __ro_after_init; static u8 sev_enc_bit; static DECLARE_RWSEM(sev_deactivate_lock); static DEFINE_MUTEX(sev_bitmap_lock); +/* Protects kvm_sev_info's enc_context_owner, mirror_vms and mirror_entry. */ +static DEFINE_MUTEX(sev_mirror_lock); unsigned int max_sev_asid; static unsigned int min_sev_asid; static unsigned int max_sev_es_asid; @@ -2018,7 +2020,6 @@ static void sev_migrate_from(struct kvm *dst_kvm, struct kvm *src_kvm) dst->asid = src->asid; dst->handle = src->handle; dst->pages_locked = src->pages_locked; - dst->enc_context_owner = src->enc_context_owner; dst->es_active = src->es_active; dst->vmsa_features = src->vmsa_features; @@ -2026,11 +2027,12 @@ static void sev_migrate_from(struct kvm *dst_kvm, struct kvm *src_kvm) src->active = false; src->handle = 0; src->pages_locked = 0; - src->enc_context_owner = NULL; src->es_active = false; list_cut_before(&dst->regions_list, &src->regions_list, &src->regions_list); + mutex_lock(&sev_mirror_lock); + /* * If this VM has mirrors, "transfer" each mirror's refcount of the * source to the destination (this KVM). The caller holds a reference @@ -2047,12 +2049,15 @@ static void sev_migrate_from(struct kvm *dst_kvm, struct kvm *src_kvm) * If this VM is a mirror, remove the old mirror from the owners list * and add the new mirror to the list. */ - if (is_mirroring_enc_context(dst_kvm)) { - struct kvm_sev_info *owner_sev_info = to_kvm_sev_info(dst->enc_context_owner); + if (is_mirroring_enc_context(src_kvm)) { + struct kvm_sev_info *owner_sev_info = to_kvm_sev_info(src->enc_context_owner); + dst->enc_context_owner = src->enc_context_owner; + src->enc_context_owner = NULL; list_del(&src->mirror_entry); list_add_tail(&dst->mirror_entry, &owner_sev_info->mirror_vms); } + mutex_unlock(&sev_mirror_lock); kvm_for_each_vcpu(i, dst_vcpu, dst_kvm) { dst_svm = to_svm(dst_vcpu); @@ -2871,11 +2876,14 @@ int sev_vm_copy_enc_context_from(struct kvm *kvm, unsigned int source_fd) * disappear until we're done with it */ source_sev = to_kvm_sev_info(source_kvm); - kvm_get_kvm(source_kvm); - list_add_tail(&mirror_sev->mirror_entry, &source_sev->mirror_vms); /* Set enc_context_owner and copy its encryption context over */ + mutex_lock(&sev_mirror_lock); + kvm_get_kvm(source_kvm); + list_add_tail(&mirror_sev->mirror_entry, &source_sev->mirror_vms); mirror_sev->enc_context_owner = source_kvm; + mutex_unlock(&sev_mirror_lock); + mirror_sev->active = true; mirror_sev->asid = source_sev->asid; mirror_sev->fd = source_sev->fd; @@ -2963,11 +2971,19 @@ void sev_vm_destroy(struct kvm *kvm) * Note, mirror VMs don't support registering encrypted regions. */ if (is_mirroring_enc_context(kvm)) { - struct kvm *owner_kvm = sev->enc_context_owner; + struct kvm *owner_kvm; - mutex_lock(&owner_kvm->lock); + mutex_lock(&sev_mirror_lock); + owner_kvm = sev->enc_context_owner; list_del(&sev->mirror_entry); - mutex_unlock(&owner_kvm->lock); + sev->enc_context_owner = NULL; + + /* + * The reference to owner_kvm cannot move after sev_mirror_lock is + * released. Release it before kvm_put_kvm() so that owner_kvm is + * never destroyed inside sev_mirror_lock. + */ + mutex_unlock(&sev_mirror_lock); kvm_put_kvm(owner_kvm); return; } diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c index d0971685034b..60495447f0ca 100644 --- a/arch/x86/kvm/svm/svm.c +++ b/arch/x86/kvm/svm/svm.c @@ -4227,13 +4227,6 @@ static void svm_flush_tlb_all(struct kvm_vcpu *vcpu) svm_flush_tlb_asid(vcpu); } -static void svm_flush_tlb_gva(struct kvm_vcpu *vcpu, gva_t gva) -{ - struct vcpu_svm *svm = to_svm(vcpu); - - invlpga(gva, svm->vmcb->control.asid); -} - static void svm_flush_tlb_guest(struct kvm_vcpu *vcpu) { kvm_register_mark_dirty(vcpu, VCPU_REG_ERAPS); @@ -4241,6 +4234,26 @@ static void svm_flush_tlb_guest(struct kvm_vcpu *vcpu) svm_flush_tlb_asid(vcpu); } +static void svm_flush_tlb_gva(struct kvm_vcpu *vcpu, gva_t gva, bool *full) +{ + struct vcpu_svm *svm = to_svm(vcpu); + + /* + * INVLPGA has had errata on Genoa and Turin, and even on older + * generations there were reports of Windows BSODs if INVLPGA + * was used for Hyper-V tlbflush. Use it only for shadow paging + * where it seems to be okay. + */ + if (!npt_enabled) { + invlpga(gva, svm->vmcb->control.asid); + return; + } + + svm_flush_tlb_guest(vcpu); + if (full) + *full = true; +} + static inline void sync_cr8_to_lapic(struct kvm_vcpu *vcpu) { struct vcpu_svm *svm = to_svm(vcpu); diff --git a/arch/x86/kvm/svm/svm.h b/arch/x86/kvm/svm/svm.h index 716be21fba33..d63e5878988a 100644 --- a/arch/x86/kvm/svm/svm.h +++ b/arch/x86/kvm/svm/svm.h @@ -109,6 +109,7 @@ struct kvm_sev_info { u64 ap_jump_table; /* SEV-ES AP Jump Table address */ u64 vmsa_features; u16 ghcb_version; /* Highest guest GHCB protocol version allowed */ + /* The three fields below are protected by sev_mirror_lock */ struct kvm *enc_context_owner; /* Owner of copied encryption context */ struct list_head mirror_vms; /* List of VMs mirroring */ struct list_head mirror_entry; /* Use as a list entry of mirrors */ diff --git a/arch/x86/kvm/vmx/main.c b/arch/x86/kvm/vmx/main.c index 83d9921277ea..f204a0fc0a57 100644 --- a/arch/x86/kvm/vmx/main.c +++ b/arch/x86/kvm/vmx/main.c @@ -535,12 +535,12 @@ static void vt_flush_tlb_current(struct kvm_vcpu *vcpu) vmx_flush_tlb_current(vcpu); } -static void vt_flush_tlb_gva(struct kvm_vcpu *vcpu, gva_t addr) +static void vt_flush_tlb_gva(struct kvm_vcpu *vcpu, gva_t addr, bool *full) { if (is_td_vcpu(vcpu)) return; - vmx_flush_tlb_gva(vcpu, addr); + vmx_flush_tlb_gva(vcpu, addr, full); } static void vt_flush_tlb_guest(struct kvm_vcpu *vcpu) diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c index cc75feec05da..b8d745f6fd22 100644 --- a/arch/x86/kvm/vmx/vmx.c +++ b/arch/x86/kvm/vmx/vmx.c @@ -3361,7 +3361,7 @@ void vmx_flush_tlb_current(struct kvm_vcpu *vcpu) vpid_sync_context(vmx_get_current_vpid(vcpu)); } -void vmx_flush_tlb_gva(struct kvm_vcpu *vcpu, gva_t addr) +void vmx_flush_tlb_gva(struct kvm_vcpu *vcpu, gva_t addr, bool *full) { /* * vpid_sync_vcpu_addr() is a nop if vpid==0, see the comment in diff --git a/arch/x86/kvm/vmx/x86_ops.h b/arch/x86/kvm/vmx/x86_ops.h index 409858074246..17595d52985c 100644 --- a/arch/x86/kvm/vmx/x86_ops.h +++ b/arch/x86/kvm/vmx/x86_ops.h @@ -82,7 +82,7 @@ void vmx_set_rflags(struct kvm_vcpu *vcpu, unsigned long rflags); bool vmx_get_if_flag(struct kvm_vcpu *vcpu); void vmx_flush_tlb_all(struct kvm_vcpu *vcpu); void vmx_flush_tlb_current(struct kvm_vcpu *vcpu); -void vmx_flush_tlb_gva(struct kvm_vcpu *vcpu, gva_t addr); +void vmx_flush_tlb_gva(struct kvm_vcpu *vcpu, gva_t addr, bool *full); void vmx_flush_tlb_guest(struct kvm_vcpu *vcpu); void vmx_set_interrupt_shadow(struct kvm_vcpu *vcpu, int mask); u32 vmx_get_interrupt_shadow(struct kvm_vcpu *vcpu); diff --git a/arch/x86/lib/retpoline.S b/arch/x86/lib/retpoline.S index 8f1fed0c3b83..f9ca1d8eabca 100644 --- a/arch/x86/lib/retpoline.S +++ b/arch/x86/lib/retpoline.S @@ -207,10 +207,24 @@ __EXPORT_THUNK(srso_alias_untrain_ret) .pushsection .text..__x86.rethunk_safe SYM_CODE_START_NOALIGN(srso_alias_safe_ret) + + /* + * Tell objtool that those are not function pointers referenced by + * __HANDLE_INTR_SAFERET(). Below too. + */ + ANNOTATE_NOENDBR + + /* + * Safe-RET sequence. If you need to change it, adjust + * handle_interrupted_saferet() too. + */ lea 8(%_ASM_SP), %_ASM_SP UNWIND_HINT_FUNC + + ANNOTATE_NOENDBR ANNOTATE_UNRET_SAFE ret + /* End of Safe-RET sequence */ int3 SYM_FUNC_END(srso_alias_safe_ret) @@ -245,8 +259,14 @@ SYM_CODE_START_LOCAL_NOALIGN(srso_untrain_ret) * the stack. */ SYM_INNER_LABEL(srso_safe_ret, SYM_L_GLOBAL) + /* + * Safe-RET sequence. If you need to change it, adjust + * handle_interrupted_saferet() too. + */ lea 8(%_ASM_SP), %_ASM_SP ret + /* End of Safe-RET sequence */ + int3 int3 /* end of movabs */ diff --git a/drivers/accel/amdxdna/amdxdna_gem.c b/drivers/accel/amdxdna/amdxdna_gem.c index 4628a2787265..1c63eff0a4a8 100644 --- a/drivers/accel/amdxdna/amdxdna_gem.c +++ b/drivers/accel/amdxdna/amdxdna_gem.c @@ -436,6 +436,23 @@ static void amdxdna_gem_dev_obj_free(struct drm_gem_object *gobj) amdxdna_gem_destroy_obj(abo); } +static void amdxdna_mark_mapp_invalid(struct amdxdna_gem_obj *abo, + struct vm_area_struct *vma) +{ + struct amdxdna_dev *xdna = to_xdna_dev(to_gobj(abo)->dev); + struct amdxdna_umap *mapp; + + down_write(&xdna->notifier_lock); + abo->mem.map_invalid = true; + list_for_each_entry(mapp, &abo->mem.umap_list, node) { + if (compare_range(mapp, vma->vm_mm, vma->vm_start, vma->vm_end)) { + mapp->invalid = true; + break; + } + } + up_write(&xdna->notifier_lock); +} + static int amdxdna_insert_pages(struct amdxdna_gem_obj *abo, struct vm_area_struct *vma) { @@ -450,26 +467,17 @@ static int amdxdna_insert_pages(struct amdxdna_gem_obj *abo, XDNA_ERR(xdna, "Failed shmem mmap %d", ret); return ret; } - - /* The buffer is based on memory pages. Fix the flag. */ - vm_flags_mod(vma, VM_MIXEDMAP, VM_PFNMAP); - ret = vm_insert_pages(vma, vma->vm_start, abo->base.pages, - &num_pages); + } else { + vma->vm_private_data = NULL; + vma->vm_ops = NULL; + ret = dma_buf_mmap(abo->dma_buf, vma, 0); if (ret) { - XDNA_ERR(xdna, "Failed insert pages %d", ret); - vma->vm_ops->close(vma); + XDNA_ERR(xdna, "Failed to mmap dma buf %d", ret); return ret; } - return 0; - } - - vma->vm_private_data = NULL; - vma->vm_ops = NULL; - ret = dma_buf_mmap(abo->dma_buf, vma, 0); - if (ret) { - XDNA_ERR(xdna, "Failed to mmap dma buf %d", ret); - return ret; + /* Drop the reference drm_gem_mmap_obj() acquired.*/ + drm_gem_object_put(to_gobj(abo)); } do { @@ -478,17 +486,14 @@ static int amdxdna_insert_pages(struct amdxdna_gem_obj *abo, fault_ret = handle_mm_fault(vma, vma->vm_start + offset, FAULT_FLAG_WRITE, NULL); if (fault_ret & VM_FAULT_ERROR) { - vma->vm_ops->close(vma); XDNA_ERR(xdna, "Fault in page failed"); - return -EFAULT; + amdxdna_mark_mapp_invalid(abo, vma); + break; } offset += PAGE_SIZE; } while (--num_pages); - /* Drop the reference drm_gem_mmap_obj() acquired.*/ - drm_gem_object_put(to_gobj(abo)); - return 0; } diff --git a/drivers/android/binder/process.rs b/drivers/android/binder/process.rs index cdd1a9079726..5b8f73ec1931 100644 --- a/drivers/android/binder/process.rs +++ b/drivers/android/binder/process.rs @@ -1586,6 +1586,10 @@ fn ioctl_write_only( cmd: u32, reader: &mut UserSliceReader, ) -> Result { + if cmd == uapi::BINDER_FREEZE { + return ioctl_freeze(reader); + } + let thread = this.get_current_thread()?; match cmd { uapi::BINDER_SET_MAX_THREADS => this.set_max_threads(reader.read()?), @@ -1597,7 +1601,6 @@ fn ioctl_write_only( uapi::BINDER_ENABLE_ONEWAY_SPAM_DETECTION => { this.set_oneway_spam_detection_enabled(reader.read()?) } - uapi::BINDER_FREEZE => ioctl_freeze(reader)?, _ => return Err(EINVAL), } Ok(()) @@ -1612,15 +1615,16 @@ fn ioctl_write_read( cmd: u32, data: UserSlice, ) -> Result { - let thread = this.get_current_thread()?; let blocking = (file.flags() & file::flags::O_NONBLOCK) == 0; match cmd { - uapi::BINDER_WRITE_READ => thread.write_read(data, blocking)?, + uapi::BINDER_WRITE_READ => this.get_current_thread()?.write_read(data, blocking)?, uapi::BINDER_GET_NODE_DEBUG_INFO => this.get_node_debug_info(data)?, uapi::BINDER_GET_NODE_INFO_FOR_REF => this.get_node_info_from_ref(data)?, uapi::BINDER_VERSION => this.version(data)?, uapi::BINDER_GET_FROZEN_INFO => get_frozen_status(data)?, - uapi::BINDER_GET_EXTENDED_ERROR => thread.get_extended_error(data)?, + uapi::BINDER_GET_EXTENDED_ERROR => { + this.get_current_thread()?.get_extended_error(data)? + } _ => return Err(EINVAL), } Ok(()) diff --git a/drivers/ata/libata-core.c b/drivers/ata/libata-core.c index 961d03fbc4d9..59e87b987d73 100644 --- a/drivers/ata/libata-core.c +++ b/drivers/ata/libata-core.c @@ -4420,6 +4420,7 @@ static const struct ata_dev_quirks_entry __ata_dev_quirks[] = { */ { "WDC WD100EFGX-68CPLN0", NULL, ATA_QUIRK_NOLPM }, { "WDC WD102KFBX-68M95N0", NULL, ATA_QUIRK_NOLPM }, + { "WDC WD141KFGX-68FH9N0", NULL, ATA_QUIRK_NOLPM }, { "WD Green 2.5 480GB", NULL, ATA_QUIRK_NOLPM }, /* diff --git a/drivers/ata/pata_sl82c105.c b/drivers/ata/pata_sl82c105.c index 2d24c6b3e9d9..25bce3fe2a10 100644 --- a/drivers/ata/pata_sl82c105.c +++ b/drivers/ata/pata_sl82c105.c @@ -264,6 +264,7 @@ static struct ata_port_operations sl82c105_port_ops = { static int sl82c105_bridge_revision(struct pci_dev *pdev) { struct pci_dev *bridge; + u8 revision; /* * The bridge should be part of the same device, but function 0. @@ -285,8 +286,9 @@ static int sl82c105_bridge_revision(struct pci_dev *pdev) /* * We need to find function 0's revision, not function 1 */ + revision = bridge->revision; pci_dev_put(bridge); - return bridge->revision; + return revision; } static void sl82c105_fixup(struct pci_dev *pdev) diff --git a/drivers/counter/microchip-tcb-capture.c b/drivers/counter/microchip-tcb-capture.c index 19d457ae4c3b..e53a8390756b 100644 --- a/drivers/counter/microchip-tcb-capture.c +++ b/drivers/counter/microchip-tcb-capture.c @@ -483,7 +483,7 @@ static int mchp_tc_probe(struct platform_device *pdev) char clk_name[7]; struct regmap *regmap; struct clk *clk[3]; - int channel; + u32 channel; int ret, i; counter = devm_counter_alloc(&pdev->dev, sizeof(*priv)); @@ -517,7 +517,7 @@ static int mchp_tc_probe(struct platform_device *pdev) priv->channel[i] = channel; - snprintf(clk_name, sizeof(clk_name), "t%d_clk", channel); + snprintf(clk_name, sizeof(clk_name), "t%u_clk", channel); clk[i] = of_clk_get_by_name(np->parent, clk_name); if (IS_ERR(clk[i])) { diff --git a/drivers/dibs/dibs_main.c b/drivers/dibs/dibs_main.c index 14c3e2d84902..4c26fd06973f 100644 --- a/drivers/dibs/dibs_main.c +++ b/drivers/dibs/dibs_main.c @@ -138,6 +138,7 @@ struct dibs_dev *dibs_dev_alloc(void) dibs = kzalloc_obj(*dibs); if (!dibs) return dibs; + spin_lock_init(&dibs->lock); dibs->dev.release = dibs_dev_release; dibs->dev.class = &dibs_class; device_initialize(&dibs->dev); @@ -186,7 +187,6 @@ int dibs_dev_add(struct dibs_dev *dibs) int i, ret; max_dmbs = dibs->ops->max_dmbs(); - spin_lock_init(&dibs->lock); dibs->dmb_clientid_arr = kzalloc(max_dmbs, GFP_KERNEL); if (!dibs->dmb_clientid_arr) return -ENOMEM; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c index 5445f75741b5..f8bf0f6b5097 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_cs.c @@ -42,6 +42,26 @@ #include "amdgpu_ras.h" #include "amdgpu_hmm.h" +/* + * Maximum IB length (dwords) for rings whose emit_ib packet format + * documents a 20-bit size field. + */ +#define AMDGPU_GFX_SDMA_IB_PACKET_SIZE_MAX_DW 0xFFFFF +#define AMDGPU_MM_IB_PACKET_SIZE_MAX_DW 0x7FFFF0 + +static u32 amdgpu_cs_ib_packet_size_max_dw(enum amdgpu_ring_type type) +{ + switch (type) { + case AMDGPU_RING_TYPE_GFX: + case AMDGPU_RING_TYPE_COMPUTE: + case AMDGPU_RING_TYPE_SDMA: + case AMDGPU_RING_TYPE_VPE: + return AMDGPU_GFX_SDMA_IB_PACKET_SIZE_MAX_DW; + default: + return AMDGPU_MM_IB_PACKET_SIZE_MAX_DW; + } +} + static int amdgpu_cs_parser_init(struct amdgpu_cs_parser *p, struct amdgpu_device *adev, struct drm_file *filp, @@ -345,7 +365,6 @@ static int amdgpu_cs_p2_ib(struct amdgpu_cs_parser *p, job = p->jobs[r]; ring = amdgpu_job_ring(job); - ib = &job->ibs[job->num_ibs++]; /* submissions to kernel queues are disabled */ if (ring->no_user_submission) @@ -374,6 +393,12 @@ static int amdgpu_cs_p2_ib(struct amdgpu_cs_parser *p, return -EINVAL; } + if (chunk_ib->ib_bytes / 4 > + amdgpu_cs_ib_packet_size_max_dw(ring->funcs->type)) + return -EINVAL; + + ib = &job->ibs[job->num_ibs++]; + if (chunk_ib->flags & AMDGPU_IB_FLAG_PREAMBLE) job->preamble_status |= AMDGPU_PREAMBLE_IB_PRESENT; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_dev_coredump.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_dev_coredump.c index 6480a344006d..45dfe1c8a11b 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_dev_coredump.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_dev_coredump.c @@ -342,7 +342,7 @@ amdgpu_devcoredump_format(char *buffer, size_t count, struct amdgpu_coredump_inf struct amdgpu_ip_block *ip_block; struct amdgpu_ring *ring; int ver, i, j; - u32 ring_idx, off; + u32 ring_idx; bool sizing_pass; sizing_pass = buffer == NULL; @@ -442,7 +442,6 @@ amdgpu_devcoredump_format(char *buffer, size_t count, struct amdgpu_coredump_inf for (i = 0; i < coredump->num_rings; i++) { ring_idx = coredump->rings[i].ring_index; ring = coredump->adev->rings[ring_idx]; - off = coredump->rings[i].offset; drm_printf(&p, "ring name: %s\n", ring->name); drm_printf(&p, "Rptr: 0x%llx Wptr: 0x%llx RB mask: %x\n", @@ -451,12 +450,18 @@ amdgpu_devcoredump_format(char *buffer, size_t count, struct amdgpu_coredump_inf ring->buf_mask); drm_printf(&p, "Ring size in dwords: %d\n", ring->ring_size / 4); + + if (!coredump->rings[i].ring_dw) { + drm_printf(&p, "Ring contents unavailable\n"); + continue; + } + drm_printf(&p, "Ring contents\n"); drm_printf(&p, "Offset \t Value\n"); for (j = 0; j < ring->ring_size; j += 4) drm_printf(&p, "0x%x \t 0x%x\n", j, - coredump->rings_dw[off + j / 4]); + coredump->rings[i].ring_dw[j / 4]); } } @@ -497,10 +502,12 @@ amdgpu_devcoredump_read(char *buffer, loff_t offset, size_t count, static void amdgpu_devcoredump_free(void *data) { struct amdgpu_coredump_info *coredump = data; + u32 i; kvfree(coredump->formatted); + for (i = 0; i < coredump->num_rings; i++) + kvfree(coredump->rings[i].ring_dw); kvfree(coredump->rings); - kvfree(coredump->rings_dw); kvfree(data); } @@ -542,9 +549,9 @@ void amdgpu_coredump(struct amdgpu_device *adev, bool skip_vram_check, struct amdgpu_coredump_info *coredump; size_t size = sizeof(*coredump); struct drm_sched_job *s_job; - u64 total_ring_size, ring_count; + u64 ring_count; struct amdgpu_ring *ring; - int i, off, idx; + int i, idx; /* No need to generate a new coredump if there's one in progress already. */ if (work_busy(&adev->coredump_work)) @@ -553,7 +560,7 @@ void amdgpu_coredump(struct amdgpu_device *adev, bool skip_vram_check, if (job && job->pasid) size += sizeof(struct amdgpu_coredump_ib_info) * job->num_ibs; - coredump = kzalloc(size, GFP_NOWAIT); + coredump = kvzalloc(size, GFP_NOWAIT); if (!coredump) return; @@ -584,7 +591,6 @@ void amdgpu_coredump(struct amdgpu_device *adev, bool skip_vram_check, /* Dump ring content if memory allocation succeeds. */ ring_count = 0; - total_ring_size = 0; for (i = 0; i < adev->num_rings; i++) { ring = adev->rings[i]; @@ -593,34 +599,34 @@ void amdgpu_coredump(struct amdgpu_device *adev, bool skip_vram_check, coredump->ring != ring) continue; - total_ring_size += ring->ring_size; ring_count++; } - coredump->rings_dw = kzalloc(total_ring_size, GFP_NOWAIT); - coredump->rings = kcalloc(ring_count, sizeof(struct amdgpu_coredump_ring), GFP_NOWAIT); - if (coredump->rings && coredump->rings_dw) { - for (i = 0, off = 0, idx = 0; i < adev->num_rings && idx < ring_count; i++) { + if (ring_count) + coredump->rings = kvcalloc(ring_count, + sizeof(struct amdgpu_coredump_ring), + GFP_NOWAIT); + if (coredump->rings) { + for (i = 0, idx = 0; i < adev->num_rings && idx < ring_count; i++) { + struct amdgpu_coredump_ring *cdump_ring; + ring = adev->rings[i]; if (atomic_read(&ring->fence_drv.last_seq) == ring->fence_drv.sync_seq && coredump->ring != ring) continue; - coredump->rings[idx].ring_index = ring->idx; - coredump->rings[idx].rptr = amdgpu_ring_get_rptr(ring); - coredump->rings[idx].wptr = amdgpu_ring_get_wptr(ring); - coredump->rings[idx].offset = off; + cdump_ring = &coredump->rings[idx]; - memcpy(&coredump->rings_dw[off], ring->ring, ring->ring_size); - off += ring->ring_size / 4; + cdump_ring->ring_dw = kvzalloc(ring->ring_size, GFP_NOWAIT); + if (cdump_ring->ring_dw) + memcpy(cdump_ring->ring_dw, ring->ring, ring->ring_size); + + cdump_ring->ring_index = ring->idx; + cdump_ring->rptr = amdgpu_ring_get_rptr(ring); + cdump_ring->wptr = amdgpu_ring_get_wptr(ring); idx++; } coredump->num_rings = idx; - } else { - kvfree(coredump->rings_dw); - kvfree(coredump->rings); - coredump->rings_dw = NULL; - coredump->rings = NULL; } coredump->adev = adev; diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_dev_coredump.h b/drivers/gpu/drm/amd/amdgpu/amdgpu_dev_coredump.h index 63f27337c09a..cb874d8c3394 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_dev_coredump.h +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_dev_coredump.h @@ -34,8 +34,8 @@ struct amdgpu_coredump_ring { u64 rptr; u64 wptr; + u32 *ring_dw; u32 ring_index; - u32 offset; }; struct amdgpu_coredump_ib_info { @@ -53,7 +53,6 @@ struct amdgpu_coredump_info { struct amdgpu_ring *ring; struct amdgpu_coredump_ring *rings; - u32 *rings_dw; u32 num_rings; /* Readable form of coredevdump, generate once to speed up diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index e5f26e5892ba..bff3e06a756c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -4194,6 +4194,8 @@ static void amdgpu_device_unmap_mmio(struct amdgpu_device *adev) iounmap(adev->rmmio); adev->rmmio = NULL; + if (adev->mman.aper_base_kaddr) + iounmap(adev->mman.aper_base_kaddr); adev->mman.aper_base_kaddr = NULL; /* Memory manager related */ diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_lockdep.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_lockdep.c index 61450af539a6..d5787d848d04 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_lockdep.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_lockdep.c @@ -135,6 +135,21 @@ int amdgpu_lockdep_init(void) lockdep_set_class(&locks->srbm_mutex, &amdgpu_srbm_lock_key); lockdep_set_class(&locks->grbm_idx_mutex, &amdgpu_grbm_lock_key); lockdep_set_class(&locks->mmio_idx_lock, &amdgpu_mmio_lock_key); + + /* + * Register fs_reclaim lock class FIRST, before taking any locks. + * + * This acquire/release pair does NOT create a static lockdep edge + * (no locks are held between acquire and release). It only registers + * the fs_reclaim lock class with lockdep. + * + * The actual fs_reclaim -> notifier_lock dependency is established at + * RUNTIME when memory reclaim invokes MMU notifiers: + * fs_reclaim (held by reclaim) -> notifier_lock (acquired in callback) + */ + fs_reclaim_acquire(GFP_KERNEL); + fs_reclaim_release(GFP_KERNEL); + /* * Take locks in the correct order to train lockdep. * This establishes the dependency chain. @@ -154,11 +169,6 @@ int amdgpu_lockdep_init(void) /* Level 6: Reset control lock */ mutex_lock(&locks->reset_lock); - /* - * Mark potential memory reclaim boundary. - * GPU operations might trigger memory allocation/reclaim. - */ - fs_reclaim_acquire(GFP_KERNEL); /* Level 7: SRBM register access */ mutex_lock(&locks->srbm_mutex); @@ -176,7 +186,6 @@ int amdgpu_lockdep_init(void) spin_unlock_irqrestore(&locks->mmio_idx_lock, flags); mutex_unlock(&locks->grbm_idx_mutex); mutex_unlock(&locks->srbm_mutex); - fs_reclaim_release(GFP_KERNEL); mutex_unlock(&locks->reset_lock); up_read(&reset_domain->sem); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c index 5fc91f45db5b..eaa86e32912e 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c @@ -2120,23 +2120,17 @@ int amdgpu_ttm_init(struct amdgpu_device *adev) /* Change the size here instead of the init above so only lpfn is affected */ amdgpu_ttm_disable_buffer_funcs(adev); #ifdef CONFIG_64BIT - if (adev->gmc.xgmi.connected_to_cpu) { - void *kaddr = devm_memremap(adev->dev, adev->gmc.aper_base, - adev->gmc.visible_vram_size, - MEMREMAP_WB); - if (IS_ERR(kaddr)) - return PTR_ERR(kaddr); - adev->mman.aper_base_kaddr = (__force void __iomem *)kaddr; - } else if (adev->gmc.is_app_apu) { +#ifdef CONFIG_X86 + if (adev->gmc.xgmi.connected_to_cpu) + adev->mman.aper_base_kaddr = ioremap_cache(adev->gmc.aper_base, + adev->gmc.visible_vram_size); + else if (adev->gmc.is_app_apu) DRM_DEBUG_DRIVER( "No need to ioremap when real vram size is 0\n"); - } else { - adev->mman.aper_base_kaddr = devm_ioremap_wc(adev->dev, - adev->gmc.aper_base, - adev->gmc.visible_vram_size); - if (!adev->mman.aper_base_kaddr) - return -ENOMEM; - } + else +#endif + adev->mman.aper_base_kaddr = ioremap_wc(adev->gmc.aper_base, + adev->gmc.visible_vram_size); #endif amdgpu_ttm_init_vram_resv_regions(adev); @@ -2287,7 +2281,10 @@ void amdgpu_ttm_fini(struct amdgpu_device *adev) amdgpu_ttm_unmark_vram_reserved(adev, AMDGPU_RESV_FW_VRAM_USAGE); amdgpu_ttm_unmark_vram_reserved(adev, AMDGPU_RESV_DRV_VRAM_USAGE); - adev->mman.aper_base_kaddr = NULL; + if (adev->mman.aper_base_kaddr) { + iounmap(adev->mman.aper_base_kaddr); + adev->mman.aper_base_kaddr = NULL; + } if (!adev->gmc.is_app_apu) amdgpu_vram_mgr_fini(adev); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c index 3efe1ae27386..b18d78720656 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_userq.c @@ -700,7 +700,12 @@ amdgpu_userq_create(struct drm_file *filp, union drm_amdgpu_userq *args) if (!adev->userq_halt_for_enforce_isolation || ((queue->queue_type != AMDGPU_HW_IP_GFX) && (queue->queue_type != AMDGPU_HW_IP_COMPUTE))) { + /* Serialize the map against an in-progress GPU reset (MES is + * unresponsive during recovery), matching amdgpu_userq_cleanup(). + */ + down_read(&adev->reset_domain->sem); r = amdgpu_userq_map_helper(queue); + up_read(&adev->reset_domain->sem); if (r) { drm_file_err(uq_mgr->file, "Failed to map Queue\n"); mutex_unlock(&uq_mgr->userq_mutex); diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c b/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c index da668a8d6abd..9d7a75670bee 100644 --- a/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c +++ b/drivers/gpu/drm/amd/amdgpu/gfx_v12_0.c @@ -1823,6 +1823,11 @@ static void gfx_v12_0_constants_init(struct amdgpu_device *adev) gfx_v12_0_get_tcc_info(adev); adev->gfx.config.pa_sc_tile_steering_override = 0; + /* Set whether texture coordinate truncation is conformant. */ + tmp = RREG32_SOC15(GC, 0, regTA_CNTL2); + adev->gfx.config.ta_cntl2_truncate_coord_mode = + REG_GET_FIELD(tmp, TA_CNTL2, TRUNCATE_COORD_MODE); + /* XXX SH_MEM regs */ /* where to put LDS, scratch, GPUVM in FSA64 space */ mutex_lock(&adev->srbm_mutex); diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v12_1.c b/drivers/gpu/drm/amd/amdgpu/gmc_v12_1.c index 855cd29cbffa..5fe43f7eab29 100644 --- a/drivers/gpu/drm/amd/amdgpu/gmc_v12_1.c +++ b/drivers/gpu/drm/amd/amdgpu/gmc_v12_1.c @@ -267,9 +267,24 @@ static bool gmc_v12_1_get_vmid_pasid_mapping_info(struct amdgpu_device *adev, * by the amdgpu vm/hsa code. */ +/** + * gmc_v12_1_use_invalidate_semaphore - judge whether to use semaphore + * + * @adev: amdgpu_device pointer + * @vmhub: vmhub type + * + */ +static bool gmc_v12_1_use_invalidate_semaphore(struct amdgpu_device *adev, + uint32_t vmhub) +{ + return ((!AMDGPU_IS_GFXHUB(vmhub)) && + (!amdgpu_sriov_vf(adev))); +} + static void gmc_v12_1_flush_vm_hub(struct amdgpu_device *adev, uint32_t vmid, unsigned int vmhub, uint32_t flush_type) { + bool use_semaphore = gmc_v12_1_use_invalidate_semaphore(adev, vmhub); struct amdgpu_vmhub *hub = &adev->vmhub[vmhub]; u32 inv_req = hub->vmhub_funcs->get_invalidate_req(vmid, flush_type); u32 tmp; @@ -283,6 +298,19 @@ static void gmc_v12_1_flush_vm_hub(struct amdgpu_device *adev, uint32_t vmid, spin_lock(&adev->gmc.invalidate_lock); + if (use_semaphore) { + for (i = 0; i < adev->usec_timeout; i++) { + /* a read return value of 1 means semaphore acuqire */ + tmp = RREG32_RLC_NO_KIQ(hub->vm_inv_eng0_sem + hub->eng_distance * eng, hub_ip); + if (tmp & 0x1) + break; + udelay(1); + } + + if (i >= adev->usec_timeout) + DRM_ERROR("Timeout waiting for sem acquire in VM flush!\n"); + } + WREG32_RLC_NO_KIQ(hub->vm_inv_eng0_req + hub->eng_distance * eng, inv_req, hub_ip); /* Wait for ACK with a delay.*/ @@ -296,6 +324,9 @@ static void gmc_v12_1_flush_vm_hub(struct amdgpu_device *adev, uint32_t vmid, udelay(1); } + if (use_semaphore) + WREG32_RLC_NO_KIQ(hub->vm_inv_eng0_sem + hub->eng_distance * eng, 0, hub_ip); + /* Issue additional private vm invalidation to MMHUB */ if (!AMDGPU_IS_GFXHUB(vmhub) && (hub->vm_l2_bank_select_reserved_cid2) && @@ -396,7 +427,7 @@ static void gmc_v12_1_flush_gpu_tlb_pasid(struct amdgpu_device *adev, if (all_hub) { /* invalidate mm_hub */ - if (test_bit(AMDGPU_MMHUB1(0), adev->vmhubs_mask)) { + if (test_bit(AMDGPU_MMHUB0(0), adev->vmhubs_mask)) { input.hub_id = AMDGPU_MMHUB0(0); adev->mes.funcs->invalidate_tlbs_pasid(&adev->mes, &input); } @@ -431,10 +462,17 @@ static void gmc_v12_1_flush_gpu_tlb_pasid(struct amdgpu_device *adev, static uint64_t gmc_v12_1_emit_flush_gpu_tlb(struct amdgpu_ring *ring, unsigned vmid, uint64_t pd_addr) { + bool use_semaphore = gmc_v12_1_use_invalidate_semaphore(ring->adev, ring->vm_hub); struct amdgpu_vmhub *hub = &ring->adev->vmhub[ring->vm_hub]; uint32_t req = hub->vmhub_funcs->get_invalidate_req(vmid, 0); unsigned eng = ring->vm_inv_eng; + if (use_semaphore) + /* a read return value of 1 means semaphore acuqire */ + amdgpu_ring_emit_reg_wait(ring, + hub->vm_inv_eng0_sem + + hub->eng_distance * eng, 0x1, 0x1); + amdgpu_ring_emit_wreg(ring, hub->ctx0_ptb_addr_lo32 + (hub->ctx_addr_distance * vmid), lower_32_bits(pd_addr)); @@ -449,6 +487,14 @@ static uint64_t gmc_v12_1_emit_flush_gpu_tlb(struct amdgpu_ring *ring, hub->eng_distance * eng, req, 1 << vmid); + if (use_semaphore) + /* + * add semaphore release after invalidation, + * write with 0 means semaphore release + */ + amdgpu_ring_emit_wreg(ring, hub->vm_inv_eng0_sem + + hub->eng_distance * eng, 0); + return pd_addr; } diff --git a/drivers/gpu/drm/amd/amdgpu/jpeg_v4_0_5.c b/drivers/gpu/drm/amd/amdgpu/jpeg_v4_0_5.c index a43582b9c876..3f61d5367d94 100644 --- a/drivers/gpu/drm/amd/amdgpu/jpeg_v4_0_5.c +++ b/drivers/gpu/drm/amd/amdgpu/jpeg_v4_0_5.c @@ -772,15 +772,28 @@ static int jpeg_v4_0_5_ring_reset(struct amdgpu_ring *ring, unsigned int vmid, struct amdgpu_fence *timedout_fence) { + struct amdgpu_device *adev = ring->adev; + u32 pg_flags = adev->pg_flags; int r; amdgpu_ring_reset_helper_begin(ring, timedout_fence); - r = jpeg_v4_0_5_stop(ring->adev); - if (r) - return r; - r = jpeg_v4_0_5_start(ring->adev); + + /* + * The DPG stop path only clears the JPEG_PG_MODE bit and never resets a + * hung JRBC, so the post-reset ring test times out and the driver falls + * back to a full MODE1 reset. Temporarily force the static power-gating + * path so the stop/start sequence actually power-cycles the JPEG block + * (JMI soft reset + static power off/on), matching the working jpeg_v4_0 + * reset. + */ + adev->pg_flags &= ~AMD_PG_SUPPORT_JPEG_DPG; + r = jpeg_v4_0_5_stop(adev); + if (!r) + r = jpeg_v4_0_5_start(adev); + adev->pg_flags = pg_flags; if (r) return r; + return amdgpu_ring_reset_helper_end(ring, timedout_fence); } diff --git a/drivers/gpu/drm/amd/amdgpu/jpeg_v5_0_0.c b/drivers/gpu/drm/amd/amdgpu/jpeg_v5_0_0.c index 72a4b2d0676f..c696a0dea32c 100644 --- a/drivers/gpu/drm/amd/amdgpu/jpeg_v5_0_0.c +++ b/drivers/gpu/drm/amd/amdgpu/jpeg_v5_0_0.c @@ -648,15 +648,28 @@ static int jpeg_v5_0_0_ring_reset(struct amdgpu_ring *ring, unsigned int vmid, struct amdgpu_fence *timedout_fence) { + struct amdgpu_device *adev = ring->adev; + u32 pg_flags = adev->pg_flags; int r; amdgpu_ring_reset_helper_begin(ring, timedout_fence); - r = jpeg_v5_0_0_stop(ring->adev); - if (r) - return r; - r = jpeg_v5_0_0_start(ring->adev); + + /* + * The DPG stop path only clears the JPEG_PG_MODE bit and never resets a + * hung JRBC, so the post-reset ring test times out and the driver falls + * back to a full MODE1 reset. Temporarily force the static power-gating + * path so the stop/start sequence actually power-cycles the JPEG block + * (JMI soft reset + ONO1 power off/on), matching the working jpeg_v4_0 + * reset. + */ + adev->pg_flags &= ~AMD_PG_SUPPORT_JPEG_DPG; + r = jpeg_v5_0_0_stop(adev); + if (!r) + r = jpeg_v5_0_0_start(adev); + adev->pg_flags = pg_flags; if (r) return r; + return amdgpu_ring_reset_helper_end(ring, timedout_fence); } diff --git a/drivers/gpu/drm/amd/amdgpu/jpeg_v5_3_0.c b/drivers/gpu/drm/amd/amdgpu/jpeg_v5_3_0.c index 2c3e6d9f52a6..33c2626abc80 100644 --- a/drivers/gpu/drm/amd/amdgpu/jpeg_v5_3_0.c +++ b/drivers/gpu/drm/amd/amdgpu/jpeg_v5_3_0.c @@ -655,15 +655,28 @@ static int jpeg_v5_3_0_ring_reset(struct amdgpu_ring *ring, unsigned int vmid, struct amdgpu_fence *timedout_fence) { + struct amdgpu_device *adev = ring->adev; + u32 pg_flags = adev->pg_flags; int r; amdgpu_ring_reset_helper_begin(ring, timedout_fence); - r = jpeg_v5_3_0_stop(ring->adev); - if (r) - return r; - r = jpeg_v5_3_0_start(ring->adev); + + /* + * The DPG stop path only clears the JPEG_PG_MODE bit and never resets a + * hung JRBC, so the post-reset ring test times out and the driver falls + * back to a full MODE1 reset. Temporarily force the static power-gating + * path so the stop/start sequence actually power-cycles the JPEG block + * (JMI soft reset + static power off/on), matching the working jpeg_v4_0 + * reset. + */ + adev->pg_flags &= ~AMD_PG_SUPPORT_JPEG_DPG; + r = jpeg_v5_3_0_stop(adev); + if (!r) + r = jpeg_v5_3_0_start(adev); + adev->pg_flags = pg_flags; if (r) return r; + return amdgpu_ring_reset_helper_end(ring, timedout_fence); } diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c index 1820547b1dde..941c1a312824 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm.c @@ -2189,6 +2189,11 @@ static int amdgpu_dm_init(struct amdgpu_device *adev) init_data.flags.unify_link_enc_assignment = true; init_data.flags.usb4_bw_alloc_support = true; } + + /* DCN201 audio desyncs using DP SS */ + if (adev->apu_flags & AMD_APU_IS_CYAN_SKILLFISH2) + init_data.flags.ignore_dpref_ss = true; + retrieve_dmi_info(&adev->dm); if (adev->dm.edp0_on_dp1_quirk) init_data.flags.support_edp0_on_dp1 = true; diff --git a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crtc.c b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crtc.c index 95d3da3c4199..06598273d481 100644 --- a/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crtc.c +++ b/drivers/gpu/drm/amd/display/amdgpu_dm/amdgpu_dm_crtc.c @@ -119,7 +119,11 @@ void amdgpu_dm_crtc_set_static_screen_optimze( struct dc_link *link = stream->link; bool set_vsync_event = !sso_enable; - if (!allow_sr_entry) + /* + * allow_sr_entry gates only entry. A disable request must still set + * the vsync events to force Replay and PSR1 out and keep them blocked. + */ + if (sso_enable && !allow_sr_entry) return; amdgpu_dm_replay_set_event(dm, stream, diff --git a/drivers/gpu/drm/amd/display/dc/hwss/dce110/dce110_hwseq.c b/drivers/gpu/drm/amd/display/dc/hwss/dce110/dce110_hwseq.c index 53fffe58602f..f5b3a2eff94a 100644 --- a/drivers/gpu/drm/amd/display/dc/hwss/dce110/dce110_hwseq.c +++ b/drivers/gpu/drm/amd/display/dc/hwss/dce110/dce110_hwseq.c @@ -1348,7 +1348,10 @@ void dce110_set_avmute(struct pipe_ctx *pipe_ctx, bool enable) * process the mute state, especially after link re-establishment * with HDMI 2.0 scrambling enabled. */ - if (enable && pipe_ctx->stream_res.tg->funcs->is_tg_enabled(pipe_ctx->stream_res.tg)) { + if (enable && pipe_ctx->stream_res.tg && + pipe_ctx->stream_res.tg->funcs->is_tg_enabled && + pipe_ctx->stream_res.tg->funcs->wait_for_state && + pipe_ctx->stream_res.tg->funcs->is_tg_enabled(pipe_ctx->stream_res.tg)) { int i; pipe_ctx->stream_res.tg->funcs->wait_for_state(pipe_ctx->stream_res.tg, CRTC_STATE_VACTIVE); diff --git a/drivers/gpu/drm/bridge/parade-ps8640.c b/drivers/gpu/drm/bridge/parade-ps8640.c index b93514023baa..39ab94a73c88 100644 --- a/drivers/gpu/drm/bridge/parade-ps8640.c +++ b/drivers/gpu/drm/bridge/parade-ps8640.c @@ -257,8 +257,14 @@ static ssize_t ps8640_aux_transfer_msg(struct drm_dp_aux *aux, addr_len[PAGE0_SWAUX_LENGTH - base] = (len == 0) ? SWAUX_NO_PAYLOAD : ((len - 1) & SWAUX_LENGTH_MASK); - regmap_bulk_write(map, PAGE0_SWAUX_ADDR_7_0, addr_len, - ARRAY_SIZE(addr_len)); + ret = regmap_bulk_write(map, PAGE0_SWAUX_ADDR_7_0, addr_len, + ARRAY_SIZE(addr_len)); + if (ret) { + DRM_DEV_ERROR(dev, + "failed to write AUX address %#x, len %zu: %d\n", + msg->address, len, ret); + return ret; + } if (len && (request == DP_AUX_NATIVE_WRITE || request == DP_AUX_I2C_WRITE)) { @@ -274,13 +280,22 @@ static ssize_t ps8640_aux_transfer_msg(struct drm_dp_aux *aux, } } - regmap_write(map, PAGE0_SWAUX_CTRL, SWAUX_SEND); + ret = regmap_write(map, PAGE0_SWAUX_CTRL, SWAUX_SEND); + if (ret) { + DRM_DEV_ERROR(dev, "failed to start AUX transfer: %d\n", ret); + return ret; + } /* Zero delay loop because i2c transactions are slow already */ - regmap_read_poll_timeout(map, PAGE0_SWAUX_CTRL, data, - !(data & SWAUX_SEND), 0, 50 * 1000); + ret = regmap_read_poll_timeout(map, PAGE0_SWAUX_CTRL, data, + !(data & SWAUX_SEND), 0, 50 * 1000); + if (ret) { + DRM_DEV_ERROR(dev, "failed to complete AUX transfer: %d\n", + ret); + return ret; + } - regmap_read(map, PAGE0_SWAUX_STATUS, &data); + ret = regmap_read(map, PAGE0_SWAUX_STATUS, &data); if (ret) { DRM_DEV_ERROR(dev, "failed to read PAGE0_SWAUX_STATUS: %d\n", ret); diff --git a/drivers/gpu/drm/drm_gem_shmem_helper.c b/drivers/gpu/drm/drm_gem_shmem_helper.c index c989459eb215..00807039f8d7 100644 --- a/drivers/gpu/drm/drm_gem_shmem_helper.c +++ b/drivers/gpu/drm/drm_gem_shmem_helper.c @@ -597,9 +597,13 @@ static vm_fault_t try_insert_pfn(struct vm_fault *vmf, unsigned int order, #ifdef CONFIG_ARCH_SUPPORTS_PMD_PFNMAP } else if (order == PMD_ORDER) { unsigned long paddr = pfn << PAGE_SHIFT; + struct vm_area_struct *vma = vmf->vma; + unsigned long start = ALIGN_DOWN(vmf->address, PMD_SIZE); + unsigned long end = start + PMD_SIZE; + bool in_range = vma->vm_start <= start && end <= vma->vm_end; bool aligned = (vmf->address & ~PMD_MASK) == (paddr & ~PMD_MASK); - if (aligned && + if (aligned && in_range && folio_test_pmd_mappable(page_folio(pfn_to_page(pfn)))) { vm_fault_t ret; diff --git a/drivers/gpu/drm/panthor/panthor_fw.c b/drivers/gpu/drm/panthor/panthor_fw.c index e45933e333b5..fa88b2f9da76 100644 --- a/drivers/gpu/drm/panthor/panthor_fw.c +++ b/drivers/gpu/drm/panthor/panthor_fw.c @@ -603,6 +603,9 @@ static int panthor_fw_load_section_entry(struct panthor_device *ptdev, return -EINVAL; } + if (!section_size) + return 0; + name_len = iter->size - iter->offset; section = drmm_kzalloc(&ptdev->base, sizeof(*section), GFP_KERNEL); diff --git a/drivers/gpu/drm/panthor/panthor_gem.c b/drivers/gpu/drm/panthor/panthor_gem.c index a1e2eb1ca7bb..54535bae2b0c 100644 --- a/drivers/gpu/drm/panthor/panthor_gem.c +++ b/drivers/gpu/drm/panthor/panthor_gem.c @@ -802,9 +802,13 @@ static vm_fault_t insert_page(struct vm_fault *vmf, unsigned int order, struct p } else if (order == PMD_ORDER) { unsigned long pfn = page_to_pfn(page); unsigned long paddr = pfn << PAGE_SHIFT; + struct vm_area_struct *vma = vmf->vma; + unsigned long start = ALIGN_DOWN(vmf->address, PMD_SIZE); + unsigned long end = start + PMD_SIZE; + bool in_range = vma->vm_start <= start && end <= vma->vm_end; bool aligned = (vmf->address & ~PMD_MASK) == (paddr & ~PMD_MASK); - if (aligned && + if (aligned && in_range && folio_test_pmd_mappable(page_folio(page))) { pfn &= PMD_MASK >> PAGE_SHIFT; return vmf_insert_pfn_pmd(vmf, pfn, vmf->flags & FAULT_FLAG_WRITE); diff --git a/drivers/gpu/drm/radeon/radeon_fence.c b/drivers/gpu/drm/radeon/radeon_fence.c index 02a40e4750c7..299865569252 100644 --- a/drivers/gpu/drm/radeon/radeon_fence.c +++ b/drivers/gpu/drm/radeon/radeon_fence.c @@ -360,6 +360,13 @@ static bool radeon_fence_is_signaled(struct dma_fence *f) if (atomic64_read(&rdev->fence_drv[ring].last_seq) >= seq) return true; + if (down_read_trylock(&rdev->exclusive_lock)) { + radeon_fence_activity(rdev, ring); + up_read(&rdev->exclusive_lock); + + if (atomic64_read(&rdev->fence_drv[ring].last_seq) >= seq) + return true; + } return false; } diff --git a/drivers/gpu/drm/v3d/v3d_drv.h b/drivers/gpu/drm/v3d/v3d_drv.h index 423bcfe331a0..8779a42c6508 100644 --- a/drivers/gpu/drm/v3d/v3d_drv.h +++ b/drivers/gpu/drm/v3d/v3d_drv.h @@ -182,6 +182,12 @@ struct v3d_dev { */ struct mutex reset_lock; + /* Ordered workqueue shared by every queue's scheduler timeout work. + * V3D reset is global to all queues, so the timeout handlers must not + * run concurrently. + */ + struct workqueue_struct *reset_wq; + /* Lock taken when creating and pushing the GPU scheduler * jobs, to keep the sched-fence seqnos in order. */ diff --git a/drivers/gpu/drm/v3d/v3d_sched.c b/drivers/gpu/drm/v3d/v3d_sched.c index c35f9f70cf8d..63dd95b828c8 100644 --- a/drivers/gpu/drm/v3d/v3d_sched.c +++ b/drivers/gpu/drm/v3d/v3d_sched.c @@ -837,6 +837,7 @@ v3d_queue_sched_init(struct v3d_dev *v3d, const struct drm_sched_backend_ops *op struct drm_sched_init_args args = { .credit_limit = 1, .timeout = msecs_to_jiffies(500), + .timeout_wq = v3d->reset_wq, .dev = v3d->drm.dev, }; @@ -851,9 +852,13 @@ v3d_sched_init(struct v3d_dev *v3d) { int ret; + v3d->reset_wq = alloc_ordered_workqueue("v3d_reset", 0); + if (!v3d->reset_wq) + return -ENOMEM; + ret = v3d_queue_sched_init(v3d, &v3d_bin_sched_ops, V3D_BIN, "v3d_bin"); if (ret) - return ret; + goto fail; ret = v3d_queue_sched_init(v3d, &v3d_render_sched_ops, V3D_RENDER, "v3d_render"); @@ -896,4 +901,6 @@ v3d_sched_fini(struct v3d_dev *v3d) if (v3d->queue[q].sched.ready) drm_sched_fini(&v3d->queue[q].sched); } + + destroy_workqueue(v3d->reset_wq); } diff --git a/drivers/gpu/drm/xe/xe_exec_queue.c b/drivers/gpu/drm/xe/xe_exec_queue.c index 1b5ca3ce578a..663decb19304 100644 --- a/drivers/gpu/drm/xe/xe_exec_queue.c +++ b/drivers/gpu/drm/xe/xe_exec_queue.c @@ -805,6 +805,9 @@ static int exec_queue_set_hang_replay_state(struct xe_device *xe, u64 __user *address = u64_to_user_ptr(value); void *ptr; + if (q->replay_state) + return -EINVAL; + ptr = vmemdup_user(address, size); if (XE_IOCTL_DBG(xe, IS_ERR(ptr))) return PTR_ERR(ptr); diff --git a/drivers/gpu/drm/xe/xe_uc.c b/drivers/gpu/drm/xe/xe_uc.c index 75091bde0d50..65f59f06177f 100644 --- a/drivers/gpu/drm/xe/xe_uc.c +++ b/drivers/gpu/drm/xe/xe_uc.c @@ -15,6 +15,7 @@ #include "xe_guc_pc.h" #include "xe_guc_rc.h" #include "xe_guc_engine_activity.h" +#include "xe_guc_submit.h" #include "xe_huc.h" #include "xe_sriov.h" #include "xe_wopcm.h" @@ -159,12 +160,14 @@ static int vf_uc_load_hw(struct xe_uc *uc) if (err) return err; - uc->guc.submission_state.enabled = true; - err = xe_guc_opt_in_features_enable(&uc->guc); if (err) return err; + err = xe_guc_submit_enable(&uc->guc); + if (err) + return err; + err = xe_gt_record_default_lrcs(uc_to_gt(uc)); if (err) return err; diff --git a/drivers/hwmon/ads7828.c b/drivers/hwmon/ads7828.c index 149cfcec78dc..f8f2f621dfea 100644 --- a/drivers/hwmon/ads7828.c +++ b/drivers/hwmon/ads7828.c @@ -106,12 +106,11 @@ static int ads7828_probe(struct i2c_client *client) struct ads7828_data *data; struct device *hwmon_dev; unsigned int vref_mv = ADS7828_INT_VREF_MV; - unsigned int vref_uv; + int vref_uv; bool diff_input = false; bool ext_vref = false; unsigned int regval; enum ads7828_chips chip; - struct regulator *reg; data = devm_kzalloc(dev, sizeof(struct ads7828_data), GFP_KERNEL); if (!data) @@ -125,9 +124,11 @@ static int ads7828_probe(struct i2c_client *client) } else if (dev->of_node) { diff_input = of_property_read_bool(dev->of_node, "ti,differential-input"); - reg = devm_regulator_get_optional(dev, "vref"); - if (!IS_ERR(reg)) { - vref_uv = regulator_get_voltage(reg); + vref_uv = devm_regulator_get_enable_read_voltage(dev, "vref"); + if (vref_uv < 0) { + if (vref_uv != -ENODEV) + return vref_uv; + } else { vref_mv = DIV_ROUND_CLOSEST(vref_uv, 1000); if (vref_mv < ADS7828_EXT_VREF_MV_MIN || vref_mv > ADS7828_EXT_VREF_MV_MAX) diff --git a/drivers/hwmon/corsair-psu.c b/drivers/hwmon/corsair-psu.c index ce958cdaef58..0cf072f86acb 100644 --- a/drivers/hwmon/corsair-psu.c +++ b/drivers/hwmon/corsair-psu.c @@ -137,13 +137,18 @@ struct corsairpsu_data { }; /* some values are SMBus LINEAR11 data which need a conversion */ -static int corsairpsu_linear11_to_int(const u16 val, const int scale) +static long corsairpsu_linear11_to_long(const u16 val, const int scale) { const int exp = ((s16)val) >> 11; - const int mant = (((s16)(val & 0x7ff)) << 5) >> 5; - const int result = mant * scale; + const int mant = ((s16)((val & 0x7ff) << 5)) >> 5; + s64 result = mant * scale; - return (exp >= 0) ? (result << exp) : (result >> -exp); + if (exp >= 0) + result *= (int)(1UL << exp); + else + result >>= -exp; + + return clamp(result, LONG_MIN, LONG_MAX); } /* the micro-controller uses percentage values to control pwm */ @@ -263,13 +268,13 @@ static int corsairpsu_get_value(struct corsairpsu_data *priv, u8 cmd, u8 rail, l case PSU_CMD_RAIL_AMPS: case PSU_CMD_TEMP0: case PSU_CMD_TEMP1: - *val = corsairpsu_linear11_to_int(tmp & 0xFFFF, 1000); + *val = corsairpsu_linear11_to_long(tmp & 0xFFFF, 1000); break; case PSU_CMD_FAN: - *val = corsairpsu_linear11_to_int(tmp & 0xFFFF, 1); + *val = corsairpsu_linear11_to_long(tmp & 0xFFFF, 1); break; case PSU_CMD_FAN_PWM_ENABLE: - *val = corsairpsu_linear11_to_int(tmp & 0xFFFF, 1); + *val = corsairpsu_linear11_to_long(tmp & 0xFFFF, 1); /* * 0 = automatic mode, means the micro-controller controls the fan using a plan * which can be modified, but changing this plan is not supported by this @@ -283,12 +288,12 @@ static int corsairpsu_get_value(struct corsairpsu_data *priv, u8 cmd, u8 rail, l *val = 2; break; case PSU_CMD_FAN_PWM: - *val = corsairpsu_linear11_to_int(tmp & 0xFFFF, 1); + *val = corsairpsu_linear11_to_long(tmp & 0xFFFF, 1); *val = corsairpsu_dutycycle_to_pwm(*val); break; case PSU_CMD_RAIL_WATTS: case PSU_CMD_TOTAL_WATTS: - *val = corsairpsu_linear11_to_int(tmp & 0xFFFF, 1000000); + *val = corsairpsu_linear11_to_long(tmp & 0xFFFF, 1000000); break; case PSU_CMD_TOTAL_UPTIME: case PSU_CMD_UPTIME: @@ -664,6 +669,8 @@ static void print_uptime(struct seq_file *seqf, u8 cmd) long val; int ret; + guard(hwmon_lock)(priv->hwmon_dev); + ret = corsairpsu_get_value(priv, cmd, 0, &val); if (ret < 0) { seq_puts(seqf, "N/A\n"); @@ -701,7 +708,7 @@ static int vendor_show(struct seq_file *seqf, void *unused) { struct corsairpsu_data *priv = seqf->private; - seq_printf(seqf, "%s\n", priv->vendor); + seq_printf(seqf, "%.*s\n", REPLY_SIZE, priv->vendor); return 0; } @@ -711,7 +718,7 @@ static int product_show(struct seq_file *seqf, void *unused) { struct corsairpsu_data *priv = seqf->private; - seq_printf(seqf, "%s\n", priv->product); + seq_printf(seqf, "%.*s\n", REPLY_SIZE, priv->product); return 0; } @@ -723,6 +730,8 @@ static int ocpmode_show(struct seq_file *seqf, void *unused) long val; int ret; + guard(hwmon_lock)(priv->hwmon_dev); + /* * The rail mode is switchable on the fly. The RAW interface can be used for this. But it * will not be included here, because I consider it somewhat dangerous for the health of the diff --git a/drivers/hwmon/hwmon.c b/drivers/hwmon/hwmon.c index 55a9a3ddd4aa..29dc90a2c3fe 100644 --- a/drivers/hwmon/hwmon.c +++ b/drivers/hwmon/hwmon.c @@ -1083,7 +1083,6 @@ EXPORT_SYMBOL_GPL(hwmon_device_register_with_info); * @dev: the parent device * @name: hwmon name attribute * @drvdata: driver data to attach to created device - * @extra_groups: pointer to list of additional non-standard attribute groups * * The use of this function is restricted. It is provided for legacy reasons * and must only be called from the thermal subsystem. @@ -1095,13 +1094,12 @@ EXPORT_SYMBOL_GPL(hwmon_device_register_with_info); */ struct device * hwmon_device_register_for_thermal(struct device *dev, const char *name, - void *drvdata, - const struct attribute_group **extra_groups) + void *drvdata) { if (!name || !dev) return ERR_PTR(-EINVAL); - return __hwmon_device_register(dev, name, drvdata, NULL, extra_groups); + return __hwmon_device_register(dev, name, drvdata, NULL, NULL); } EXPORT_SYMBOL_NS_GPL(hwmon_device_register_for_thermal, "HWMON_THERMAL"); diff --git a/drivers/hwmon/ltc4282.c b/drivers/hwmon/ltc4282.c index cc698803f8bf..b1675dc5b3c7 100644 --- a/drivers/hwmon/ltc4282.c +++ b/drivers/hwmon/ltc4282.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include #include @@ -137,7 +138,7 @@ struct ltc4282_state { */ struct ltc4282_cache in0_1_cache[LTC4282_CHAN_VGPIO]; u32 vsense_max; - long power_max; + s64 power_max; u32 rsense; u16 vdd; u16 vfs_out; @@ -613,13 +614,12 @@ static int ltc4282_read(struct device *dev, enum hwmon_sensor_types type, } static int ltc4282_write_power_byte(const struct ltc4282_state *st, u32 reg, - long val) + s64 val) { u32 power; u64 temp; - if (val > st->power_max) - val = st->power_max; + val = clamp(val, 0, st->power_max); temp = val * int_pow(U8_MAX, 2) * st->rsense; power = DIV64_U64_ROUND_CLOSEST(temp, @@ -629,7 +629,7 @@ static int ltc4282_write_power_byte(const struct ltc4282_state *st, u32 reg, } static int ltc4282_write_power_word(const struct ltc4282_state *st, u32 reg, - long val) + u64 val) { u64 temp = int_pow(U16_MAX, 2) * st->rsense, temp_2; __be16 __raw; @@ -930,8 +930,11 @@ static int ltc4282_curr_reset_hist(struct ltc4282_state *st) static int ltc4282_write_curr(struct ltc4282_state *st, u32 attr, long val) { + s32 ulimit = min_t(u64, INT_MAX, + div_u64((u64)INT_MAX * DECA * MICRO, st->rsense)); + u64 val64 = clamp(val, 0, ulimit); /* need to pass it in millivolt */ - u32 in = DIV_ROUND_CLOSEST_ULL((u64)val * st->rsense, DECA * MICRO); + u32 in = DIV_ROUND_CLOSEST_ULL(val64 * st->rsense, DECA * MICRO); switch (attr) { case hwmon_curr_max: @@ -1222,7 +1225,8 @@ static int ltc4282_set_max_limits(struct ltc4282_state *st) return ret; /* Power is given by ISENSE * Vout. */ - st->power_max = DIV_ROUND_CLOSEST(st->vsense_max * DECA * MILLI, st->rsense) * st->vfs_out; + st->power_max = DIV_ROUND_CLOSEST_ULL((u64)st->vsense_max * DECA * MILLI, + st->rsense) * st->vfs_out; ret = ltc4282_write_power_byte(st, LTC4282_POWER_MAX, st->power_max); if (ret) return ret; @@ -1390,7 +1394,7 @@ static int ltc4282_setup(struct ltc4282_state *st, struct device *dev) if (!ret) { int reg_val; - switch (val) { + switch (st->vsense_max) { case 12500: reg_val = 0; break; diff --git a/drivers/hwmon/nzxt-smart2.c b/drivers/hwmon/nzxt-smart2.c index ff0c0bee0e83..efc78a8f58c8 100644 --- a/drivers/hwmon/nzxt-smart2.c +++ b/drivers/hwmon/nzxt-smart2.c @@ -754,7 +754,11 @@ static int nzxt_smart2_hid_probe(struct hid_device *hdev, hid_device_io_start(hdev); - init_device(drvdata, UPDATE_INTERVAL_DEFAULT_MS); + ret = init_device(drvdata, UPDATE_INTERVAL_DEFAULT_MS); + if (ret) { + dev_err(&hdev->dev, "init_device failed: %d\n", ret); + goto out_hw_close; + } drvdata->hwmon = hwmon_device_register_with_info(&hdev->dev, "nzxtsmart2", drvdata, diff --git a/drivers/hwmon/pmbus/lm25066.c b/drivers/hwmon/pmbus/lm25066.c index 2be50cf1bbde..c797e2e8258f 100644 --- a/drivers/hwmon/pmbus/lm25066.c +++ b/drivers/hwmon/pmbus/lm25066.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include "pmbus.h" @@ -542,8 +543,8 @@ static int lm25066_probe(struct i2c_client *client) if (of_property_read_u32(client->dev.of_node, "shunt-resistor-micro-ohms", &shunt)) shunt = 1000; - info->m[PSC_CURRENT_IN] = info->m[PSC_CURRENT_IN] * shunt / 1000; - info->m[PSC_POWER] = info->m[PSC_POWER] * shunt / 1000; + info->m[PSC_CURRENT_IN] = DIV_ROUND_CLOSEST_ULL((u64)info->m[PSC_CURRENT_IN] * shunt, 1000); + info->m[PSC_POWER] = DIV_ROUND_CLOSEST_ULL((u64)info->m[PSC_POWER] * shunt, 1000); #if IS_ENABLED(CONFIG_SENSORS_LM25066_REGULATOR) /* LM25056 doesn't support OPERATION */ diff --git a/drivers/hwmon/pmbus/pmbus_core.c b/drivers/hwmon/pmbus/pmbus_core.c index 5567d37f13fe..3b219ed5386f 100644 --- a/drivers/hwmon/pmbus/pmbus_core.c +++ b/drivers/hwmon/pmbus/pmbus_core.c @@ -45,7 +45,7 @@ module_param(wp, int, 0444); struct pmbus_sensor { struct pmbus_sensor *next; char name[PMBUS_NAME_SIZE]; /* sysfs sensor name */ - struct device_attribute attribute; + struct sensor_device_attribute attribute; u8 page; /* page number */ u8 phase; /* phase number, 0xff for all phases */ u16 reg; /* register */ @@ -68,7 +68,7 @@ struct pmbus_boolean { struct pmbus_label { char name[PMBUS_NAME_SIZE]; /* sysfs label name */ - struct device_attribute attribute; + struct sensor_device_attribute attribute; char label[PMBUS_NAME_SIZE]; /* label */ }; #define to_pmbus_label(_attr) \ @@ -1241,7 +1241,8 @@ static ssize_t pmbus_show_sensor(struct device *dev, struct device_attribute *devattr, char *buf) { struct i2c_client *client = to_i2c_client(dev->parent); - struct pmbus_sensor *sensor = to_pmbus_sensor(devattr); + struct sensor_device_attribute *attr = to_sensor_dev_attr(devattr); + struct pmbus_sensor *sensor = to_pmbus_sensor(attr); struct pmbus_data *data = i2c_get_clientdata(client); s64 val; @@ -1261,7 +1262,8 @@ static ssize_t pmbus_set_sensor(struct device *dev, { struct i2c_client *client = to_i2c_client(dev->parent); struct pmbus_data *data = i2c_get_clientdata(client); - struct pmbus_sensor *sensor = to_pmbus_sensor(devattr); + struct sensor_device_attribute *attr = to_sensor_dev_attr(devattr); + struct pmbus_sensor *sensor = to_pmbus_sensor(attr); s64 val; int ret; u16 regval; @@ -1283,7 +1285,8 @@ static ssize_t pmbus_set_sensor(struct device *dev, static ssize_t pmbus_show_label(struct device *dev, struct device_attribute *da, char *buf) { - struct pmbus_label *label = to_pmbus_label(da); + struct sensor_device_attribute *attr = to_sensor_dev_attr(da); + struct pmbus_label *label = to_pmbus_label(attr); return sysfs_emit(buf, "%s\n", label->label); } @@ -1436,8 +1439,8 @@ static struct pmbus_sensor *pmbus_add_sensor(struct pmbus_data *data, bool update, bool readonly, bool writeonly, bool convert) { + struct sensor_device_attribute *a; struct pmbus_sensor *sensor; - struct device_attribute *a; sensor = devm_kzalloc(data->dev, sizeof(*sensor), GFP_KERNEL); if (!sensor) @@ -1461,12 +1464,11 @@ static struct pmbus_sensor *pmbus_add_sensor(struct pmbus_data *data, sensor->update = update; sensor->convert = convert; sensor->data = -ENODATA; - pmbus_dev_attr_init(a, sensor->name, - readonly ? 0444 : 0644, - writeonly ? pmbus_show_zero : pmbus_show_sensor, - pmbus_set_sensor); + pmbus_attr_init(a, sensor->name, readonly ? 0444 : 0644, + writeonly ? pmbus_show_zero : pmbus_show_sensor, + pmbus_set_sensor, -1); - if (pmbus_add_attribute(data, &a->attr)) + if (pmbus_add_attribute(data, &a->dev_attr.attr)) return NULL; sensor->next = data->sensors; @@ -1483,8 +1485,8 @@ static int pmbus_add_label(struct pmbus_data *data, const char *name, int seq, const char *lstring, int index, int phase) { + struct sensor_device_attribute *a; struct pmbus_label *label; - struct device_attribute *a; label = devm_kzalloc(data->dev, sizeof(*label), GFP_KERNEL); if (!label) @@ -1508,8 +1510,8 @@ static int pmbus_add_label(struct pmbus_data *data, lstring, index, phase); } - pmbus_dev_attr_init(a, label->name, 0444, pmbus_show_label, NULL); - return pmbus_add_attribute(data, &a->attr); + pmbus_attr_init(a, label->name, 0444, pmbus_show_label, NULL, -1); + return pmbus_add_attribute(data, &a->dev_attr.attr); } /* @@ -2397,7 +2399,7 @@ struct pmbus_samples_attr { struct pmbus_samples_reg { int page; struct pmbus_samples_attr *attr; - struct device_attribute dev_attr; + struct sensor_device_attribute attribute; }; static struct pmbus_samples_attr pmbus_samples_registers[] = { @@ -2419,14 +2421,15 @@ static struct pmbus_samples_attr pmbus_samples_registers[] = { } }; -#define to_samples_reg(x) container_of(x, struct pmbus_samples_reg, dev_attr) +#define to_samples_reg(x) container_of(x, struct pmbus_samples_reg, attribute) static ssize_t pmbus_show_samples(struct device *dev, struct device_attribute *devattr, char *buf) { int val; struct i2c_client *client = to_i2c_client(dev->parent); - struct pmbus_samples_reg *reg = to_samples_reg(devattr); + struct sensor_device_attribute *attr = to_sensor_dev_attr(devattr); + struct pmbus_samples_reg *reg = to_samples_reg(attr); scoped_guard(pmbus_lock, client) { val = _pmbus_read_word_data(client, reg->page, 0xff, reg->attr->reg); @@ -2444,7 +2447,8 @@ static ssize_t pmbus_set_samples(struct device *dev, int ret; long val; struct i2c_client *client = to_i2c_client(dev->parent); - struct pmbus_samples_reg *reg = to_samples_reg(devattr); + struct sensor_device_attribute *attr = to_sensor_dev_attr(devattr); + struct pmbus_samples_reg *reg = to_samples_reg(attr); if (kstrtol(buf, 0, &val) < 0) return -EINVAL; @@ -2459,6 +2463,7 @@ static ssize_t pmbus_set_samples(struct device *dev, static int pmbus_add_samples_attr(struct pmbus_data *data, int page, struct pmbus_samples_attr *attr) { + struct sensor_device_attribute *a; struct pmbus_samples_reg *reg; reg = devm_kzalloc(data->dev, sizeof(*reg), GFP_KERNEL); @@ -2468,10 +2473,12 @@ static int pmbus_add_samples_attr(struct pmbus_data *data, int page, reg->attr = attr; reg->page = page; - pmbus_dev_attr_init(®->dev_attr, attr->name, 0644, - pmbus_show_samples, pmbus_set_samples); + a = ®->attribute; - return pmbus_add_attribute(data, ®->dev_attr.attr); + pmbus_attr_init(a, attr->name, 0644, + pmbus_show_samples, pmbus_set_samples, -1); + + return pmbus_add_attribute(data, &a->dev_attr.attr); } static int pmbus_add_samples_attributes(struct i2c_client *client, @@ -2979,9 +2986,15 @@ static void pmbus_notify(struct pmbus_data *data, int page, int reg, int flags) struct device_attribute *da = to_dev_attr(data->group.attrs[i]); struct sensor_device_attribute *attr = to_sensor_dev_attr(da); int index = attr->index; - u16 smask = pb_index_to_mask(index); - u8 spage = pb_index_to_page(index); - u16 sreg = pb_index_to_reg(index); + u16 smask, sreg; + u8 spage; + + if (index == -1) + continue; + + smask = pb_index_to_mask(index); + spage = pb_index_to_page(index); + sreg = pb_index_to_reg(index); if (reg == sreg && page == spage && (smask & flags)) { dev_dbg(data->dev, "sysfs notify: %s", da->attr.name); @@ -3427,6 +3440,8 @@ static int pmbus_write_smbalert_mask(struct i2c_client *client, u8 page, u8 reg, { int ret; + guard(pmbus_lock)(client); + ret = _pmbus_write_word_data(client, page, PMBUS_SMBALERT_MASK, reg | (val << 8)); /* @@ -3662,6 +3677,8 @@ static void pmbus_init_debugfs(struct i2c_client *client, if (!entries) return; + guard(pmbus_lock)(client); + /* * Add device-specific entries. * Please note that the PMBUS standard allows all registers to be diff --git a/drivers/input/evdev.c b/drivers/input/evdev.c index c7325226cb86..3a718d600006 100644 --- a/drivers/input/evdev.c +++ b/drivers/input/evdev.c @@ -21,6 +21,7 @@ #include #include #include +#include #include #include #include "input-compat.h" @@ -67,8 +68,10 @@ static size_t evdev_get_mask_cnt(unsigned int type) [EV_SND] = SND_CNT, [EV_FF] = FF_CNT, }; + unsigned long mask = array_index_mask_nospec(type, EV_CNT); - return (type < EV_CNT) ? counts[type] : 0; + /* Returns 0 for out-of-bounds types, including speculatively */ + return counts[type & mask] & mask; } /* requires the buffer lock to be held */ @@ -146,11 +149,11 @@ static void __evdev_queue_syn_dropped(struct evdev_client *client) struct timespec64 ts = ktime_to_timespec64(ev_time[client->clk_type]); struct input_event ev; + memset(&ev, 0, sizeof(ev)); ev.input_event_sec = ts.tv_sec; ev.input_event_usec = ts.tv_nsec / NSEC_PER_USEC; ev.type = EV_SYN; ev.code = SYN_DROPPED; - ev.value = 0; client->buffer[client->head++] = ev; client->head &= client->bufsize - 1; @@ -218,20 +221,20 @@ static void __pass_event(struct evdev_client *client, client->head &= client->bufsize - 1; if (unlikely(client->head == client->tail)) { + struct input_event ev; + + memset(&ev, 0, sizeof(ev)); + ev.input_event_sec = event->input_event_sec; + ev.input_event_usec = event->input_event_usec; + ev.type = EV_SYN; + ev.code = SYN_DROPPED; + /* * This effectively "drops" all unconsumed events, leaving * EV_SYN/SYN_DROPPED plus the newest event in the queue. */ client->tail = (client->head - 2) & (client->bufsize - 1); - - client->buffer[client->tail] = (struct input_event) { - .input_event_sec = event->input_event_sec, - .input_event_usec = event->input_event_usec, - .type = EV_SYN, - .code = SYN_DROPPED, - .value = 0, - }; - + client->buffer[client->tail] = ev; client->packet_head = client->tail; } @@ -253,6 +256,8 @@ static void evdev_pass_values(struct evdev_client *client, if (client->revoked) return; + memset(&event, 0, sizeof(event)); + ts = ktime_to_timespec64(ev_time[client->clk_type]); event.input_event_sec = ts.tv_sec; event.input_event_usec = ts.tv_nsec / NSEC_PER_USEC; diff --git a/drivers/input/joystick/iforce/iforce-packets.c b/drivers/input/joystick/iforce/iforce-packets.c index effa76bfd8f9..01fee14054fb 100644 --- a/drivers/input/joystick/iforce/iforce-packets.c +++ b/drivers/input/joystick/iforce/iforce-packets.c @@ -155,6 +155,9 @@ void iforce_process_packet(struct iforce *iforce, switch (packet_id) { case 0x01: /* joystick position data */ + if (len < 7) + break; + input_report_abs(dev, ABS_X, (__s16) get_unaligned_le16(data)); input_report_abs(dev, ABS_Y, @@ -170,6 +173,9 @@ void iforce_process_packet(struct iforce *iforce, break; case 0x03: /* wheel position data */ + if (len < 7) + break; + input_report_abs(dev, ABS_WHEEL, (__s16) get_unaligned_le16(data)); input_report_abs(dev, ABS_GAS, 255 - data[2]); @@ -181,6 +187,9 @@ void iforce_process_packet(struct iforce *iforce, break; case 0x02: /* status report */ + if (len < 2) + break; + input_report_key(dev, BTN_DEAD, data[0] & 0x02); input_sync(dev); @@ -200,7 +209,7 @@ void iforce_process_packet(struct iforce *iforce, } } - for (j = 3; j < len; j += 2) + for (j = 3; j + sizeof(u16) <= len; j += sizeof(u16)) mark_core_as_ready(iforce, get_unaligned_le16(data + j)); break; diff --git a/drivers/input/joystick/iforce/iforce-usb.c b/drivers/input/joystick/iforce/iforce-usb.c index 0482eaaecf39..f04370e4191e 100644 --- a/drivers/input/joystick/iforce/iforce-usb.c +++ b/drivers/input/joystick/iforce/iforce-usb.c @@ -158,6 +158,9 @@ static void iforce_usb_irq(struct urb *urb) goto exit; } + if (!urb->actual_length) + goto exit; + iforce_process_packet(iforce, iforce_usb->data_in[0], iforce_usb->data_in + 1, urb->actual_length - 1); diff --git a/drivers/input/joystick/psxpad-spi.c b/drivers/input/joystick/psxpad-spi.c index f902a56d011f..aac77a2cfe46 100644 --- a/drivers/input/joystick/psxpad-spi.c +++ b/drivers/input/joystick/psxpad-spi.c @@ -373,6 +373,7 @@ static int psxpad_spi_probe(struct spi_device *spi) return err; } + spi_set_drvdata(spi, pad); pm_runtime_enable(&spi->dev); return 0; diff --git a/drivers/input/joystick/xpad.c b/drivers/input/joystick/xpad.c index feb8f368f834..2da0b7f1722a 100644 --- a/drivers/input/joystick/xpad.c +++ b/drivers/input/joystick/xpad.c @@ -394,6 +394,7 @@ static const struct xpad_device { { 0x3285, 0x0646, "Nacon Pro Compact", 0, XTYPE_XBOXONE }, { 0x3285, 0x0662, "Nacon Revolution5 Pro", 0, XTYPE_XBOX360 }, { 0x3285, 0x0663, "Nacon Evol-X", 0, XTYPE_XBOXONE }, + { 0x3507, 0x000b, "ZENAIM LEVERLESS", 0, XTYPE_XBOX360 }, { 0x3537, 0x1004, "GameSir T4 Kaleid", 0, XTYPE_XBOX360 }, { 0x3537, 0x100f, "GameSir Nova 2 Lite", 0, XTYPE_XBOX360 }, { 0x3537, 0x1010, "GameSir G7 SE", 0, XTYPE_XBOXONE }, @@ -557,6 +558,7 @@ static const struct usb_device_id xpad_table[] = { XPAD_XBOX360_VENDOR(0x31e3), /* Wooting Keyboards */ XPAD_XBOX360_VENDOR(0x3285), /* Nacon GC-100 */ XPAD_XBOXONE_VENDOR(0x3285), /* Nacon Evol-X */ + XPAD_XBOX360_VENDOR(0x3507), /* ZENAIM Controllers */ XPAD_XBOX360_VENDOR(0x3537), /* GameSir Controllers */ XPAD_XBOXONE_VENDOR(0x3537), /* GameSir Controllers */ XPAD_XBOX360_VENDOR(0x3651), /* CRKD Controllers */ diff --git a/drivers/input/keyboard/atkbd.c b/drivers/input/keyboard/atkbd.c index 8cb4dc6fb165..4c82e988260e 100644 --- a/drivers/input/keyboard/atkbd.c +++ b/drivers/input/keyboard/atkbd.c @@ -1916,6 +1916,13 @@ static const struct dmi_system_id atkbd_dmi_quirk_table[] __initconst = { }, .callback = atkbd_deactivate_fixup, }, + { + .matches = { + DMI_MATCH(DMI_SYS_VENDOR, "HONOR"), + DMI_MATCH(DMI_PRODUCT_NAME, "BCC-N"), + }, + .callback = atkbd_deactivate_fixup, + }, { .matches = { DMI_MATCH(DMI_SYS_VENDOR, "HONOR"), @@ -1923,6 +1930,14 @@ static const struct dmi_system_id atkbd_dmi_quirk_table[] __initconst = { }, .callback = atkbd_deactivate_fixup, }, + { + /* HONOR MagicBook Pro 14 2026 */ + .matches = { + DMI_MATCH(DMI_SYS_VENDOR, "HONOR"), + DMI_MATCH(DMI_PRODUCT_NAME, "ZQC-P"), + }, + .callback = atkbd_deactivate_fixup, + }, { /* Lenovo Yoga Air 14 (83QK) */ .matches = { @@ -1932,9 +1947,10 @@ static const struct dmi_system_id atkbd_dmi_quirk_table[] __initconst = { .callback = atkbd_deactivate_fixup, }, { + /* Xiaomi Book Pro 14 (TM2424) */ .matches = { - DMI_MATCH(DMI_SYS_VENDOR, "HONOR"), - DMI_MATCH(DMI_PRODUCT_NAME, "BCC-N"), + DMI_MATCH(DMI_SYS_VENDOR, "XIAOMI"), + DMI_MATCH(DMI_PRODUCT_NAME, "Xiaomi Book Pro 14"), }, .callback = atkbd_deactivate_fixup, }, diff --git a/drivers/input/misc/cs40l50-vibra.c b/drivers/input/misc/cs40l50-vibra.c index 996d6c38cca4..7ef4534fea2f 100644 --- a/drivers/input/misc/cs40l50-vibra.c +++ b/drivers/input/misc/cs40l50-vibra.c @@ -139,10 +139,10 @@ static struct cs40l50_effect *cs40l50_find_effect(int id, struct list_head *effe static int cs40l50_effect_bank_set(struct cs40l50_work *work_data, struct cs40l50_effect *effect) { - s16 bank_type = work_data->custom_data[0] & CS40L50_CUSTOM_DATA_MASK; + u32 bank_type = work_data->custom_data[0] & CS40L50_CUSTOM_DATA_MASK; if (bank_type >= CS40L50_WVFRM_BANK_NUM) { - dev_err(work_data->vib->dev, "Invalid bank (%d)\n", bank_type); + dev_err(work_data->vib->dev, "Invalid bank (%u)\n", bank_type); return -EINVAL; } @@ -326,6 +326,12 @@ static int cs40l50_add(struct input_dev *dev, struct ff_effect *effect, return -EINVAL; } + if (periodic->custom_len < CS40L50_OWT_CUSTOM_DATA_SIZE) { + dev_err(vib->dev, "Invalid custom data length (%u)\n", + periodic->custom_len); + return -EINVAL; + } + work_data.custom_data = memdup_array_user(effect->u.periodic.custom_data, effect->u.periodic.custom_len, sizeof(s16)); diff --git a/drivers/input/mouse/byd.c b/drivers/input/mouse/byd.c index f5770a3af2f1..5fc3c629590a 100644 --- a/drivers/input/mouse/byd.c +++ b/drivers/input/mouse/byd.c @@ -423,7 +423,7 @@ static void byd_disconnect(struct psmouse *psmouse) struct byd_data *priv = psmouse->private; if (priv) { - timer_delete(&priv->timer); + timer_shutdown_sync(&priv->timer); kfree(psmouse->private); psmouse->private = NULL; } diff --git a/drivers/input/mouse/focaltech.c b/drivers/input/mouse/focaltech.c index 43f9939b7c63..d3ad4af5aa09 100644 --- a/drivers/input/mouse/focaltech.c +++ b/drivers/input/mouse/focaltech.c @@ -197,7 +197,7 @@ static void focaltech_process_rel_packet(struct psmouse *psmouse, { struct focaltech_data *priv = psmouse->private; struct focaltech_hw_state *state = &priv->state; - int finger1, finger2; + unsigned int finger1, finger2; state->pressed = packet[0] >> 7; finger1 = ((packet[0] >> 4) & 0x7) - 1; diff --git a/drivers/input/rmi4/rmi_f54.c b/drivers/input/rmi4/rmi_f54.c index 61909e1a39e2..6c6cdec7da9e 100644 --- a/drivers/input/rmi4/rmi_f54.c +++ b/drivers/input/rmi4/rmi_f54.c @@ -104,7 +104,9 @@ struct f54_data { enum rmi_f54_report_type report_type; u8 *report_data; + size_t max_report_size; int report_size; + int report_error; bool is_busy; struct mutex status_mutex; @@ -339,6 +341,12 @@ static void rmi_f54_buffer_queue(struct vb2_buffer *vb) mutex_lock(&f54->data_mutex); } + if (f54->report_error) { + dev_err(&f54->fn->dev, "Error acquiring report: %d\n", f54->report_error); + state = VB2_BUF_STATE_ERROR; + goto data_done; + } + ptr = vb2_plane_vaddr(vb, 0); if (!ptr) { dev_err(&f54->fn->dev, "Error acquiring frame ptr\n"); @@ -444,7 +452,12 @@ static int rmi_f54_set_input(struct f54_data *f54, unsigned int i) static int rmi_f54_vidioc_s_input(struct file *file, void *priv, unsigned int i) { - return rmi_f54_set_input(video_drvdata(file), i); + struct f54_data *f54 = video_drvdata(file); + + if (vb2_is_busy(&f54->queue)) + return -EBUSY; + + return rmi_f54_set_input(f54, i); } static int rmi_f54_vidioc_g_input(struct file *file, void *priv, @@ -545,7 +558,14 @@ static void rmi_f54_work(struct work_struct *work) dev_err(&fn->dev, "Bad report size, report type=%d\n", f54->report_type); error = -EINVAL; - goto error; /* retry won't help */ + goto out; /* retry won't help */ + } + + if (report_size > f54->max_report_size) { + dev_err(&fn->dev, "Report size %d exceeds buffer size %zu\n", + report_size, f54->max_report_size); + error = -EINVAL; + goto out; } /* @@ -556,7 +576,7 @@ static void rmi_f54_work(struct work_struct *work) &command); if (error) { dev_err(&fn->dev, "Failed to read back command\n"); - goto error; + goto out; } if (command & F54_GET_REPORT) { if (time_after(jiffies, f54->timeout)) { @@ -564,7 +584,7 @@ static void rmi_f54_work(struct work_struct *work) error = -ETIMEDOUT; } report_size = 0; - goto error; + goto out; } rmi_dbg(RMI_DEBUG_FN, &fn->dev, "Get report command completed, reading data\n"); @@ -579,7 +599,7 @@ static void rmi_f54_work(struct work_struct *work) fifo, sizeof(fifo)); if (error) { dev_err(&fn->dev, "Failed to set fifo start offset\n"); - goto abort; + goto out; } error = rmi_read_block(fn->rmi_dev, fn->fd.data_base_addr + @@ -588,16 +608,17 @@ static void rmi_f54_work(struct work_struct *work) if (error) { dev_err(&fn->dev, "%s: read [%d bytes] returned %d\n", __func__, size, error); - goto abort; + goto out; } } -abort: - f54->report_size = error ? 0 : report_size; -error: +out: if (error) report_size = 0; + f54->report_size = report_size; + f54->report_error = error; + if (report_size == 0 && !error) { queue_delayed_work(f54->workqueue, &f54->work, msecs_to_jiffies(1)); @@ -678,8 +699,8 @@ static int rmi_f54_probe(struct rmi_function *fn) rx = f54->num_rx_electrodes; tx = f54->num_tx_electrodes; - f54->report_data = devm_kzalloc(&fn->dev, - array3_size(tx, rx, sizeof(u16)), + f54->max_report_size = array3_size(tx, rx, sizeof(u16)); + f54->report_data = devm_kzalloc(&fn->dev, f54->max_report_size, GFP_KERNEL); if (f54->report_data == NULL) return -ENOMEM; diff --git a/drivers/input/rmi4/rmi_f55.c b/drivers/input/rmi4/rmi_f55.c index 488adaca4dd0..a0877d32a914 100644 --- a/drivers/input/rmi4/rmi_f55.c +++ b/drivers/input/rmi4/rmi_f55.c @@ -54,10 +54,10 @@ static int rmi_f55_detect(struct rmi_function *fn) f55->num_tx_electrodes = f55->qry[F55_NUM_TX_OFFSET]; f55->cfg_num_rx_electrodes = f55->num_rx_electrodes; - f55->cfg_num_tx_electrodes = f55->num_rx_electrodes; + f55->cfg_num_tx_electrodes = f55->num_tx_electrodes; drv_data->num_rx_electrodes = f55->cfg_num_rx_electrodes; - drv_data->num_tx_electrodes = f55->cfg_num_rx_electrodes; + drv_data->num_tx_electrodes = f55->cfg_num_tx_electrodes; if (f55->qry[F55_PHYS_CHAR_OFFSET] & F55_CAP_SENSOR_ASSIGN) { int i, total; diff --git a/drivers/input/touchscreen/edt-ft5x06.c b/drivers/input/touchscreen/edt-ft5x06.c index d3b1177185a3..d6c3d033b83d 100644 --- a/drivers/input/touchscreen/edt-ft5x06.c +++ b/drivers/input/touchscreen/edt-ft5x06.c @@ -331,6 +331,8 @@ static irqreturn_t edt_ft5x06_ts_isr(int irq, void *dev_id) swap(x, y); id = (buf[2] >> 4) & 0x0f; + if (id >= tsdata->max_support_points) + continue; input_mt_slot(tsdata->input, id); if (input_mt_report_slot_state(tsdata->input, MT_TOOL_FINGER, diff --git a/drivers/input/touchscreen/hynitron_cstxxx.c b/drivers/input/touchscreen/hynitron_cstxxx.c index f6139b1a8681..49fb35c830dc 100644 --- a/drivers/input/touchscreen/hynitron_cstxxx.c +++ b/drivers/input/touchscreen/hynitron_cstxxx.c @@ -312,6 +312,12 @@ static void cst3xx_touch_report(struct i2c_client *client) return; touch_cnt = buf[5] & CST3XX_TOUCH_COUNT_MASK; + if (touch_cnt > ts_data->chip->max_touch_num) { + dev_err(&client->dev, "cst3xx invalid touch count (%d vs %d max)\n", + touch_cnt, ts_data->chip->max_touch_num); + return; + } + /* * Check the check bit of the last touch slot. The check bit is * always present after touch point 1 for valid data, and then @@ -334,9 +340,10 @@ static void cst3xx_touch_report(struct i2c_client *client) finger_id = (buf[idx] >> 4) & 0x0f; /* Sanity check we don't have more fingers than we expect */ - if (ts_data->chip->max_touch_num < finger_id) { - dev_err(&client->dev, "cst3xx touch read failure\n"); - break; + if (finger_id >= ts_data->chip->max_touch_num) { + dev_err(&client->dev, + "cst3xx invalid finger id %d\n", finger_id); + return; } /* sw value of 0 means no touch, 0x03 means touch */ diff --git a/drivers/misc/fastrpc.c b/drivers/misc/fastrpc.c index f3a49384586d..eb6c2a78d3c7 100644 --- a/drivers/misc/fastrpc.c +++ b/drivers/misc/fastrpc.c @@ -492,6 +492,7 @@ static void fastrpc_channel_ctx_free(struct kref *ref) cctx = container_of(ref, struct fastrpc_channel_ctx, refcount); + idr_destroy(&cctx->ctx_idr); kfree(cctx); } @@ -1305,10 +1306,12 @@ static int fastrpc_internal_invoke(struct fastrpc_user *fl, u32 kernel, } if (err == -ERESTARTSYS) { + spin_lock(&fl->lock); list_for_each_entry_safe(buf, b, &fl->mmaps, node) { list_del(&buf->node); list_add_tail(&buf->node, &fl->cctx->invoke_interrupted_mmaps); } + spin_unlock(&fl->lock); } if (err) @@ -1370,7 +1373,9 @@ static int fastrpc_init_create_static_process(struct fastrpc_user *fl, err = PTR_ERR(name); goto err; } - + inbuf.client_id = fl->client_id; + inbuf.namelen = init.namelen; + inbuf.pageslen = 0; if (!fl->cctx->remote_heap) { err = fastrpc_remote_heap_alloc(fl, fl->sctx->dev, init.memlen, &fl->cctx->remote_heap); @@ -1393,12 +1398,10 @@ static int fastrpc_init_create_static_process(struct fastrpc_user *fl, goto err_map; } scm_done = true; + inbuf.pageslen = 1; } } - inbuf.client_id = fl->client_id; - inbuf.namelen = init.namelen; - inbuf.pageslen = 0; fl->pd = USER_PD; args[0].ptr = (u64)(uintptr_t)&inbuf; @@ -1673,7 +1676,7 @@ static int fastrpc_device_open(struct inode *inode, struct file *filp) dev_err(&cctx->rpdev->dev, "No session available\n"); mutex_destroy(&fl->mutex); kfree(fl); - + fastrpc_channel_ctx_put(cctx); return -EBUSY; } @@ -1889,9 +1892,6 @@ static int fastrpc_req_munmap_impl(struct fastrpc_user *fl, struct fastrpc_buf * &args[0]); if (!err) { dev_dbg(dev, "unmmap\tpt 0x%09lx OK\n", buf->raddr); - spin_lock(&fl->lock); - list_del(&buf->node); - spin_unlock(&fl->lock); fastrpc_buf_free(buf); } else { dev_err(dev, "unmmap\tpt 0x%09lx ERROR\n", buf->raddr); @@ -1905,6 +1905,7 @@ static int fastrpc_req_munmap(struct fastrpc_user *fl, char __user *argp) struct fastrpc_buf *buf = NULL, *iter, *b; struct fastrpc_req_munmap req; struct device *dev = fl->sctx->dev; + int err; if (copy_from_user(&req, argp, sizeof(req))) return -EFAULT; @@ -1912,6 +1913,7 @@ static int fastrpc_req_munmap(struct fastrpc_user *fl, char __user *argp) spin_lock(&fl->lock); list_for_each_entry_safe(iter, b, &fl->mmaps, node) { if ((iter->raddr == req.vaddrout) && (iter->size == req.size)) { + list_del(&iter->node); buf = iter; break; } @@ -1924,7 +1926,14 @@ static int fastrpc_req_munmap(struct fastrpc_user *fl, char __user *argp) return -EINVAL; } - return fastrpc_req_munmap_impl(fl, buf); + err = fastrpc_req_munmap_impl(fl, buf); + if (err) { + spin_lock(&fl->lock); + list_add_tail(&buf->node, &fl->mmaps); + spin_unlock(&fl->lock); + } + + return err; } static int fastrpc_req_mmap(struct fastrpc_user *fl, char __user *argp) diff --git a/drivers/misc/mei/client.c b/drivers/misc/mei/client.c index 643b0039cc72..26d2b2742d50 100644 --- a/drivers/misc/mei/client.c +++ b/drivers/misc/mei/client.c @@ -425,18 +425,24 @@ static void mei_io_tx_list_free_cl(struct list_head *head, } /** - * mei_io_list_free_fp - free cb from a list that matches file pointer + * mei_io_rd_list_free_fp - free cb from a rd_completed list that matches file pointer * - * @head: io list + * @cl: host client * @fp: file pointer (matching cb file object), may be NULL */ -static void mei_io_list_free_fp(struct list_head *head, const struct file *fp) +static void mei_io_rd_list_free_fp(struct mei_cl *cl, const struct file *fp) { struct mei_cl_cb *cb, *next; + LIST_HEAD(cmpl_list); - list_for_each_entry_safe(cb, next, head, list) + spin_lock(&cl->rd_completed_lock); + list_for_each_entry_safe(cb, next, &cl->rd_completed, list) if (!fp || fp == cb->fp) - mei_io_cb_free(cb); + list_move(&cb->list, &cmpl_list); + spin_unlock(&cl->rd_completed_lock); + + list_for_each_entry_safe(cb, next, &cmpl_list, list) + mei_io_cb_free(cb); } /** @@ -565,9 +571,7 @@ int mei_cl_flush_queues(struct mei_cl *cl, const struct file *fp) mei_io_list_flush_cl(&cl->dev->ctrl_rd_list, cl); mei_cl_free_pending(cl); } - spin_lock(&cl->rd_completed_lock); - mei_io_list_free_fp(&cl->rd_completed, fp); - spin_unlock(&cl->rd_completed_lock); + mei_io_rd_list_free_fp(cl, fp); return 0; } @@ -1401,7 +1405,7 @@ void mei_cl_add_rd_completed(struct mei_cl *cl, struct mei_cl_cb *cb) } /** - * mei_cl_del_rd_completed - free read completed callback with lock + * mei_cl_del_rd_completed - unlink read completed callback with lock and free it * * @cl: host client * @cb: callback block @@ -1410,8 +1414,9 @@ void mei_cl_add_rd_completed(struct mei_cl *cl, struct mei_cl_cb *cb) void mei_cl_del_rd_completed(struct mei_cl *cl, struct mei_cl_cb *cb) { spin_lock(&cl->rd_completed_lock); - mei_io_cb_free(cb); + list_del_init(&cb->list); spin_unlock(&cl->rd_completed_lock); + mei_io_cb_free(cb); } /** diff --git a/drivers/net/bonding/bond_alb.c b/drivers/net/bonding/bond_alb.c index 2d37b07c8215..839f7482dc18 100644 --- a/drivers/net/bonding/bond_alb.c +++ b/drivers/net/bonding/bond_alb.c @@ -1534,8 +1534,8 @@ void bond_alb_monitor(struct work_struct *work) struct bonding *bond = container_of(work, struct bonding, alb_work.work); struct alb_bond_info *bond_info = &(BOND_ALB_INFO(bond)); + struct slave *slave, *curr; struct list_head *iter; - struct slave *slave; if (!bond_has_slaves(bond)) { atomic_set(&bond_info->tx_rebalance_counter, 0); @@ -1597,9 +1597,11 @@ void bond_alb_monitor(struct work_struct *work) * because a slave was disabled then * it can now leave promiscuous mode. */ - dev_set_promiscuity(rtnl_dereference(bond->curr_active_slave)->dev, - -1); - bond_info->primary_is_promisc = 0; + curr = rtnl_dereference(bond->curr_active_slave); + if (bond_info->primary_is_promisc && curr) { + dev_set_promiscuity(curr->dev, -1); + bond_info->primary_is_promisc = 0; + } rtnl_unlock(); rcu_read_lock(); diff --git a/drivers/net/ethernet/amd/pds_core/core.h b/drivers/net/ethernet/amd/pds_core/core.h index b7fe9ad73349..a1d41329209f 100644 --- a/drivers/net/ethernet/amd/pds_core/core.h +++ b/drivers/net/ethernet/amd/pds_core/core.h @@ -171,6 +171,7 @@ struct pdsc { struct timer_list wdtimer; unsigned int wdtimer_period; struct work_struct health_work; + bool health_stopped; struct devlink_health_reporter *fw_reporter; u32 fw_recoveries; diff --git a/drivers/net/ethernet/amd/pds_core/main.c b/drivers/net/ethernet/amd/pds_core/main.c index 8d94a4d70395..9a2c64198d03 100644 --- a/drivers/net/ethernet/amd/pds_core/main.c +++ b/drivers/net/ethernet/amd/pds_core/main.c @@ -470,8 +470,10 @@ static void pdsc_stop_health_thread(struct pdsc *pdsc) return; timer_shutdown_sync(&pdsc->wdtimer); - if (pdsc->health_work.func) - cancel_work_sync(&pdsc->health_work); + if (pdsc->health_work.func && !pdsc->health_stopped) { + disable_work_sync(&pdsc->health_work); + pdsc->health_stopped = true; + } } static void pdsc_restart_health_thread(struct pdsc *pdsc) @@ -479,6 +481,10 @@ static void pdsc_restart_health_thread(struct pdsc *pdsc) if (pdsc->pdev->is_virtfn) return; + if (pdsc->health_stopped) { + enable_work(&pdsc->health_work); + pdsc->health_stopped = false; + } timer_setup(&pdsc->wdtimer, pdsc_wdtimer_cb, 0); mod_timer(&pdsc->wdtimer, jiffies + 1); } @@ -555,7 +561,11 @@ static pci_ers_result_t pdsc_pci_error_detected(struct pci_dev *pdev, pci_channel_state_t error) { if (error == pci_channel_io_frozen) { + struct pdsc *pdsc = pci_get_drvdata(pdev); + pdsc_reset_prepare(pdev); + if (!pdev->is_virtfn) + cancel_work_sync(&pdsc->pci_reset_work); return PCI_ERS_RESULT_NEED_RESET; } diff --git a/drivers/net/ethernet/aquantia/atlantic/aq_ring.c b/drivers/net/ethernet/aquantia/atlantic/aq_ring.c index 8ff07de2bd52..e1193c6719d9 100644 --- a/drivers/net/ethernet/aquantia/atlantic/aq_ring.c +++ b/drivers/net/ethernet/aquantia/atlantic/aq_ring.c @@ -360,6 +360,35 @@ bool aq_ring_tx_clean(struct aq_ring_s *self) return !!budget; } +void aq_ring_tx_deinit(struct aq_ring_s *self) +{ + if (!self) + return; + + for (; self->sw_head != self->sw_tail; + self->sw_head = aq_ring_next_dx(self, self->sw_head)) { + struct aq_ring_buff_s *buff = &self->buff_ring[self->sw_head]; + struct device *ndev = aq_nic_get_dev(self->aq_nic); + + if (buff->is_mapped) { + if (buff->is_sop) { + dma_unmap_single(ndev, buff->pa, buff->len, + DMA_TO_DEVICE); + } else { + dma_unmap_page(ndev, buff->pa, buff->len, + DMA_TO_DEVICE); + } + } + + if (buff->is_eop) { + if (buff->skb) + dev_kfree_skb_any(buff->skb); + else if (buff->xdpf) + xdp_return_frame(buff->xdpf); + } + } +} + static void aq_rx_checksum(struct aq_ring_s *self, struct aq_ring_buff_s *buff, struct sk_buff *skb) @@ -921,15 +950,29 @@ int aq_ring_rx_fill(struct aq_ring_s *self) void aq_ring_rx_deinit(struct aq_ring_s *self) { - if (!self) + unsigned int i; + + if (!self || !self->buff_ring) return; - for (; self->sw_head != self->sw_tail; - self->sw_head = aq_ring_next_dx(self, self->sw_head)) { - struct aq_ring_buff_s *buff = &self->buff_ring[self->sw_head]; + /* Release every page still owned by the ring. + * + * Walking [sw_head, sw_tail) is not enough: refill is batched + * (aq_ring_rx_fill() waits for AQ_CFG_RX_REFILL_THRES free slots), + * so slots that were cleaned but not yet reposted accumulate in the + * [sw_tail, sw_head) gap, and they keep their page for reuse. Walk + * the whole ring and release whatever is left. + */ + for (i = 0; i < self->size; i++) { + struct aq_ring_buff_s *buff = &self->buff_ring[i]; + + if (!buff->rxdata.page) + continue; aq_free_rxpage(&buff->rxdata, aq_nic_get_dev(self->aq_nic)); } + + self->sw_head = self->sw_tail; } void aq_ring_free(struct aq_ring_s *self) diff --git a/drivers/net/ethernet/aquantia/atlantic/aq_ring.h b/drivers/net/ethernet/aquantia/atlantic/aq_ring.h index a70b880ada67..6431cc62962f 100644 --- a/drivers/net/ethernet/aquantia/atlantic/aq_ring.h +++ b/drivers/net/ethernet/aquantia/atlantic/aq_ring.h @@ -202,6 +202,7 @@ void aq_ring_update_queue_state(struct aq_ring_s *ring); void aq_ring_queue_wake(struct aq_ring_s *ring); void aq_ring_queue_stop(struct aq_ring_s *ring); bool aq_ring_tx_clean(struct aq_ring_s *self); +void aq_ring_tx_deinit(struct aq_ring_s *self); int aq_xdp_xmit(struct net_device *dev, int num_frames, struct xdp_frame **frames, u32 flags); int aq_ring_rx_clean(struct aq_ring_s *self, diff --git a/drivers/net/ethernet/aquantia/atlantic/aq_vec.c b/drivers/net/ethernet/aquantia/atlantic/aq_vec.c index 2f9033ceed8c..05814fea0f5f 100644 --- a/drivers/net/ethernet/aquantia/atlantic/aq_vec.c +++ b/drivers/net/ethernet/aquantia/atlantic/aq_vec.c @@ -275,7 +275,7 @@ void aq_vec_deinit(struct aq_vec_s *self) for (i = 0U; self->tx_rings > i; ++i) { ring = self->ring[i]; - aq_ring_tx_clean(&ring[AQ_VEC_TX_ID]); + aq_ring_tx_deinit(&ring[AQ_VEC_TX_ID]); aq_ring_rx_deinit(&ring[AQ_VEC_RX_ID]); } diff --git a/drivers/net/ethernet/broadcom/bnge/bnge_auxr.c b/drivers/net/ethernet/broadcom/bnge/bnge_auxr.c index 67e93e17d4d9..0955b488b6fe 100644 --- a/drivers/net/ethernet/broadcom/bnge/bnge_auxr.c +++ b/drivers/net/ethernet/broadcom/bnge/bnge_auxr.c @@ -141,12 +141,15 @@ static void bnge_aux_dev_release(struct device *dev) { struct bnge_auxr_priv *aux_priv = container_of(dev, struct bnge_auxr_priv, aux_dev.dev); - struct bnge_dev *bd = pci_get_drvdata(aux_priv->auxr_dev->pdev); + struct bnge_auxr_dev *auxr_dev = aux_priv->auxr_dev; + struct bnge_dev *bd = pci_get_drvdata(to_pci_dev(dev->parent)); ida_free(&bnge_aux_dev_ids, aux_priv->id); - kfree(aux_priv->auxr_dev->auxr_info); + if (auxr_dev) { + kfree(auxr_dev->auxr_info); + kfree(auxr_dev); + } bd->auxr_dev = NULL; - kfree(aux_priv->auxr_dev); kfree(aux_priv); bd->aux_priv = NULL; } diff --git a/drivers/net/ethernet/broadcom/bnge/bnge_netdev.c b/drivers/net/ethernet/broadcom/bnge/bnge_netdev.c index 6f7ef506d4e1..ac4c93e5b634 100644 --- a/drivers/net/ethernet/broadcom/bnge/bnge_netdev.c +++ b/drivers/net/ethernet/broadcom/bnge/bnge_netdev.c @@ -2768,8 +2768,6 @@ static int bnge_init_nic(struct bnge_net *bn) err_free_ring_grps: bnge_free_ring_grps(bn); - return rc; - err_free_rx_ring_pair_bufs: bnge_free_rx_ring_pair_bufs(bn); return rc; diff --git a/drivers/net/ethernet/broadcom/bnge/bnge_resc.c b/drivers/net/ethernet/broadcom/bnge/bnge_resc.c index 0e94f092813e..4711dd4945ff 100644 --- a/drivers/net/ethernet/broadcom/bnge/bnge_resc.c +++ b/drivers/net/ethernet/broadcom/bnge/bnge_resc.c @@ -163,7 +163,8 @@ static int bnge_adjust_rings(struct bnge_dev *bd, u16 *rx, u16 tx_chunks = bnge_num_tx_to_cp(bd, *tx); if (tx_chunks != *tx) { - u16 tx_saved = tx_chunks, rc; + u16 tx_saved = tx_chunks; + int rc; rc = bnge_fix_rings_count(rx, &tx_chunks, max_nq, sh); if (rc) diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.c b/drivers/net/ethernet/broadcom/bnxt/bnxt.c index 7513618793da..25099077fe4f 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.c @@ -4611,11 +4611,14 @@ static void bnxt_init_one_rx_agg_ring_rxbd(struct bnxt *bp, type = ((u32)rxr->rx_page_size << RX_BD_LEN_SHIFT) | RX_BD_TYPE_RX_AGG_BD; - /* On P7, setting EOP will cause the chip to disable - * Relaxed Ordering (RO) for TPA data. Disable EOP for - * potentially higher performance with RO. + /* Disable EOP if TPA is enabled to prevent overlapping zero + * padding with the next segment's data. On P7_PLUS, EOP will + * automatically disable Relaxed Ordering (RO) to prevent + * potential data corruption (and may degrade performance). On + * older chips, RO will not be automatically disabled and may + * cause corruption. */ - if (BNXT_CHIP_P5_AND_MINUS(bp) || !(bp->flags & BNXT_FLAG_TPA)) + if (!(bp->flags & BNXT_FLAG_TPA)) type |= RX_BD_FLAGS_AGG_EOP; bnxt_init_rxbd_pages(ring, type); @@ -6704,22 +6707,36 @@ int bnxt_get_nr_rss_ctxs(struct bnxt *bp, int rx_rings) static void bnxt_fill_hw_rss_tbl(struct bnxt *bp, struct bnxt_vnic_info *vnic) { bool no_rss = !(vnic->flags & BNXT_VNIC_RSS_FLAG); - u16 i, j; + u16 i, j, min_j = bp->rx_nr_rings - 1; + + if (!vnic->rss_table) + goto skip_rss_tbl; /* Fill the RSS indirection table with ring group ids */ for (i = 0, j = 0; i < HW_HASH_INDEX_SIZE; i++) { if (!no_rss) j = bp->rss_indir_tbl[i]; + min_j = min(j, min_j); vnic->rss_table[i] = cpu_to_le16(vnic->fw_grp_ids[j]); } + +skip_rss_tbl: + if (vnic->rss_table && !no_rss) + vnic->default_rx_ring = min_j; + else if (vnic->flags & BNXT_VNIC_RFS_FLAG) + vnic->default_rx_ring = vnic->vnic_id - 1; + else if ((vnic->vnic_id == 1) && BNXT_CHIP_TYPE_NITRO_A0(bp)) + vnic->default_rx_ring = bp->rx_nr_rings - 1; + else + vnic->default_rx_ring = 0; } static void bnxt_fill_hw_rss_tbl_p5(struct bnxt *bp, struct bnxt_vnic_info *vnic) { + u16 tbl_size, i, min_j = bp->rx_nr_rings - 1; __le16 *ring_tbl = vnic->rss_table; struct bnxt_rx_ring_info *rxr; - u16 tbl_size, i; tbl_size = bnxt_get_rxfh_indir_size(bp->dev); @@ -6732,6 +6749,7 @@ static void bnxt_fill_hw_rss_tbl_p5(struct bnxt *bp, j = ethtool_rxfh_context_indir(vnic->rss_ctx)[i]; else j = bp->rss_indir_tbl[i]; + min_j = min(j, min_j); rxr = &bp->rx_ring[j]; ring_id = rxr->rx_ring_struct.fw_ring_id; @@ -6739,19 +6757,15 @@ static void bnxt_fill_hw_rss_tbl_p5(struct bnxt *bp, ring_id = bnxt_cp_ring_for_rx(bp, rxr); *ring_tbl++ = cpu_to_le16(ring_id); } + vnic->default_rx_ring = min_j; } static void __bnxt_hwrm_vnic_set_rss(struct bnxt *bp, struct hwrm_vnic_rss_cfg_input *req, struct bnxt_vnic_info *vnic) { - if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS) { - bnxt_fill_hw_rss_tbl_p5(bp, vnic); - if (bp->flags & BNXT_FLAG_CHIP_P7) - req->flags |= VNIC_RSS_CFG_REQ_FLAGS_IPSEC_HASH_TYPE_CFG_SUPPORT; - } else { - bnxt_fill_hw_rss_tbl(bp, vnic); - } + if (bp->flags & BNXT_FLAG_CHIP_P7) + req->flags |= VNIC_RSS_CFG_REQ_FLAGS_IPSEC_HASH_TYPE_CFG_SUPPORT; if (bp->rss_hash_delta) { req->hash_type = cpu_to_le32(bp->rss_hash_delta); @@ -6803,6 +6817,7 @@ static int bnxt_hwrm_vnic_set_rss_p5(struct bnxt *bp, if (!set_rss) return hwrm_req_send(bp, req); + bnxt_fill_hw_rss_tbl_p5(bp, vnic); __bnxt_hwrm_vnic_set_rss(bp, req, vnic); ring_tbl_map = vnic->rss_table_dma_addr; nr_ctxs = bnxt_get_nr_rss_ctxs(bp, bp->rx_nr_rings); @@ -6939,8 +6954,9 @@ int bnxt_hwrm_vnic_cfg(struct bnxt *bp, struct bnxt_vnic_info *vnic) return rc; if (bp->flags & BNXT_FLAG_CHIP_P5_PLUS) { - struct bnxt_rx_ring_info *rxr = &bp->rx_ring[0]; + struct bnxt_rx_ring_info *rxr; + rxr = &bp->rx_ring[vnic->default_rx_ring]; req->default_rx_ring_id = cpu_to_le16(rxr->rx_ring_struct.fw_ring_id); req->default_cmpl_ring_id = @@ -6973,13 +6989,7 @@ int bnxt_hwrm_vnic_cfg(struct bnxt *bp, struct bnxt_vnic_info *vnic) req->cos_rule = cpu_to_le16(0xffff); } - if (vnic->flags & BNXT_VNIC_RSS_FLAG) - ring = 0; - else if (vnic->flags & BNXT_VNIC_RFS_FLAG) - ring = vnic->vnic_id - 1; - else if ((vnic->vnic_id == 1) && BNXT_CHIP_TYPE_NITRO_A0(bp)) - ring = bp->rx_nr_rings - 1; - + ring = vnic->default_rx_ring; grp_idx = bp->rx_ring[ring].bnapi->index; req->dflt_ring_grp = cpu_to_le16(bp->grp_info[grp_idx].fw_grp_id); req->lb_rule = cpu_to_le16(0xffff); @@ -10866,6 +10876,7 @@ static int __bnxt_setup_vnic(struct bnxt *bp, struct bnxt_vnic_info *vnic) } skip_rss_ctx: + bnxt_fill_hw_rss_tbl(bp, vnic); /* configure default vnic, ring grp */ rc = bnxt_hwrm_vnic_cfg(bp, vnic); if (rc) { @@ -11090,6 +11101,11 @@ static int bnxt_set_vnic_mru_p5(struct bnxt *bp, struct bnxt_vnic_info *vnic, vnic->vnic_id, rc); return rc; } + if (rxr_id == vnic->default_rx_ring) { + rc = bnxt_hwrm_vnic_cfg(bp, vnic); + if (rc) + return rc; + } } vnic->mru = mru; bnxt_hwrm_vnic_update(bp, vnic, @@ -11171,6 +11187,9 @@ static int bnxt_setup_nitroa0_vnic(struct bnxt *bp) return rc; } + /* Setup the proper default RX ring */ + bnxt_fill_hw_rss_tbl(bp, vnic); + rc = bnxt_hwrm_vnic_cfg(bp, vnic); if (rc) { netdev_err(bp->dev, "Cannot allocate special vnic for NS2 A0: %x\n", @@ -16217,6 +16236,7 @@ static int bnxt_queue_mem_alloc(struct net_device *dev, clone->rx_next_cons = 0; clone->need_head_pool = false; clone->rx_page_size = qcfg->rx_page_size; + clone->rx_agg_bmap = NULL; rc = bnxt_alloc_rx_page_pool(bp, clone, rxr->page_pool->p.nid); if (rc) @@ -16269,6 +16289,8 @@ static int bnxt_queue_mem_alloc(struct net_device *dev, bnxt_free_one_tpa_info(bp, clone); err_free_rx_agg_ring: bnxt_free_ring(bp, &clone->rx_agg_ring_struct.ring_mem); + kfree(clone->rx_agg_bmap); + clone->rx_agg_bmap = NULL; err_free_rx_ring: bnxt_free_ring(bp, &clone->rx_ring_struct.ring_mem); err_rxq_info_unreg: diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt.h b/drivers/net/ethernet/broadcom/bnxt/bnxt.h index 6335dfc14c98..44ea8fc7b766 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt.h +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt.h @@ -1334,6 +1334,7 @@ struct bnxt_vnic_info { #define BNXT_VNIC_RSSCTX_FLAG 0x40 struct ethtool_rxfh_context *rss_ctx; u32 vnic_id; + u16 default_rx_ring; }; struct bnxt_rss_ctx { diff --git a/drivers/net/ethernet/broadcom/bnxt/bnxt_ptp.c b/drivers/net/ethernet/broadcom/bnxt/bnxt_ptp.c index 5d41dc1bc782..b89293dfd2d1 100644 --- a/drivers/net/ethernet/broadcom/bnxt/bnxt_ptp.c +++ b/drivers/net/ethernet/broadcom/bnxt/bnxt_ptp.c @@ -495,12 +495,15 @@ static int bnxt_ptp_enable(struct ptp_clock_info *ptp_info, return rc; case PTP_CLK_REQ_PPS: /* Configure PHC PPS IN */ - rc = bnxt_ptp_cfg_pin(bp, 0, BNXT_PPS_PIN_PPS_IN); + pin_id = 0; + if (!on) + break; + rc = bnxt_ptp_cfg_pin(bp, pin_id, BNXT_PPS_PIN_PPS_IN); if (rc) return rc; rc = bnxt_ptp_cfg_event(bp, BNXT_PPS_EVENT_INTERNAL); if (!rc) - ptp->pps_info.pins[0].event = BNXT_PPS_EVENT_INTERNAL; + ptp->pps_info.pins[pin_id].event = BNXT_PPS_EVENT_INTERNAL; return rc; default: netdev_err(ptp->bp->dev, "Unrecognized PIN function\n"); diff --git a/drivers/net/ethernet/cisco/enic/enic_main.c b/drivers/net/ethernet/cisco/enic/enic_main.c index e7125b818087..d98f7e7ccab9 100644 --- a/drivers/net/ethernet/cisco/enic/enic_main.c +++ b/drivers/net/ethernet/cisco/enic/enic_main.c @@ -3011,8 +3011,9 @@ static void enic_remove(struct pci_dev *pdev) if (netdev) { struct enic *enic = netdev_priv(netdev); - cancel_work_sync(&enic->reset); - cancel_work_sync(&enic->change_mtu_work); + disable_work_sync(&enic->reset); + disable_work_sync(&enic->tx_hang_reset); + disable_work_sync(&enic->change_mtu_work); unregister_netdev(netdev); enic_dev_deinit(enic); vnic_dev_close(enic->vdev); diff --git a/drivers/net/ethernet/hisilicon/hix5hd2_gmac.c b/drivers/net/ethernet/hisilicon/hix5hd2_gmac.c index e3e7f2270560..02282dc86faf 100644 --- a/drivers/net/ethernet/hisilicon/hix5hd2_gmac.c +++ b/drivers/net/ethernet/hisilicon/hix5hd2_gmac.c @@ -1282,7 +1282,6 @@ static void hix5hd2_dev_remove(struct platform_device *pdev) struct net_device *ndev = platform_get_drvdata(pdev); struct hix5hd2_priv *priv = netdev_priv(ndev); - netif_napi_del(&priv->napi); unregister_netdev(ndev); mdiobus_unregister(priv->bus); mdiobus_free(priv->bus); diff --git a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.c b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.c index fc8587c80813..164c3ecf195c 100644 --- a/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.c +++ b/drivers/net/ethernet/hisilicon/hns3/hns3pf/hclge_main.c @@ -9498,12 +9498,8 @@ static int hclge_init_ae_dev(struct hnae3_ae_dev *ae_dev) if (ret) goto err_ptp_uninit; - if (hdev->hw.mac.media_type != HNAE3_MEDIA_TYPE_COPPER) { + if (hdev->hw.mac.media_type != HNAE3_MEDIA_TYPE_COPPER) hdev->hw.mac.req_autoneg = hdev->hw.mac.autoneg; - if (hdev->hw.mac.autoneg == AUTONEG_DISABLE && - hdev->hw.mac.speed != SPEED_UNKNOWN) - hdev->hw.mac.req_speed = hdev->hw.mac.speed; - } ret = hclge_set_autoneg_speed_dup(hdev); if (ret) { diff --git a/drivers/net/ethernet/intel/igc/igc_main.c b/drivers/net/ethernet/intel/igc/igc_main.c index b3883a5a7d7a..39043d8ca1bf 100644 --- a/drivers/net/ethernet/intel/igc/igc_main.c +++ b/drivers/net/ethernet/intel/igc/igc_main.c @@ -3082,7 +3082,7 @@ static void igc_xdp_xmit_zc(struct igc_ring *ring) meta_req.tx_buffer = bi; meta_req.meta = meta; meta_req.used_desc = 0; - xsk_tx_metadata_request(meta, &igc_xsk_tx_metadata_ops, + xsk_tx_metadata_request(pool, &meta, &igc_xsk_tx_metadata_ops, &meta_req); /* xsk_tx_metadata_request() may have updated next_to_use */ @@ -7585,11 +7585,13 @@ static int __igc_resume(struct device *dev, bool rpm) err = __igc_open(netdev, true); if (!rpm) rtnl_unlock(); - if (!err) - netif_device_attach(netdev); + if (err) + return err; } - return err; + netif_device_attach(netdev); + + return 0; } static int igc_resume(struct device *dev) diff --git a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_tc.c b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_tc.c index 0b46ec29e64e..039fd47ebf52 100644 --- a/drivers/net/ethernet/marvell/octeontx2/nic/otx2_tc.c +++ b/drivers/net/ethernet/marvell/octeontx2/nic/otx2_tc.c @@ -54,10 +54,12 @@ static void otx2_get_egress_burst_cfg(struct otx2_nic *nic, u32 burst, if (burst) { *burst_exp = ilog2(burst) ? ilog2(burst) - 1 : 0; tmp = burst - rounddown_pow_of_two(burst); - if (burst < max_mantissa) + if (burst <= max_mantissa) { *burst_mantissa = tmp * 2; - else + } else { + WARN_ON(*burst_exp < 7); *burst_mantissa = tmp / (1ULL << (*burst_exp - 7)); + } } else { *burst_exp = MAX_BURST_EXPONENT; *burst_mantissa = max_mantissa; diff --git a/drivers/net/ethernet/marvell/prestera/prestera_pci.c b/drivers/net/ethernet/marvell/prestera/prestera_pci.c index 2989a77e3b42..1ad0e62a8433 100644 --- a/drivers/net/ethernet/marvell/prestera/prestera_pci.c +++ b/drivers/net/ethernet/marvell/prestera/prestera_pci.c @@ -684,6 +684,9 @@ static int prestera_fw_hdr_parse(struct prestera_fw *fw) struct prestera_fw_header *hdr; u32 magic; + if (fw->bin->size < sizeof(*hdr)) + return -EINVAL; + hdr = (struct prestera_fw_header *)fw->bin->data; magic = be32_to_cpu(hdr->magic_number); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.c b/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.c index adcc73e2a5b3..1493d8106c1a 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/diag/fw_tracer.c @@ -1025,13 +1025,11 @@ struct mlx5_fw_tracer *mlx5_fw_tracer_create(struct mlx5_core_dev *dev) tracer = kvzalloc_obj(*tracer); if (!tracer) - return ERR_PTR(-ENOMEM); + return NULL; tracer->work_queue = create_singlethread_workqueue("mlx5_fw_tracer"); - if (!tracer->work_queue) { - err = -ENOMEM; + if (!tracer->work_queue) goto free_tracer; - } tracer->dev = dev; @@ -1073,7 +1071,7 @@ struct mlx5_fw_tracer *mlx5_fw_tracer_create(struct mlx5_core_dev *dev) destroy_workqueue(tracer->work_queue); free_tracer: kvfree(tracer); - return ERR_PTR(err); + return NULL; } static int fw_tracer_event(struct notifier_block *nb, unsigned long action, void *data); @@ -1084,7 +1082,7 @@ int mlx5_fw_tracer_init(struct mlx5_fw_tracer *tracer) struct mlx5_core_dev *dev; int err; - if (IS_ERR_OR_NULL(tracer)) + if (!tracer) return 0; if (!tracer->str_db.loaded) @@ -1134,7 +1132,7 @@ int mlx5_fw_tracer_init(struct mlx5_fw_tracer *tracer) /* Stop tracer + Cleanup HW resources */ void mlx5_fw_tracer_cleanup(struct mlx5_fw_tracer *tracer) { - if (IS_ERR_OR_NULL(tracer)) + if (!tracer) return; mutex_lock(&tracer->state_lock); @@ -1163,7 +1161,7 @@ void mlx5_fw_tracer_cleanup(struct mlx5_fw_tracer *tracer) /* Free software resources (Buffers, etc ..) */ void mlx5_fw_tracer_destroy(struct mlx5_fw_tracer *tracer) { - if (IS_ERR_OR_NULL(tracer)) + if (!tracer) return; mlx5_core_dbg(tracer->dev, "FWTracer: Destroy\n"); @@ -1215,7 +1213,7 @@ int mlx5_fw_tracer_reload(struct mlx5_fw_tracer *tracer) struct mlx5_core_dev *dev; int err; - if (IS_ERR_OR_NULL(tracer)) + if (!tracer) return 0; dev = tracer->dev; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en.h b/drivers/net/ethernet/mellanox/mlx5/core/en.h index 6867a5aed42c..822859c360e7 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en.h @@ -483,7 +483,7 @@ typedef int (*mlx5e_fp_xmit_xdp_frame_check)(struct mlx5e_xdpsq *); typedef bool (*mlx5e_fp_xmit_xdp_frame)(struct mlx5e_xdpsq *, struct mlx5e_xmit_data *, int, - struct xsk_tx_metadata *); + struct xsk_tx_metadata **); struct mlx5e_xdpsq { /* data path */ diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/tc_priv.h b/drivers/net/ethernet/mellanox/mlx5/core/en/tc_priv.h index 28cab4bf525c..7bfe7cdc5770 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/tc_priv.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/tc_priv.h @@ -30,6 +30,7 @@ enum { MLX5E_TC_FLOW_FLAG_FAILED = MLX5E_TC_FLOW_BASE + 9, MLX5E_TC_FLOW_FLAG_SAMPLE = MLX5E_TC_FLOW_BASE + 10, MLX5E_TC_FLOW_FLAG_USE_ACT_STATS = MLX5E_TC_FLOW_BASE + 11, + MLX5E_TC_FLOW_FLAG_PEER = MLX5E_TC_FLOW_BASE + 12, }; struct mlx5e_tc_flow_parse_attr { diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c index d8c7cb8837d7..77ea51bfbaae 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.c @@ -452,11 +452,11 @@ INDIRECT_CALLABLE_SCOPE int mlx5e_xmit_xdp_frame_check_mpwqe(struct mlx5e_xdpsq INDIRECT_CALLABLE_SCOPE bool mlx5e_xmit_xdp_frame(struct mlx5e_xdpsq *sq, struct mlx5e_xmit_data *xdptxd, - int check_result, struct xsk_tx_metadata *meta); + int check_result, struct xsk_tx_metadata **meta); INDIRECT_CALLABLE_SCOPE bool mlx5e_xmit_xdp_frame_mpwqe(struct mlx5e_xdpsq *sq, struct mlx5e_xmit_data *xdptxd, - int check_result, struct xsk_tx_metadata *meta) + int check_result, struct xsk_tx_metadata **meta) { struct mlx5e_tx_mpwqe *session = &sq->mpwqe; struct mlx5e_xdpsq_stats *stats = sq->stats; @@ -504,7 +504,10 @@ mlx5e_xmit_xdp_frame_mpwqe(struct mlx5e_xdpsq *sq, struct mlx5e_xmit_data *xdptx * and it's safe to complete it at any time. */ mlx5e_xdp_mpwqe_session_start(sq); - xsk_tx_metadata_request(meta, &mlx5e_xsk_tx_metadata_ops, &session->wqe->eth); + if (meta) + xsk_tx_metadata_request(sq->xsk_pool, meta, + &mlx5e_xsk_tx_metadata_ops, + &session->wqe->eth); } mlx5e_xdp_mpwqe_add_dseg(sq, p, stats); @@ -535,7 +538,7 @@ INDIRECT_CALLABLE_SCOPE int mlx5e_xmit_xdp_frame_check(struct mlx5e_xdpsq *sq) INDIRECT_CALLABLE_SCOPE bool mlx5e_xmit_xdp_frame(struct mlx5e_xdpsq *sq, struct mlx5e_xmit_data *xdptxd, - int check_result, struct xsk_tx_metadata *meta) + int check_result, struct xsk_tx_metadata **meta) { struct mlx5e_xmit_data_frags *xdptxdf = container_of(xdptxd, struct mlx5e_xmit_data_frags, xd); @@ -649,7 +652,9 @@ mlx5e_xmit_xdp_frame(struct mlx5e_xdpsq *sq, struct mlx5e_xmit_data *xdptxd, sq->pc += num_wqebbs; - xsk_tx_metadata_request(meta, &mlx5e_xsk_tx_metadata_ops, eseg); + if (meta) + xsk_tx_metadata_request(sq->xsk_pool, meta, + &mlx5e_xsk_tx_metadata_ops, eseg); sq->doorbell_cseg = cseg; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.h b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.h index 3c54f8962664..5b8f4094d553 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xdp.h @@ -114,11 +114,11 @@ extern const struct xsk_tx_metadata_ops mlx5e_xsk_tx_metadata_ops; INDIRECT_CALLABLE_DECLARE(bool mlx5e_xmit_xdp_frame_mpwqe(struct mlx5e_xdpsq *sq, struct mlx5e_xmit_data *xdptxd, int check_result, - struct xsk_tx_metadata *meta)); + struct xsk_tx_metadata **meta)); INDIRECT_CALLABLE_DECLARE(bool mlx5e_xmit_xdp_frame(struct mlx5e_xdpsq *sq, struct mlx5e_xmit_data *xdptxd, int check_result, - struct xsk_tx_metadata *meta)); + struct xsk_tx_metadata **meta)); INDIRECT_CALLABLE_DECLARE(int mlx5e_xmit_xdp_frame_check_mpwqe(struct mlx5e_xdpsq *sq)); INDIRECT_CALLABLE_DECLARE(int mlx5e_xmit_xdp_frame_check(struct mlx5e_xdpsq *sq)); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/tx.c b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/tx.c index 8aeab4b21035..3d19dad8f868 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/tx.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en/xsk/tx.c @@ -105,7 +105,7 @@ bool mlx5e_xsk_tx(struct mlx5e_xdpsq *sq, unsigned int budget) ret = INDIRECT_CALL_2(sq->xmit_xdp_frame, mlx5e_xmit_xdp_frame_mpwqe, mlx5e_xmit_xdp_frame, sq, &xdptxd, - check_result, meta); + check_result, &meta); if (unlikely(!ret)) { if (sq->mpwqe.wqe) mlx5e_xdp_mpwqe_complete(sq); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c index 7d47a1da8b6b..f0407a850ea8 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_main.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_main.c @@ -1939,8 +1939,10 @@ int mlx5e_open_txqsq(struct mlx5e_channel *c, u32 tisn, int txq_ix, void mlx5e_activate_txqsq(struct mlx5e_txqsq *sq) { sq->txq = netdev_get_tx_queue(sq->netdev, sq->txq_ix); + /* Reset BQL only when the SQ has no bytes in flight. */ + if (sq->cc == sq->pc) + netdev_tx_reset_queue(sq->txq); set_bit(MLX5E_SQ_STATE_ENABLED, &sq->state); - netdev_tx_reset_queue(sq->txq); netif_tx_start_queue(sq->txq); netif_queue_set_napi(sq->netdev, sq->txq_ix, NETDEV_QUEUE_TYPE_TX, sq->cq.napi); } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c b/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c index 1bc7b9019124..b290beb4369a 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/en_tc.c @@ -2161,7 +2161,8 @@ static void mlx5e_tc_del_flow(struct mlx5e_priv *priv, if (mlx5e_is_eswitch_flow(flow)) { struct mlx5_devcom_comp_dev *devcom = flow->priv->mdev->priv.eswitch->devcom; - if (!mlx5_devcom_for_each_peer_begin(devcom)) { + if (flow_flag_test(flow, PEER) || + !mlx5_devcom_for_each_peer_begin(devcom)) { mlx5e_tc_del_fdb_flow(priv, flow); return; } @@ -4628,6 +4629,7 @@ static int mlx5e_tc_add_fdb_peer_flow(struct flow_cls_offload *f, else in_mdev = priv->mdev; + flow_flags |= BIT(MLX5E_TC_FLOW_FLAG_PEER); parse_attr = flow->attr->parse_attr; peer_flow = __mlx5e_add_fdb_flow(peer_priv, f, flow_flags, parse_attr->filter_dev, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c index 907ee83a722d..247e5d85ec36 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/eswitch_offloads.c @@ -3986,7 +3986,7 @@ static void esw_offloads_steering_cleanup(struct mlx5_eswitch *esw) mutex_destroy(&esw->fdb_table.offloads.vports.lock); } -static void esw_vfs_changed_event_handler(struct mlx5_eswitch *esw) +static void esw_changed_event_handler(struct mlx5_eswitch *esw) { struct mlx5_esw_pf_info host_pf_info; u16 new_num_vfs; @@ -3999,6 +3999,11 @@ static void esw_vfs_changed_event_handler(struct mlx5_eswitch *esw) host_pf_info = mlx5_esw_get_host_pf_info(esw->dev, out); new_num_vfs = host_pf_info.num_of_vfs; + if (host_pf_info.pf_disabled) { + mlx5_sf_table_esw_changed_event_handler(esw->dev); + mlx5_sf_hw_table_esw_changed_event_handler(esw->dev); + } + if (new_num_vfs == esw->esw_funcs.num_vfs || host_pf_info.pf_disabled) goto free; @@ -4091,8 +4096,7 @@ int mlx5_esw_funcs_changed_handler(struct notifier_block *nb, esw_funcs = mlx5_nb_cof(nb, struct mlx5_esw_functions, nb); esw = container_of(esw_funcs, struct mlx5_eswitch, esw_funcs); - ret = mlx5_esw_add_work(esw, esw_vfs_changed_event_handler, - GFP_ATOMIC); + ret = mlx5_esw_add_work(esw, esw_changed_event_handler, GFP_ATOMIC); if (ret) return NOTIFY_DONE; diff --git a/drivers/net/ethernet/mellanox/mlx5/core/sf/devlink.c b/drivers/net/ethernet/mellanox/mlx5/core/sf/devlink.c index b6cecbcc392d..4564d460416b 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/sf/devlink.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/sf/devlink.c @@ -561,3 +561,32 @@ bool mlx5_sf_table_empty(const struct mlx5_core_dev *dev) return xa_empty(&table->function_ids); } + +void mlx5_sf_table_esw_changed_event_handler(struct mlx5_core_dev *dev) +{ + struct mlx5_sf_table *table = dev->priv.sf_table; + unsigned long index; + struct mlx5_sf *sf; + + trace_mlx5_sf_host_pf_disabled(dev); + + if (!table) + return; + + mutex_lock(&table->sf_state_lock); + xa_for_each(&table->function_ids, index, sf) { + if (!sf->controller) + continue; + + if (sf->hw_state == MLX5_VHCA_STATE_IN_USE) + sf->hw_state = MLX5_VHCA_STATE_ACTIVE; + else if (sf->hw_state == MLX5_VHCA_STATE_TEARDOWN_REQUEST) + sf->hw_state = MLX5_VHCA_STATE_ALLOCATED; + else + continue; + trace_mlx5_sf_update_state(table->dev, sf->port_index, + sf->controller, sf->hw_fn_id, + sf->hw_state); + } + mutex_unlock(&table->sf_state_lock); +} diff --git a/drivers/net/ethernet/mellanox/mlx5/core/sf/diag/sf_tracepoint.h b/drivers/net/ethernet/mellanox/mlx5/core/sf/diag/sf_tracepoint.h index 302ce00da5a9..9dc88a7e30ca 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/sf/diag/sf_tracepoint.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/sf/diag/sf_tracepoint.h @@ -11,6 +11,14 @@ #include #include "sf/vhca_event.h" +TRACE_EVENT(mlx5_sf_host_pf_disabled, + TP_PROTO(const struct mlx5_core_dev *dev), + TP_ARGS(dev), + TP_STRUCT__entry(__string(devname, dev_name(dev->device))), + TP_fast_assign(__assign_str(devname);), + TP_printk("(%s)\n", __get_str(devname)) +); + TRACE_EVENT(mlx5_sf_add, TP_PROTO(const struct mlx5_core_dev *dev, unsigned int port_index, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/sf/hw_table.c b/drivers/net/ethernet/mellanox/mlx5/core/sf/hw_table.c index 0bc9146a3598..95a8b1e64ba4 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/sf/hw_table.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/sf/hw_table.c @@ -459,3 +459,25 @@ bool mlx5_sf_hw_table_supported(const struct mlx5_core_dev *dev) { return !!dev->priv.sf_hw_table; } + +void mlx5_sf_hw_table_esw_changed_event_handler(struct mlx5_core_dev *dev) +{ + struct mlx5_sf_hw_table *table; + struct mlx5_sf_hwc_table *hwc; + int i; + + table = dev->priv.sf_hw_table; + if (!table) + return; + + mutex_lock(&table->table_lock); + hwc = &table->hwc[MLX5_SF_HWC_EXT_HOST]; + for (i = 0; i < hwc->max_fn; i++) { + struct mlx5_sf_hw *sf_hw; + + sf_hw = &hwc->sfs[i]; + if (sf_hw->allocated && sf_hw->pending_delete) + mlx5_sf_hw_table_hwc_sf_free(dev, hwc, i); + } + mutex_unlock(&table->table_lock); +} diff --git a/drivers/net/ethernet/mellanox/mlx5/core/sf/sf.h b/drivers/net/ethernet/mellanox/mlx5/core/sf/sf.h index d8a934a0e968..ed784682b9b1 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/sf/sf.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/sf/sf.h @@ -15,12 +15,14 @@ void mlx5_sf_hw_table_cleanup(struct mlx5_core_dev *dev); int mlx5_sf_hw_notifier_init(struct mlx5_core_dev *dev); void mlx5_sf_hw_notifier_cleanup(struct mlx5_core_dev *dev); void mlx5_sf_hw_table_destroy(struct mlx5_core_dev *dev); +void mlx5_sf_hw_table_esw_changed_event_handler(struct mlx5_core_dev *dev); int mlx5_sf_notifiers_init(struct mlx5_core_dev *dev); int mlx5_sf_table_init(struct mlx5_core_dev *dev); void mlx5_sf_notifiers_cleanup(struct mlx5_core_dev *dev); void mlx5_sf_table_cleanup(struct mlx5_core_dev *dev); bool mlx5_sf_table_empty(const struct mlx5_core_dev *dev); +void mlx5_sf_table_esw_changed_event_handler(struct mlx5_core_dev *dev); int mlx5_devlink_sf_port_new(struct devlink *devlink, const struct devlink_port_new_attrs *add_attr, @@ -60,6 +62,11 @@ static inline void mlx5_sf_hw_table_destroy(struct mlx5_core_dev *dev) { } +static inline void +mlx5_sf_hw_table_esw_changed_event_handler(struct mlx5_core_dev *dev) +{ +} + static inline int mlx5_sf_notifiers_init(struct mlx5_core_dev *dev) { return 0; @@ -83,6 +90,11 @@ static inline bool mlx5_sf_table_empty(const struct mlx5_core_dev *dev) return true; } +static inline void +mlx5_sf_table_esw_changed_event_handler(struct mlx5_core_dev *dev) +{ +} + #endif #endif diff --git a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c index 3801f9d45278..a71f0df26378 100644 --- a/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c +++ b/drivers/net/ethernet/stmicro/stmmac/stmmac_main.c @@ -2748,8 +2748,8 @@ static bool stmmac_xdp_xmit_zc(struct stmmac_priv *priv, u32 queue, u32 budget) meta_req.set_ic = &set_ic; meta_req.tbs = tx_q->tbs; meta_req.edesc = &tx_q->dma_entx[entry]; - xsk_tx_metadata_request(meta, &stmmac_xsk_tx_metadata_ops, - &meta_req); + xsk_tx_metadata_request(pool, &meta, + &stmmac_xsk_tx_metadata_ops, &meta_req); if (set_ic) { tx_q->tx_count_frames = 0; stmmac_set_tx_ic(priv, tx_desc); @@ -4134,6 +4134,15 @@ static int __stmmac_open(struct net_device *dev, dma_conf->tx_queue[i].tbs = priv->dma_conf.tx_queue[i].tbs; memcpy(&priv->dma_conf, dma_conf, sizeof(*dma_conf)); + /* The PHY is suspended when the interface is reopened without + * disconnecting the PHY, e.g. on MTU change. IEEE 802.3 allows PHYs + * to stop their receive clock while powered down, but the DMA + * software reset in stmmac_hw_setup() requires a running receive + * clock, and phylink_start() below resumes the PHY only after the + * hardware setup. Resume a suspended PHY here first. + */ + phylink_prepare_resume(priv->phylink); + stmmac_reset_queues_param(priv); ret = stmmac_hw_setup(dev); diff --git a/drivers/net/ovpn/main.c b/drivers/net/ovpn/main.c index 9993c1dfe471..168cfe9b59a9 100644 --- a/drivers/net/ovpn/main.c +++ b/drivers/net/ovpn/main.c @@ -35,25 +35,11 @@ static void ovpn_priv_free(struct net_device *net) static int ovpn_mp_alloc(struct ovpn_priv *ovpn) { - struct in_device *dev_v4; int i; if (ovpn->mode != OVPN_MODE_MP) return 0; - dev_v4 = __in_dev_get_rtnl(ovpn->dev); - if (dev_v4) { - /* disable redirects as Linux gets confused by ovpn - * handling same-LAN routing. - * This happens because a multipeer interface is used as - * relay point between hosts in the same subnet, while - * in a classic LAN this would not be needed because the - * two hosts would be able to talk directly. - */ - IN_DEV_CONF_SET(dev_v4, SEND_REDIRECTS, false); - IPV4_DEVCONF_ALL(dev_net(ovpn->dev), SEND_REDIRECTS) = false; - } - /* the peer container is fairly large, therefore we allocate it only in * MP mode */ @@ -97,9 +83,38 @@ static void ovpn_net_uninit(struct net_device *dev) gro_cells_destroy(&ovpn->gro_cells); } +static int ovpn_net_open(struct net_device *dev) +{ + struct ovpn_priv *ovpn = netdev_priv(dev); + struct in_device *dev_v4; + + /* the IPv4 in_device (and thus its config) is recreated whenever the + * interface is moved to a new netns, so redirects must be disabled on + * every bring-up rather than once at creation time, otherwise the + * setting is silently lost after such a move + */ + if (ovpn->mode == OVPN_MODE_MP) { + dev_v4 = __in_dev_get_rtnl(dev); + if (dev_v4) { + /* disable redirects as Linux gets confused by ovpn + * handling same-LAN routing. + * This happens because a multipeer interface is used as + * relay point between hosts in the same subnet, while + * in a classic LAN this would not be needed because the + * two hosts would be able to talk directly. + */ + IN_DEV_CONF_SET(dev_v4, SEND_REDIRECTS, false); + IPV4_DEVCONF_ALL(dev_net(dev), SEND_REDIRECTS) = false; + } + } + + return 0; +} + static const struct net_device_ops ovpn_netdev_ops = { .ndo_init = ovpn_net_init, .ndo_uninit = ovpn_net_uninit, + .ndo_open = ovpn_net_open, .ndo_start_xmit = ovpn_net_xmit, }; @@ -183,6 +198,7 @@ static int ovpn_newlink(struct net_device *dev, struct ovpn_priv *ovpn = netdev_priv(dev); struct nlattr **data = params->data; enum ovpn_mode mode = OVPN_MODE_P2P; + int ret; if (data && data[IFLA_OVPN_MODE]) { mode = nla_get_u8(data[IFLA_OVPN_MODE]); @@ -207,7 +223,17 @@ static int ovpn_newlink(struct net_device *dev, else netif_carrier_off(dev); - return register_netdevice(dev); + ret = register_netdevice(dev); + if (ret < 0) + return ret; + + return 0; +} + +static size_t ovpn_get_size(const struct net_device *dev) +{ + /* IFLA_OVPN_MODE */ + return nla_total_size(sizeof(u8)); } static int ovpn_fill_info(struct sk_buff *skb, const struct net_device *dev) @@ -228,13 +254,17 @@ static struct rtnl_link_ops ovpn_link_ops = { .policy = ovpn_policy, .maxtype = IFLA_OVPN_MAX, .newlink = ovpn_newlink, + .get_size = ovpn_get_size, .fill_info = ovpn_fill_info, }; static int __init ovpn_init(void) { - int err = rtnl_link_register(&ovpn_link_ops); + int err; + ovpn_tcp_init(); + + err = rtnl_link_register(&ovpn_link_ops); if (err) { pr_err("ovpn: can't register rtnl link ops: %d\n", err); return err; @@ -246,8 +276,6 @@ static int __init ovpn_init(void) goto unreg_rtnl; } - ovpn_tcp_init(); - return 0; unreg_rtnl: diff --git a/drivers/net/ovpn/netlink-gen.c b/drivers/net/ovpn/netlink-gen.c index 2147cec7c2c5..92d2fdc17c2e 100644 --- a/drivers/net/ovpn/netlink-gen.c +++ b/drivers/net/ovpn/netlink-gen.c @@ -16,6 +16,14 @@ static const struct netlink_range_validation ovpn_a_peer_id_range = { .max = 16777215ULL, }; +static const struct netlink_range_validation ovpn_a_peer_keepalive_interval_range = { + .max = 86400ULL, +}; + +static const struct netlink_range_validation ovpn_a_peer_keepalive_timeout_range = { + .max = 86400ULL, +}; + static const struct netlink_range_validation ovpn_a_peer_tx_id_range = { .max = 16777215ULL, }; @@ -68,8 +76,8 @@ const struct nla_policy ovpn_peer_nl_policy[OVPN_A_PEER_TX_ID + 1] = { [OVPN_A_PEER_LOCAL_IPV4] = { .type = NLA_BE32, }, [OVPN_A_PEER_LOCAL_IPV6] = NLA_POLICY_EXACT_LEN(16), [OVPN_A_PEER_LOCAL_PORT] = NLA_POLICY_MIN(NLA_BE16, 1), - [OVPN_A_PEER_KEEPALIVE_INTERVAL] = { .type = NLA_U32, }, - [OVPN_A_PEER_KEEPALIVE_TIMEOUT] = { .type = NLA_U32, }, + [OVPN_A_PEER_KEEPALIVE_INTERVAL] = NLA_POLICY_FULL_RANGE(NLA_U32, &ovpn_a_peer_keepalive_interval_range), + [OVPN_A_PEER_KEEPALIVE_TIMEOUT] = NLA_POLICY_FULL_RANGE(NLA_U32, &ovpn_a_peer_keepalive_timeout_range), [OVPN_A_PEER_DEL_REASON] = NLA_POLICY_MAX(NLA_U32, 4), [OVPN_A_PEER_VPN_RX_BYTES] = { .type = NLA_UINT, }, [OVPN_A_PEER_VPN_TX_BYTES] = { .type = NLA_UINT, }, @@ -97,8 +105,8 @@ const struct nla_policy ovpn_peer_new_input_nl_policy[OVPN_A_PEER_TX_ID + 1] = { [OVPN_A_PEER_VPN_IPV6] = NLA_POLICY_EXACT_LEN(16), [OVPN_A_PEER_LOCAL_IPV4] = { .type = NLA_BE32, }, [OVPN_A_PEER_LOCAL_IPV6] = NLA_POLICY_EXACT_LEN(16), - [OVPN_A_PEER_KEEPALIVE_INTERVAL] = { .type = NLA_U32, }, - [OVPN_A_PEER_KEEPALIVE_TIMEOUT] = { .type = NLA_U32, }, + [OVPN_A_PEER_KEEPALIVE_INTERVAL] = NLA_POLICY_FULL_RANGE(NLA_U32, &ovpn_a_peer_keepalive_interval_range), + [OVPN_A_PEER_KEEPALIVE_TIMEOUT] = NLA_POLICY_FULL_RANGE(NLA_U32, &ovpn_a_peer_keepalive_timeout_range), [OVPN_A_PEER_TX_ID] = NLA_POLICY_FULL_RANGE(NLA_U32, &ovpn_a_peer_tx_id_range), }; @@ -112,8 +120,8 @@ const struct nla_policy ovpn_peer_set_input_nl_policy[OVPN_A_PEER_TX_ID + 1] = { [OVPN_A_PEER_VPN_IPV6] = NLA_POLICY_EXACT_LEN(16), [OVPN_A_PEER_LOCAL_IPV4] = { .type = NLA_BE32, }, [OVPN_A_PEER_LOCAL_IPV6] = NLA_POLICY_EXACT_LEN(16), - [OVPN_A_PEER_KEEPALIVE_INTERVAL] = { .type = NLA_U32, }, - [OVPN_A_PEER_KEEPALIVE_TIMEOUT] = { .type = NLA_U32, }, + [OVPN_A_PEER_KEEPALIVE_INTERVAL] = NLA_POLICY_FULL_RANGE(NLA_U32, &ovpn_a_peer_keepalive_interval_range), + [OVPN_A_PEER_KEEPALIVE_TIMEOUT] = NLA_POLICY_FULL_RANGE(NLA_U32, &ovpn_a_peer_keepalive_timeout_range), [OVPN_A_PEER_TX_ID] = NLA_POLICY_FULL_RANGE(NLA_U32, &ovpn_a_peer_tx_id_range), }; diff --git a/drivers/net/ovpn/netlink.c b/drivers/net/ovpn/netlink.c index 4c66c1ec497e..4dad85294198 100644 --- a/drivers/net/ovpn/netlink.c +++ b/drivers/net/ovpn/netlink.c @@ -534,6 +534,12 @@ int ovpn_nl_peer_set_doit(struct sk_buff *skb, struct genl_info *info) */ if (ret > 0) ovpn_peer_hash_vpn_ip(peer); + /* if the remote endpoint was updated, the by_transp_addr hash bucket + * also needs to be refreshed, otherwise incoming packets from the new + * remote address would fail the lockless lookup + */ + if (attrs[OVPN_A_PEER_REMOTE_IPV4] || attrs[OVPN_A_PEER_REMOTE_IPV6]) + ovpn_peer_hash_transp_addr(peer); spin_unlock_bh(&ovpn->lock); ovpn_peer_put(peer); diff --git a/drivers/net/ovpn/peer.c b/drivers/net/ovpn/peer.c index a21d02ac715e..b0519f9840d8 100644 --- a/drivers/net/ovpn/peer.c +++ b/drivers/net/ovpn/peer.c @@ -189,6 +189,9 @@ int ovpn_peer_reset_sockaddr(struct ovpn_peer *peer, &(*__tbl1)[ovpn_get_hash_slot(*__tbl1, _key, _key_len)];\ }) +static void __ovpn_peer_hash_transp_addr(struct ovpn_peer *peer, + const struct ovpn_bind *bind); + /** * ovpn_peer_endpoints_update - update remote or local endpoint for peer * @peer: peer to update the remote endpoint for @@ -196,7 +199,6 @@ int ovpn_peer_reset_sockaddr(struct ovpn_peer *peer, */ void ovpn_peer_endpoints_update(struct ovpn_peer *peer, struct sk_buff *skb) { - struct hlist_nulls_head *nhead; struct sockaddr_storage ss; struct sockaddr_in6 *sa6; bool reset_cache = false; @@ -220,9 +222,16 @@ void ovpn_peer_endpoints_update(struct ovpn_peer *peer, struct sk_buff *skb) */ local_ip = &ip_hdr(skb)->daddr; sa = (struct sockaddr_in *)&ss; - sa->sin_family = AF_INET; - sa->sin_addr.s_addr = ip_hdr(skb)->saddr; - sa->sin_port = udp_hdr(skb)->source; + /* use a designated initializer so the sin_zero padding + * is zeroed (it ends up in the by_transp_addr hash key) + * without memset-ing the whole sockaddr_storage on the + * RX fast path + */ + *sa = (struct sockaddr_in) { + .sin_family = AF_INET, + .sin_addr.s_addr = ip_hdr(skb)->saddr, + .sin_port = udp_hdr(skb)->source, + }; salen = sizeof(*sa); reset_cache = true; break; @@ -248,11 +257,19 @@ void ovpn_peer_endpoints_update(struct ovpn_peer *peer, struct sk_buff *skb) */ local_ip = &ipv6_hdr(skb)->daddr; sa6 = (struct sockaddr_in6 *)&ss; - sa6->sin6_family = AF_INET6; - sa6->sin6_addr = ipv6_hdr(skb)->saddr; - sa6->sin6_port = udp_hdr(skb)->source; - sa6->sin6_scope_id = ipv6_iface_scope_id(&ipv6_hdr(skb)->saddr, - skb->skb_iif); + /* use a designated initializer so the sin6_flowinfo + * padding is zeroed (it ends up in the by_transp_addr + * hash key) without memset-ing the whole + * sockaddr_storage on the RX fast path + */ + *sa6 = (struct sockaddr_in6) { + .sin6_family = AF_INET6, + .sin6_addr = ipv6_hdr(skb)->saddr, + .sin6_port = udp_hdr(skb)->source, + .sin6_scope_id = + ipv6_iface_scope_id(&ipv6_hdr(skb)->saddr, + skb->skb_iif), + }; salen = sizeof(*sa6); reset_cache = true; break; @@ -295,42 +312,25 @@ void ovpn_peer_endpoints_update(struct ovpn_peer *peer, struct sk_buff *skb) ovpn_nl_peer_float_notify(peer, &ss); /* rehashing is required only in MP mode as P2P has one peer - * only and thus there is no hashtable + * only and thus there is no hashtable. + * + * This function may be invoked concurrently, so re-read peer->bind + * under the proper locks and rehash against its current value. */ - if (peer->ovpn->mode == OVPN_MODE_MP) { - spin_lock_bh(&peer->ovpn->lock); - spin_lock_bh(&peer->lock); - bind = rcu_dereference_protected(peer->bind, - lockdep_is_held(&peer->lock)); - if (unlikely(!bind)) { - spin_unlock_bh(&peer->lock); - spin_unlock_bh(&peer->ovpn->lock); - return; - } + if (peer->ovpn->mode != OVPN_MODE_MP) + return; - /* This function may be invoked concurrently, therefore another - * float may have happened in parallel: perform rehashing - * using the peer->bind->remote directly as key - */ - - switch (bind->remote.in4.sin_family) { - case AF_INET: - salen = sizeof(*sa); - break; - case AF_INET6: - salen = sizeof(*sa6); - break; - } - - /* remove old hashing */ - hlist_nulls_del_init_rcu(&peer->hash_entry_transp_addr); - /* re-add with new transport address */ - nhead = ovpn_get_hash_head(peer->ovpn->peers->by_transp_addr, - &bind->remote, salen); - hlist_nulls_add_head_rcu(&peer->hash_entry_transp_addr, nhead); - spin_unlock_bh(&peer->lock); - spin_unlock_bh(&peer->ovpn->lock); - } + /* This function may be invoked concurrently, therefore another + * float may have happened in parallel: re-acquire the locks and + * rehash using the peer->bind->remote directly as key + */ + spin_lock_bh(&peer->ovpn->lock); + spin_lock_bh(&peer->lock); + bind = rcu_dereference_protected(peer->bind, + lockdep_is_held(&peer->lock)); + __ovpn_peer_hash_transp_addr(peer, bind); + spin_unlock_bh(&peer->lock); + spin_unlock_bh(&peer->ovpn->lock); return; unlock: spin_unlock_bh(&peer->lock); @@ -896,6 +896,83 @@ bool ovpn_peer_check_by_src(struct ovpn_priv *ovpn, struct sk_buff *skb, return match; } +/* Move @peer to the by_transp_addr bucket matching its current bind. + * + * Caller must hold both peer->ovpn->lock and peer->lock, and must have + * already dereferenced a valid (non-NULL) peer->bind, passed in as @bind. + */ +static void __ovpn_peer_hash_transp_addr(struct ovpn_peer *peer, + const struct ovpn_bind *bind) +{ + struct sockaddr_storage sa = {}; + struct hlist_nulls_head *nhead; + struct sockaddr_in6 *sa6; + struct sockaddr_in *sa4; + size_t salen; + + lockdep_assert_held(&peer->ovpn->lock); + lockdep_assert_held(&peer->lock); + + if (WARN_ON_ONCE(!bind)) + return; + + /* peer may have been concurrently removed between the caller's + * initial lookup and our acquisition of ovpn->lock; skip the + * rehash so we don't re-insert a removed peer + */ + if (unlikely(hlist_unhashed(&peer->hash_entry_id))) + return; + + /* Build the hash key from the transport identity only + * (family/address/port), matching ovpn_peer_add_mp() and the lookup + * in ovpn_peer_get_by_transp_addr(). Hashing bind->remote directly + * would fold in sin6_scope_id (set on the float path but never by the + * lookup), scattering the peer into a bucket lookups cannot reach. + */ + switch (bind->remote.in4.sin_family) { + case AF_INET: + sa4 = (struct sockaddr_in *)&sa; + sa4->sin_family = AF_INET; + sa4->sin_addr.s_addr = bind->remote.in4.sin_addr.s_addr; + sa4->sin_port = bind->remote.in4.sin_port; + salen = sizeof(*sa4); + break; + case AF_INET6: + sa6 = (struct sockaddr_in6 *)&sa; + sa6->sin6_family = AF_INET6; + sa6->sin6_addr = bind->remote.in6.sin6_addr; + sa6->sin6_port = bind->remote.in6.sin6_port; + salen = sizeof(*sa6); + break; + default: + return; + } + + /* remove old hashing (no-op if entry is not currently linked) */ + hlist_nulls_del_init_rcu(&peer->hash_entry_transp_addr); + /* re-add with current transport address */ + nhead = ovpn_get_hash_head(peer->ovpn->peers->by_transp_addr, &sa, + salen); + hlist_nulls_add_head_rcu(&peer->hash_entry_transp_addr, nhead); +} + +void ovpn_peer_hash_transp_addr(struct ovpn_peer *peer) +{ + struct ovpn_bind *bind; + + lockdep_assert_held(&peer->ovpn->lock); + + /* rehashing makes sense only in multipeer mode */ + if (peer->ovpn->mode != OVPN_MODE_MP) + return; + + spin_lock_bh(&peer->lock); + bind = rcu_dereference_protected(peer->bind, + lockdep_is_held(&peer->lock)); + __ovpn_peer_hash_transp_addr(peer, bind); + spin_unlock_bh(&peer->lock); +} + void ovpn_peer_hash_vpn_ip(struct ovpn_peer *peer) { struct hlist_nulls_head *nhead; @@ -906,6 +983,13 @@ void ovpn_peer_hash_vpn_ip(struct ovpn_peer *peer) if (peer->ovpn->mode != OVPN_MODE_MP) return; + /* peer may have been concurrently removed between the caller's + * initial lookup and our acquisition of ovpn->lock; skip the + * rehash so we don't re-insert a removed peer + */ + if (hlist_unhashed(&peer->hash_entry_id)) + return; + if (peer->vpn_addrs.ipv4.s_addr != htonl(INADDR_ANY)) { /* remove potential old hashing */ hlist_nulls_del_init_rcu(&peer->hash_entry_addr4); @@ -1165,7 +1249,7 @@ static void ovpn_peer_release_p2p(struct ovpn_priv *ovpn, struct sock *sk, } if (sk) { - ovpn_sock = rcu_access_pointer(peer->sock); + ovpn_sock = rcu_dereference_bh(peer->sock); if (!ovpn_sock || ovpn_sock->sk != sk) { spin_unlock_bh(&ovpn->lock); return; diff --git a/drivers/net/ovpn/peer.h b/drivers/net/ovpn/peer.h index 86c8cffada6d..dfa5c0037e02 100644 --- a/drivers/net/ovpn/peer.h +++ b/drivers/net/ovpn/peer.h @@ -150,6 +150,7 @@ struct ovpn_peer *ovpn_peer_get_by_id(struct ovpn_priv *ovpn, u32 peer_id); struct ovpn_peer *ovpn_peer_get_by_dst(struct ovpn_priv *ovpn, struct sk_buff *skb); void ovpn_peer_hash_vpn_ip(struct ovpn_peer *peer); +void ovpn_peer_hash_transp_addr(struct ovpn_peer *peer); bool ovpn_peer_check_by_src(struct ovpn_priv *ovpn, struct sk_buff *skb, struct ovpn_peer *peer); diff --git a/drivers/net/ovpn/socket.c b/drivers/net/ovpn/socket.c index 517caa64a4fe..6cbeb2caaeec 100644 --- a/drivers/net/ovpn/socket.c +++ b/drivers/net/ovpn/socket.c @@ -162,6 +162,15 @@ struct ovpn_socket *ovpn_socket_new(struct socket *sock, struct ovpn_peer *peer) rcu_read_lock(); ovpn_sock = rcu_dereference_sk_user_data(sk); if (ovpn_sock) { + /* something else filled the sk_user_data without + * setting the encap_type. Reject the socket. + */ + if (!type) { + ovpn_sock = ERR_PTR(-EBUSY); + rcu_read_unlock(); + goto sock_release; + } + /* socket owned by another ovpn instance, we can't use it */ if (ovpn_sock->ovpn != peer->ovpn) { ovpn_sock = ERR_PTR(-EBUSY); diff --git a/drivers/net/phy/mediatek/mtk.h b/drivers/net/phy/mediatek/mtk.h index 320f76ffa81f..79e09727b81b 100644 --- a/drivers/net/phy/mediatek/mtk.h +++ b/drivers/net/phy/mediatek/mtk.h @@ -53,15 +53,15 @@ #define MTK_GPHY_LED_RX_BLINK_SET (MTK_PHY_LED_BLINK_1000RX | \ MTK_PHY_LED_BLINK_100RX | \ MTK_PHY_LED_BLINK_10RX) -#define MTK_GPHY_LED_TX_BLINK_SET (MTK_PHY_LED_BLINK_1000RX | \ - MTK_PHY_LED_BLINK_100RX | \ - MTK_PHY_LED_BLINK_10RX) +#define MTK_GPHY_LED_TX_BLINK_SET (MTK_PHY_LED_BLINK_1000TX | \ + MTK_PHY_LED_BLINK_100TX | \ + MTK_PHY_LED_BLINK_10TX) #define MTK_2P5GPHY_LED_ON_SET (MTK_PHY_LED_ON_LINK2500 | \ MTK_GPHY_LED_ON_SET) #define MTK_2P5GPHY_LED_RX_BLINK_SET (MTK_PHY_LED_BLINK_2500RX | \ MTK_GPHY_LED_RX_BLINK_SET) -#define MTK_2P5GPHY_LED_TX_BLINK_SET (MTK_PHY_LED_BLINK_2500RX | \ +#define MTK_2P5GPHY_LED_TX_BLINK_SET (MTK_PHY_LED_BLINK_2500TX | \ MTK_GPHY_LED_TX_BLINK_SET) #define MTK_PHY_LED_STATE_FORCE_ON 0 diff --git a/drivers/net/tap.c b/drivers/net/tap.c index fae115915c8e..5d2d34d24ce8 100644 --- a/drivers/net/tap.c +++ b/drivers/net/tap.c @@ -1074,10 +1074,21 @@ static int tap_get_user_xdp(struct tap_queue *q, struct xdp_buff *xdp) skb_reset_mac_header(skb); skb->protocol = eth_hdr(skb)->h_proto; + rcu_read_lock(); + tap = rcu_dereference(q->tap); + if (!tap) { + kfree_skb(skb); + rcu_read_unlock(); + return 0; + } + skb->dev = tap->dev; + if (vnet_hdr_len) { err = tun_vnet_hdr_to_skb(q->flags, skb, gso); - if (err) + if (err) { + rcu_read_unlock(); goto err_kfree; + } } /* Move network header to the right position for VLAN tagged packets */ @@ -1085,15 +1096,8 @@ static int tap_get_user_xdp(struct tap_queue *q, struct xdp_buff *xdp) vlan_get_protocol_and_depth(skb, skb->protocol, &depth) != 0) skb_set_network_header(skb, depth); - rcu_read_lock(); - tap = rcu_dereference(q->tap); - if (tap) { - skb->dev = tap->dev; - skb_probe_transport_header(skb); - dev_queue_xmit(skb); - } else { - kfree_skb(skb); - } + skb_probe_transport_header(skb); + dev_queue_xmit(skb); rcu_read_unlock(); return 0; diff --git a/drivers/net/thunderbolt/main.c b/drivers/net/thunderbolt/main.c index 02a91650561a..98893732bc6e 100644 --- a/drivers/net/thunderbolt/main.c +++ b/drivers/net/thunderbolt/main.c @@ -386,11 +386,16 @@ static void tbnet_tear_down(struct tbnet *net, bool send_logout) break; } - tb_ring_stop(net->rx_ring.ring); - tb_ring_stop(net->tx_ring.ring); - tbnet_free_buffers(&net->rx_ring); - tbnet_free_buffers(&net->tx_ring); - + /* Tear the paths down before stopping the rings. This mirrors + * tbnet_connected_work(), which enables the paths last so the + * Rx ring is primed before packets can arrive. Stopping a + * ring zeroes its descriptor base and tbnet_free_buffers() + * unmaps and frees the frame buffers, leaving anything still + * in flight with nowhere to drain to; + * __tb_path_deactivate_hop() then waits for the hop's + * 'pending' bit, which on some host routers never clears in + * that state. + */ ret = tb_xdomain_disable_paths(net->xd, net->local_transmit_path, net->tx_ring.ring->hop, @@ -399,6 +404,11 @@ static void tbnet_tear_down(struct tbnet *net, bool send_logout) if (ret) netdev_warn(net->dev, "failed to disable DMA paths\n"); + tb_ring_stop(net->rx_ring.ring); + tb_ring_stop(net->tx_ring.ring); + tbnet_free_buffers(&net->rx_ring); + tbnet_free_buffers(&net->tx_ring); + tb_xdomain_release_in_hopid(net->xd, net->remote_transmit_path); net->remote_transmit_path = 0; } @@ -925,12 +935,8 @@ static int tbnet_open(struct net_device *dev) netif_carrier_off(dev); - flags = RING_FLAG_FRAME; - /* Only enable full E2E if the other end supports it too */ - if (tbnet_e2e && net->svc->prtcstns & TBNET_E2E) - flags |= RING_FLAG_E2E; - - ring = tb_ring_alloc_tx(xd->tb->nhi, -1, TBNET_RING_SIZE, flags); + ring = tb_ring_alloc_tx(xd->tb->nhi, -1, TBNET_RING_SIZE, + RING_FLAG_FRAME); if (!ring) { netdev_err(dev, "failed to allocate Tx ring\n"); return -ENOMEM; @@ -949,6 +955,11 @@ static int tbnet_open(struct net_device *dev) sof_mask = BIT(TBIP_PDF_FRAME_START); eof_mask = BIT(TBIP_PDF_FRAME_END); + flags = RING_FLAG_FRAME; + /* Only enable full E2E if the other end supports it too */ + if (tbnet_e2e && net->svc->prtcstns & TBNET_E2E) + flags |= RING_FLAG_E2E; + ring = tb_ring_alloc_rx(xd->tb->nhi, -1, TBNET_RING_SIZE, flags, net->tx_ring.ring->hop, sof_mask, eof_mask, tbnet_start_poll, net); diff --git a/drivers/net/usb/ax88179_178a.c b/drivers/net/usb/ax88179_178a.c index 98f899ea2e94..81d8412ce8e2 100644 --- a/drivers/net/usb/ax88179_178a.c +++ b/drivers/net/usb/ax88179_178a.c @@ -1487,8 +1487,10 @@ ax88179_tx_fixup(struct usbnet *dev, struct sk_buff *skb, gfp_t flags) headroom = skb_headroom(skb) - 8; - if ((dev->net->features & NETIF_F_SG) && skb_linearize(skb)) + if ((dev->net->features & NETIF_F_SG) && skb_linearize(skb)) { + dev_kfree_skb_any(skb); return NULL; + } if ((skb_header_cloned(skb) || headroom < 0) && pskb_expand_head(skb, headroom < 0 ? 8 : 0, 0, GFP_ATOMIC)) { diff --git a/drivers/net/usb/ipheth.c b/drivers/net/usb/ipheth.c index bb1364f85bd1..2b490114d232 100644 --- a/drivers/net/usb/ipheth.c +++ b/drivers/net/usb/ipheth.c @@ -490,6 +490,7 @@ static int ipheth_open(struct net_device *net) if (retval) return retval; + enable_delayed_work(&dev->carrier_work); schedule_delayed_work(&dev->carrier_work, IPHETH_CARRIER_CHECK_TIMEOUT); return retval; } @@ -499,7 +500,11 @@ static int ipheth_close(struct net_device *net) struct ipheth_device *dev = netdev_priv(net); netif_stop_queue(net); - cancel_delayed_work_sync(&dev->carrier_work); + /* A TX URB can still complete with an error after this point and + * try to re-arm the carrier work. Disable it instead of cancelling + * it, so that such a schedule_delayed_work() is a no-op. + */ + disable_delayed_work_sync(&dev->carrier_work); return 0; } @@ -629,6 +634,10 @@ static int ipheth_probe(struct usb_interface *intf, } INIT_DELAYED_WORK(&dev->carrier_work, ipheth_carrier_check_work); + /* Armed only between ipheth_open() and ipheth_close(). Start out + * disabled so the enable/disable counts balance from the first open. + */ + disable_delayed_work(&dev->carrier_work); retval = ipheth_alloc_urbs(dev); if (retval) { diff --git a/drivers/net/usb/usbnet.c b/drivers/net/usb/usbnet.c index 25518635b7b7..a19ecf718f36 100644 --- a/drivers/net/usb/usbnet.c +++ b/drivers/net/usb/usbnet.c @@ -1794,7 +1794,7 @@ usbnet_probe(struct usb_interface *udev, const struct usb_device_id *prod) */ dev->hard_mtu = net->mtu + net->hard_header_len; net->min_mtu = 0; - net->max_mtu = ETH_MAX_MTU; + net->max_mtu = net->mtu; net->netdev_ops = &usbnet_netdev_ops; net->watchdog_timeo = TX_TIMEOUT_JIFFIES; @@ -1804,6 +1804,7 @@ usbnet_probe(struct usb_interface *udev, const struct usb_device_id *prod) // allow device-specific bind/init procedures // NOTE net->name still not usable ... if (info->bind) { + net->max_mtu = ETH_MAX_MTU; status = info->bind(dev, udev); if (status < 0) goto out1; diff --git a/drivers/nvmem/apple-spmi-nvmem.c b/drivers/nvmem/apple-spmi-nvmem.c index 88614005d5ce..7acb0c07d6ab 100644 --- a/drivers/nvmem/apple-spmi-nvmem.c +++ b/drivers/nvmem/apple-spmi-nvmem.c @@ -18,6 +18,22 @@ static const struct regmap_config apple_spmi_regmap_config = { .max_register = 0xffff, }; +static int apple_spmi_nvmem_read(void *priv, unsigned int offset, void *val, + size_t bytes) +{ + struct regmap *map = priv; + + return regmap_bulk_read(map, offset, val, bytes); +} + +static int apple_spmi_nvmem_write(void *priv, unsigned int offset, void *val, + size_t bytes) +{ + struct regmap *map = priv; + + return regmap_bulk_write(map, offset, val, bytes); +} + static int apple_spmi_nvmem_probe(struct spmi_device *sdev) { struct regmap *regmap; @@ -28,8 +44,8 @@ static int apple_spmi_nvmem_probe(struct spmi_device *sdev) .word_size = 1, .stride = 1, .size = 0xffff, - .reg_read = (void *)regmap_bulk_read, - .reg_write = (void *)regmap_bulk_write, + .reg_read = apple_spmi_nvmem_read, + .reg_write = apple_spmi_nvmem_write, }; regmap = devm_regmap_init_spmi_ext(sdev, &apple_spmi_regmap_config); diff --git a/drivers/nvmem/core.c b/drivers/nvmem/core.c index e871181751f3..2f971b059704 100644 --- a/drivers/nvmem/core.c +++ b/drivers/nvmem/core.c @@ -786,7 +786,7 @@ static int nvmem_validate_keepouts(struct nvmem_device *nvmem) return 0; } -static int nvmem_add_cells_from_dt(struct nvmem_device *nvmem, struct device_node *np) +int nvmem_add_cells_from_dt(struct nvmem_device *nvmem, struct device_node *np) { struct device *dev = &nvmem->dev; const __be32 *addr; @@ -834,29 +834,13 @@ static int nvmem_add_cells_from_dt(struct nvmem_device *nvmem, struct device_nod return 0; } +EXPORT_SYMBOL_GPL(nvmem_add_cells_from_dt); static int nvmem_add_cells_from_legacy_of(struct nvmem_device *nvmem) { return nvmem_add_cells_from_dt(nvmem, nvmem->dev.of_node); } -static int nvmem_add_cells_from_fixed_layout(struct nvmem_device *nvmem) -{ - struct device_node *layout_np; - int err = 0; - - layout_np = of_nvmem_layout_get_container(nvmem); - if (!layout_np) - return 0; - - if (of_device_is_compatible(layout_np, "fixed-layout")) - err = nvmem_add_cells_from_dt(nvmem, layout_np); - - of_node_put(layout_np); - - return err; -} - int nvmem_layout_register(struct nvmem_layout *layout) { int ret; @@ -1005,10 +989,6 @@ struct nvmem_device *nvmem_register(const struct nvmem_config *config) goto err_remove_cells; } - rval = nvmem_add_cells_from_fixed_layout(nvmem); - if (rval) - goto err_remove_cells; - dev_dbg(&nvmem->dev, "Registering nvmem device %s\n", config->name); rval = device_add(&nvmem->dev); diff --git a/drivers/nvmem/internals.h b/drivers/nvmem/internals.h index 18fed57270e5..f6d452b6a28f 100644 --- a/drivers/nvmem/internals.h +++ b/drivers/nvmem/internals.h @@ -35,6 +35,8 @@ struct nvmem_device { bool sysfs_cells_populated; }; +int nvmem_add_cells_from_dt(struct nvmem_device *nvmem, struct device_node *np); + #if IS_ENABLED(CONFIG_OF) int nvmem_layout_bus_register(void); void nvmem_layout_bus_unregister(void); diff --git a/drivers/nvmem/layouts.c b/drivers/nvmem/layouts.c index b90584e1b99e..07a34be9669c 100644 --- a/drivers/nvmem/layouts.c +++ b/drivers/nvmem/layouts.c @@ -125,11 +125,6 @@ static int nvmem_layout_create_device(struct nvmem_device *nvmem, return 0; } -static const struct of_device_id of_nvmem_layout_skip_table[] = { - { .compatible = "fixed-layout", }, - {} -}; - static int nvmem_layout_bus_populate(struct nvmem_device *nvmem, struct device_node *layout_dn) { @@ -142,12 +137,6 @@ static int nvmem_layout_bus_populate(struct nvmem_device *nvmem, return 0; } - /* Fixed layouts are parsed manually somewhere else for now */ - if (of_match_node(of_nvmem_layout_skip_table, layout_dn)) { - pr_debug("%s() - skipping %pOF node\n", __func__, layout_dn); - return 0; - } - if (of_node_check_flag(layout_dn, OF_POPULATED_BUS)) { pr_debug("%s() - skipping %pOF, already populated\n", __func__, layout_dn); diff --git a/drivers/nvmem/layouts/Makefile b/drivers/nvmem/layouts/Makefile index 4940c9db0665..dd6c6c70b1a9 100644 --- a/drivers/nvmem/layouts/Makefile +++ b/drivers/nvmem/layouts/Makefile @@ -3,6 +3,7 @@ # Makefile for nvmem layouts. # +obj-$(CONFIG_NVMEM_LAYOUTS) += fixed-layout.o obj-$(CONFIG_NVMEM_LAYOUT_SL28_VPD) += sl28vpd.o obj-$(CONFIG_NVMEM_LAYOUT_ONIE_TLV) += onie-tlv.o obj-$(CONFIG_NVMEM_LAYOUT_U_BOOT_ENV) += u-boot-env.o diff --git a/drivers/nvmem/layouts/fixed-layout.c b/drivers/nvmem/layouts/fixed-layout.c new file mode 100644 index 000000000000..635d448b3dd2 --- /dev/null +++ b/drivers/nvmem/layouts/fixed-layout.c @@ -0,0 +1,58 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* + * Copyright 2026 Bootlin + * + * Authors: Mathieu Dubois-Briand + */ + +#include +#include + +#include "../internals.h" + +static int fixed_layout_add_cells(struct nvmem_layout *layout) +{ + struct device_node *np; + int ret; + + np = of_nvmem_layout_get_container(layout->nvmem); + if (!np) + return -ENOENT; + + ret = nvmem_add_cells_from_dt(layout->nvmem, np); + of_node_put(np); + + return ret; +} + +static int fixed_layout_probe(struct nvmem_layout *layout) +{ + layout->add_cells = fixed_layout_add_cells; + + return nvmem_layout_register(layout); +} + +static void fixed_layout_remove(struct nvmem_layout *layout) +{ + nvmem_layout_unregister(layout); +} + +static const struct of_device_id fixed_layout_of_match_table[] = { + { .compatible = "fixed-layout", }, + {}, +}; + +static struct nvmem_layout_driver fixed_layout_layout = { + .driver = { + .name = "fixed-layout", + .of_match_table = fixed_layout_of_match_table, + }, + .probe = fixed_layout_probe, + .remove = fixed_layout_remove, +}; +module_nvmem_layout_driver(fixed_layout_layout); + +MODULE_AUTHOR("Mathieu Dubois-Briand"); +MODULE_LICENSE("GPL"); +MODULE_DEVICE_TABLE(of, fixed_layout_of_match_table); +MODULE_DESCRIPTION("NVMEM fixed-layout driver"); diff --git a/drivers/pinctrl/qcom/pinctrl-ipq8064.c b/drivers/pinctrl/qcom/pinctrl-ipq8064.c index 78d320d56be6..0a50486337d3 100644 --- a/drivers/pinctrl/qcom/pinctrl-ipq8064.c +++ b/drivers/pinctrl/qcom/pinctrl-ipq8064.c @@ -480,7 +480,7 @@ static const char * const ps_hold_groups[] = { }; static const struct pinfunction ipq8064_functions[] = { - IPQ_PIN_FUNCTION(gpio), + IPQ_GPIO_PIN_FUNCTION(gpio), IPQ_PIN_FUNCTION(mdio), IPQ_PIN_FUNCTION(ssbi), IPQ_PIN_FUNCTION(spmi), @@ -507,19 +507,19 @@ static const struct pinfunction ipq8064_functions[] = { IPQ_PIN_FUNCTION(usb2_hsic), IPQ_PIN_FUNCTION(rgmii2), IPQ_PIN_FUNCTION(sata), - IPQ_PIN_FUNCTION(pcie1_rst), + IPQ_GPIO_PIN_FUNCTION(pcie1_rst), IPQ_PIN_FUNCTION(pcie1_prsnt), IPQ_PIN_FUNCTION(pcie1_pwren_n), IPQ_PIN_FUNCTION(pcie1_pwren), IPQ_PIN_FUNCTION(pcie1_pwrflt), IPQ_PIN_FUNCTION(pcie1_clk_req), - IPQ_PIN_FUNCTION(pcie2_rst), + IPQ_GPIO_PIN_FUNCTION(pcie2_rst), IPQ_PIN_FUNCTION(pcie2_prsnt), IPQ_PIN_FUNCTION(pcie2_pwren_n), IPQ_PIN_FUNCTION(pcie2_pwren), IPQ_PIN_FUNCTION(pcie2_pwrflt), IPQ_PIN_FUNCTION(pcie2_clk_req), - IPQ_PIN_FUNCTION(pcie3_rst), + IPQ_GPIO_PIN_FUNCTION(pcie3_rst), IPQ_PIN_FUNCTION(pcie3_prsnt), IPQ_PIN_FUNCTION(pcie3_pwren_n), IPQ_PIN_FUNCTION(pcie3_pwren), diff --git a/drivers/pinctrl/qcom/pinctrl-ipq9650.c b/drivers/pinctrl/qcom/pinctrl-ipq9650.c index 64e443aa31b2..5399d7ad934c 100644 --- a/drivers/pinctrl/qcom/pinctrl-ipq9650.c +++ b/drivers/pinctrl/qcom/pinctrl-ipq9650.c @@ -310,11 +310,11 @@ static const char *const audio_sec_mclk_out0_groups[] = { }; static const char *const audio_sec_mclk_in1_groups[] = { - "gpio37", + "gpio39", }; static const char *const audio_sec_mclk_out1_groups[] = { - "gpio37", + "gpio39", }; static const char *const audio_sec_groups[] = { diff --git a/drivers/pinctrl/qcom/pinctrl-msm.h b/drivers/pinctrl/qcom/pinctrl-msm.h index 4fbff61de6bb..b94ba1a4177e 100644 --- a/drivers/pinctrl/qcom/pinctrl-msm.h +++ b/drivers/pinctrl/qcom/pinctrl-msm.h @@ -24,6 +24,11 @@ struct pinctrl_pin_desc; fname##_groups, \ ARRAY_SIZE(fname##_groups)) +#define IPQ_GPIO_PIN_FUNCTION(fname) \ + [IPQ_MUX_##fname] = PINCTRL_GPIO_PINFUNCTION(#fname, \ + fname##_groups, \ + ARRAY_SIZE(fname##_groups)) + #define MSM_PIN_FUNCTION(fname) \ [msm_mux_##fname] = PINCTRL_PINFUNCTION(#fname, \ fname##_groups, \ diff --git a/drivers/ptp/ptp_ocp.c b/drivers/ptp/ptp_ocp.c index 35e911f1ad78..3d26ec1f7b9e 100644 --- a/drivers/ptp/ptp_ocp.c +++ b/drivers/ptp/ptp_ocp.c @@ -2177,9 +2177,11 @@ ptp_ocp_devlink_info_get(struct devlink *devlink, struct devlink_info_req *req, if (err) return err; + snprintf(buf, sizeof(buf), "%.*s", OCP_BOARD_ID_LEN, + (const char *)bp->board_id); err = devlink_info_version_fixed_put(req, DEVLINK_INFO_VERSION_GENERIC_BOARD_ID, - bp->board_id); + buf); if (err) return err; diff --git a/drivers/s390/cio/vfio_ccw_async.c b/drivers/s390/cio/vfio_ccw_async.c index 420d89ba7f83..4aff0b58fa5d 100644 --- a/drivers/s390/cio/vfio_ccw_async.c +++ b/drivers/s390/cio/vfio_ccw_async.c @@ -8,6 +8,7 @@ */ #include +#include #include "vfio_ccw_private.h" @@ -24,11 +25,20 @@ static ssize_t vfio_ccw_async_region_read(struct vfio_ccw_private *private, return -EINVAL; mutex_lock(&private->io_mutex); + + if (i >= private->num_regions) { + ret = -EINVAL; + goto out_unlock; + } + + i = array_index_nospec(i, private->num_regions); region = private->region[i].data; if (copy_to_user(buf, (void *)region + pos, count)) ret = -EFAULT; else ret = count; + +out_unlock: mutex_unlock(&private->io_mutex); return ret; } @@ -48,6 +58,12 @@ static ssize_t vfio_ccw_async_region_write(struct vfio_ccw_private *private, if (!mutex_trylock(&private->io_mutex)) return -EAGAIN; + if (i >= private->num_regions) { + ret = -EINVAL; + goto out_unlock; + } + + i = array_index_nospec(i, private->num_regions); region = private->region[i].data; if (copy_from_user((void *)region + pos, buf, count)) { ret = -EFAULT; diff --git a/drivers/s390/cio/vfio_ccw_chp.c b/drivers/s390/cio/vfio_ccw_chp.c index 38c176cf6295..7708eb4d6de0 100644 --- a/drivers/s390/cio/vfio_ccw_chp.c +++ b/drivers/s390/cio/vfio_ccw_chp.c @@ -9,6 +9,7 @@ */ #include +#include #include #include "vfio_ccw_private.h" @@ -26,6 +27,13 @@ static ssize_t vfio_ccw_schib_region_read(struct vfio_ccw_private *private, return -EINVAL; mutex_lock(&private->io_mutex); + + if (i >= private->num_regions) { + ret = -EINVAL; + goto out; + } + + i = array_index_nospec(i, private->num_regions); region = private->region[i].data; if (cio_update_schib(sch)) { @@ -85,19 +93,30 @@ static ssize_t vfio_ccw_crw_region_read(struct vfio_ccw_private *private, loff_t pos = *ppos & VFIO_CCW_OFFSET_MASK; struct ccw_crw_region *region; struct vfio_ccw_crw *crw; + unsigned long flags; int ret; if (pos + count > sizeof(*region)) return -EINVAL; + mutex_lock(&private->io_mutex); + if (i >= private->num_regions) { + ret = -EINVAL; + goto out; + } + + i = array_index_nospec(i, private->num_regions); + region = private->region[i].data; + + spin_lock_irqsave(&private->crw_lock, flags); crw = list_first_entry_or_null(&private->crw, struct vfio_ccw_crw, next); if (crw) list_del(&crw->next); - mutex_lock(&private->io_mutex); - region = private->region[i].data; + /* Drop CRW lock while copying to userspace */ + spin_unlock_irqrestore(&private->crw_lock, flags); if (crw) memcpy(®ion->crw, &crw->crw, sizeof(region->crw)); @@ -108,14 +127,16 @@ static ssize_t vfio_ccw_crw_region_read(struct vfio_ccw_private *private, ret = count; region->crw = 0; - - mutex_unlock(&private->io_mutex); - kfree(crw); /* Notify the guest if more CRWs are on our queue */ + spin_lock_irqsave(&private->crw_lock, flags); if (!list_empty(&private->crw) && private->crw_trigger) eventfd_signal(private->crw_trigger); + spin_unlock_irqrestore(&private->crw_lock, flags); + +out: + mutex_unlock(&private->io_mutex); return ret; } diff --git a/drivers/s390/cio/vfio_ccw_cp.c b/drivers/s390/cio/vfio_ccw_cp.c index 7561aa7d3e01..58722c4baa25 100644 --- a/drivers/s390/cio/vfio_ccw_cp.c +++ b/drivers/s390/cio/vfio_ccw_cp.c @@ -233,6 +233,7 @@ static void convert_ccw0_to_ccw1(struct ccw1 *source, unsigned long len) } #define idal_is_2k(_cp) (!(_cp)->orb.cmd.c64 || (_cp)->orb.cmd.i2k) +#define get_idaw_size(_cp) ((_cp)->orb.cmd.c64 ? sizeof(u64) : sizeof(u32)) /* * Helpers to operate ccwchain. @@ -332,6 +333,7 @@ static struct ccwchain *ccwchain_alloc(struct channel_program *cp, int len) goto out_err; list_add_tail(&chain->next, &cp->ccwchain_list); + cp->ccwchain_count++; return chain; @@ -376,11 +378,9 @@ static void ccwchain_cda_free(struct ccwchain *chain, int idx) static int ccwchain_calc_length(u64 iova, struct channel_program *cp) { struct ccw1 *ccw = cp->guest_cp; - int cnt = 0; - - do { - cnt++; + int cnt; + for (cnt = 1; cnt <= CCWCHAIN_LEN_MAX; cnt++, ccw++) { /* * We want to keep counting if the current CCW has the * command-chaining flag enabled, or if it is a TIC CCW @@ -390,15 +390,10 @@ static int ccwchain_calc_length(u64 iova, struct channel_program *cp) * after the TIC, depending on the results of its operation. */ if (!ccw_is_chain(ccw) && !is_tic_within_range(ccw, iova, cnt)) - break; + return cnt; + } - ccw++; - } while (cnt < CCWCHAIN_LEN_MAX + 1); - - if (cnt == CCWCHAIN_LEN_MAX + 1) - cnt = -EINVAL; - - return cnt; + return -EINVAL; } static int tic_target_chain_exists(struct ccw1 *tic, struct channel_program *cp) @@ -441,6 +436,10 @@ static int ccwchain_handle_ccw(dma32_t cda, struct channel_program *cp) if (len < 0) return len; + /* Limit number of chains in a single channel program */ + if (cp->ccwchain_count >= CCWCHAIN_COUNT_MAX) + return -EINVAL; + /* Need alloc a new chain for this one. */ chain = ccwchain_alloc(cp, len); if (!chain) @@ -455,9 +454,6 @@ static int ccwchain_handle_ccw(dma32_t cda, struct channel_program *cp) /* Loop for tics on this new chain. */ ret = ccwchain_loop_tic(chain, cp); - if (ret) - ccwchain_free(chain); - return ret; } @@ -486,6 +482,23 @@ static int ccwchain_loop_tic(struct ccwchain *chain, struct channel_program *cp) return 0; } +static int ccwchain_build_ccws(dma32_t cda, struct channel_program *cp) +{ + struct ccwchain *chain, *temp; + int ret; + + ret = ccwchain_handle_ccw(cda, cp); + + if (ret) { + /* Cleanup if an error occurred */ + list_for_each_entry_safe(chain, temp, &cp->ccwchain_list, next) { + ccwchain_free(chain); + } + } + + return ret; +} + static int ccwchain_fetch_tic(struct ccw1 *ccw, struct channel_program *cp) { @@ -511,7 +524,8 @@ static dma64_t *get_guest_idal(struct ccw1 *ccw, struct channel_program *cp, int &container_of(cp, struct vfio_ccw_private, cp)->vdev; dma64_t *idaws; dma32_t *idaws_f1; - int idal_len = idaw_nr * sizeof(*idaws); + u64 first_idaw; + int idal_len = idaw_nr * get_idaw_size(cp); int idaw_size = idal_is_2k(cp) ? PAGE_SIZE / 2 : PAGE_SIZE; int idaw_mask = ~(idaw_size - 1); int i, ret; @@ -527,6 +541,18 @@ static dma64_t *get_guest_idal(struct ccw1 *ccw, struct channel_program *cp, int kfree(idaws); return ERR_PTR(ret); } + + idaws_f1 = (dma32_t *)idaws; + if (cp->orb.cmd.c64) + first_idaw = dma64_to_u64(idaws[0]); + else + first_idaw = dma32_to_u32(idaws_f1[0]); + + /* Unexpected mismatch from earlier read */ + if (first_idaw != cp->guest_iova) { + kfree(idaws); + return ERR_PTR(-EINVAL); + } } else { /* Fabricate an IDAL based off CCW data address */ if (cp->orb.cmd.c64) { @@ -568,7 +594,7 @@ static int ccw_count_idaws(struct ccw1 *ccw, struct vfio_device *vdev = &container_of(cp, struct vfio_ccw_private, cp)->vdev; u64 iova; - int size = cp->orb.cmd.c64 ? sizeof(u64) : sizeof(u32); + int size = get_idaw_size(cp); int ret; int bytes = 1; @@ -592,6 +618,9 @@ static int ccw_count_idaws(struct ccw1 *ccw, iova = dma32_to_u32(ccw->cda); } + /* Save the read address for later */ + cp->guest_iova = iova; + /* Format-1 IDAWs operate on 2K each */ if (!cp->orb.cmd.c64) return idal_2k_nr_words((void *)iova, bytes); @@ -731,11 +760,12 @@ int cp_init(struct channel_program *cp, union orb *orb) vdev->dev, "Prefetching channel program even though prefetch not specified in ORB"); + cp->ccwchain_count = 0; INIT_LIST_HEAD(&cp->ccwchain_list); memcpy(&cp->orb, orb, sizeof(*orb)); /* Build a ccwchain for the first CCW segment */ - ret = ccwchain_handle_ccw(orb->cmd.cpa, cp); + ret = ccwchain_build_ccws(orb->cmd.cpa, cp); if (!ret) cp->initialized = true; @@ -947,17 +977,23 @@ void cp_update_scsw(struct channel_program *cp, union scsw *scsw) */ bool cp_iova_pinned(struct channel_program *cp, u64 iova, u64 length) { + struct vfio_ccw_private *private = + container_of(cp, struct vfio_ccw_private, cp); struct ccwchain *chain; int i; if (!cp->initialized) return false; + mutex_lock(&private->io_mutex); list_for_each_entry(chain, &cp->ccwchain_list, next) { for (i = 0; i < chain->ch_len; i++) - if (page_array_iova_pinned(&chain->ch_pa[i], iova, length)) + if (page_array_iova_pinned(&chain->ch_pa[i], iova, length)) { + mutex_unlock(&private->io_mutex); return true; + } } + mutex_unlock(&private->io_mutex); return false; } diff --git a/drivers/s390/cio/vfio_ccw_cp.h b/drivers/s390/cio/vfio_ccw_cp.h index fc31eb699807..9af98ff12d67 100644 --- a/drivers/s390/cio/vfio_ccw_cp.h +++ b/drivers/s390/cio/vfio_ccw_cp.h @@ -23,11 +23,19 @@ */ #define CCWCHAIN_LEN_MAX 256 +/* + * Maximum number of chains + */ +#define CCWCHAIN_COUNT_MAX 16 + /** * struct channel_program - manage information for channel program * @ccwchain_list: list head of ccwchains * @orb: orb for the currently processed ssch request * @initialized: whether this instance is actually initialized + * @guest_cp: copy of guest channel program + * @ccwchain_count: number of channel program segments (linked by TIC) + * @guest_iova: first data address of a guest channel program * * @ccwchain_list is the head of a ccwchain list, that contents the * translated result of the guest channel program that pointed out by @@ -38,6 +46,8 @@ struct channel_program { union orb orb; bool initialized; struct ccw1 *guest_cp; + unsigned int ccwchain_count; + u64 guest_iova; }; int cp_init(struct channel_program *cp, union orb *orb); diff --git a/drivers/s390/cio/vfio_ccw_drv.c b/drivers/s390/cio/vfio_ccw_drv.c index 1a095085bc72..ab6b518cc353 100644 --- a/drivers/s390/cio/vfio_ccw_drv.c +++ b/drivers/s390/cio/vfio_ccw_drv.c @@ -91,6 +91,7 @@ void vfio_ccw_sch_io_todo(struct work_struct *work) is_final = !(scsw_actl(&irb->scsw) & (SCSW_ACTL_DEVACT | SCSW_ACTL_SCHACT)); + mutex_lock(&private->io_mutex); if (scsw_is_solicited(&irb->scsw)) { cp_update_scsw(&private->cp, &irb->scsw); if (is_final && private->state == VFIO_CCW_STATE_CP_PENDING) { @@ -98,9 +99,7 @@ void vfio_ccw_sch_io_todo(struct work_struct *work) cp_is_finished = true; } } - mutex_lock(&private->io_mutex); memcpy(private->io_region->irb_area, irb, sizeof(*irb)); - mutex_unlock(&private->io_mutex); /* * Reset to IDLE only if processing of a channel program @@ -110,6 +109,7 @@ void vfio_ccw_sch_io_todo(struct work_struct *work) */ if (cp_is_finished) private->state = VFIO_CCW_STATE_IDLE; + mutex_unlock(&private->io_mutex); if (private->io_trigger) eventfd_signal(private->io_trigger); @@ -118,11 +118,25 @@ void vfio_ccw_sch_io_todo(struct work_struct *work) void vfio_ccw_crw_todo(struct work_struct *work) { struct vfio_ccw_private *private; + unsigned long flags; private = container_of(work, struct vfio_ccw_private, crw_work); + spin_lock_irqsave(&private->crw_lock, flags); if (!list_empty(&private->crw) && private->crw_trigger) eventfd_signal(private->crw_trigger); + spin_unlock_irqrestore(&private->crw_lock, flags); +} + +void vfio_ccw_notoper_todo(struct work_struct *work) +{ + struct vfio_ccw_private *private; + + private = container_of(work, struct vfio_ccw_private, notoper_work); + + mutex_lock(&private->io_mutex); + cp_free(&private->cp); + mutex_unlock(&private->io_mutex); } /* @@ -275,6 +289,7 @@ static void vfio_ccw_queue_crw(struct vfio_ccw_private *private, unsigned int rsid) { struct vfio_ccw_crw *crw; + unsigned long flags; /* * If unable to allocate a CRW, just drop the event and @@ -292,7 +307,9 @@ static void vfio_ccw_queue_crw(struct vfio_ccw_private *private, crw->crw.erc = erc; crw->crw.rsid = rsid; + spin_lock_irqsave(&private->crw_lock, flags); list_add_tail(&crw->next, &private->crw); + spin_unlock_irqrestore(&private->crw_lock, flags); queue_work(vfio_ccw_work_q, &private->crw_work); } diff --git a/drivers/s390/cio/vfio_ccw_fsm.c b/drivers/s390/cio/vfio_ccw_fsm.c index 4d7988ea47ef..5fd94e9d5c61 100644 --- a/drivers/s390/cio/vfio_ccw_fsm.c +++ b/drivers/s390/cio/vfio_ccw_fsm.c @@ -170,8 +170,8 @@ static void fsm_notoper(struct vfio_ccw_private *private, css_sched_sch_todo(sch, SCH_TODO_UNREG); private->state = VFIO_CCW_STATE_NOT_OPER; - /* This is usually handled during CLOSE event */ - cp_free(&private->cp); + /* This routine could be called from IRQ context, so defer */ + queue_work(vfio_ccw_work_q, &private->notoper_work); } /* @@ -410,7 +410,11 @@ static void fsm_close(struct vfio_ccw_private *private, private->state = VFIO_CCW_STATE_STANDBY; spin_unlock_irq(&sch->lock); + + mutex_lock(&private->io_mutex); cp_free(&private->cp); + mutex_unlock(&private->io_mutex); + return; err_unlock: diff --git a/drivers/s390/cio/vfio_ccw_ops.c b/drivers/s390/cio/vfio_ccw_ops.c index 45ec722d25ea..5ce91285c7d5 100644 --- a/drivers/s390/cio/vfio_ccw_ops.c +++ b/drivers/s390/cio/vfio_ccw_ops.c @@ -54,6 +54,8 @@ static int vfio_ccw_mdev_init_dev(struct vfio_device *vdev) INIT_LIST_HEAD(&private->crw); INIT_WORK(&private->io_work, vfio_ccw_sch_io_todo); INIT_WORK(&private->crw_work, vfio_ccw_crw_todo); + INIT_WORK(&private->notoper_work, vfio_ccw_notoper_todo); + spin_lock_init(&private->crw_lock); private->cp.guest_cp = kzalloc_objs(struct ccw1, CCWCHAIN_LEN_MAX); if (!private->cp.guest_cp) @@ -130,11 +132,28 @@ static void vfio_ccw_mdev_release_dev(struct vfio_device *vdev) struct vfio_ccw_private *private = container_of(vdev, struct vfio_ccw_private, vdev); struct vfio_ccw_crw *crw, *temp; + unsigned long flags; + /* + * Ensure these work items are fully drained, so none can + * fire after being released. + * + * notoper_work should have nothing to do here, because only + * open devices could have channel_program resources in use + * and those would be released during close. Nevertheless, + * call flush here as well to be certain anything that was + * allocated is freed. + */ + cancel_work_sync(&private->io_work); + cancel_work_sync(&private->crw_work); + flush_work(&private->notoper_work); + + spin_lock_irqsave(&private->crw_lock, flags); list_for_each_entry_safe(crw, temp, &private->crw, next) { list_del(&crw->next); kfree(crw); } + spin_unlock_irqrestore(&private->crw_lock, flags); kmem_cache_free(vfio_ccw_crw_region, private->crw_region); kmem_cache_free(vfio_ccw_schib_region, private->schib_region); @@ -202,6 +221,19 @@ static void vfio_ccw_mdev_close_device(struct vfio_device *vdev) container_of(vdev, struct vfio_ccw_private, vdev); vfio_ccw_fsm_event(private, VFIO_CCW_EVENT_CLOSE); + + /* + * Ensure these work items are drained, in the event the + * device is re-opened instead of released. + * + * notoper_work needs to be given a chance to run if it + * is queued, so any memory associated with the channel + * program can be returned. + */ + cancel_work_sync(&private->io_work); + cancel_work_sync(&private->crw_work); + flush_work(&private->notoper_work); + vfio_ccw_unregister_dev_regions(private); } @@ -243,6 +275,7 @@ static ssize_t vfio_ccw_mdev_read(struct vfio_device *vdev, return vfio_ccw_mdev_read_io_region(private, buf, count, ppos); default: index -= VFIO_CCW_NUM_REGIONS; + index = array_index_nospec(index, private->num_regions); return private->region[index].ops->read(private, buf, count, ppos); } @@ -295,6 +328,7 @@ static ssize_t vfio_ccw_mdev_write(struct vfio_device *vdev, return vfio_ccw_mdev_write_io_region(private, buf, count, ppos); default: index -= VFIO_CCW_NUM_REGIONS; + index = array_index_nospec(index, private->num_regions); return private->region[index].ops->write(private, buf, count, ppos); } @@ -338,11 +372,8 @@ static int vfio_ccw_mdev_ioctl_get_region_info(struct vfio_device *vdev, VFIO_CCW_NUM_REGIONS + private->num_regions) return -EINVAL; - info->index = array_index_nospec(info->index, - VFIO_CCW_NUM_REGIONS + - private->num_regions); - i = info->index - VFIO_CCW_NUM_REGIONS; + i = array_index_nospec(i, private->num_regions); info->offset = VFIO_CCW_INDEX_TO_OFFSET(info->index); info->size = private->region[i].size; diff --git a/drivers/s390/cio/vfio_ccw_private.h b/drivers/s390/cio/vfio_ccw_private.h index 0501d4bbcdbd..3bd0171d38d0 100644 --- a/drivers/s390/cio/vfio_ccw_private.h +++ b/drivers/s390/cio/vfio_ccw_private.h @@ -88,7 +88,8 @@ struct vfio_ccw_parent { * @state: internal state of the device * @completion: synchronization helper of the I/O completion * @io_region: MMIO region to input/output I/O arguments/results - * @io_mutex: protect against concurrent update of I/O regions + * @io_mutex: protect against concurrent update of I/O resources + * and @cp lifecycle * @region: additional regions for other subchannel operations * @cmd_region: MMIO region for asynchronous I/O commands other than START * @schib_region: MMIO region for SCHIB information @@ -97,11 +98,14 @@ struct vfio_ccw_parent { * @cp: channel program for the current I/O operation * @irb: irb info received from interrupt * @scsw: scsw info + * @crw_lock: serialization of CRW list information + * @crw: list of Channel Report Word elements * @io_trigger: eventfd ctx for signaling userspace I/O results * @crw_trigger: eventfd ctx for signaling userspace CRW information * @req_trigger: eventfd ctx for signaling userspace to return device * @io_work: work for deferral process of I/O handling * @crw_work: work for deferral process of CRW handling + * @notoper_work: work for deferred processing in not-operational state */ struct vfio_ccw_private { struct vfio_device vdev; @@ -118,6 +122,8 @@ struct vfio_ccw_private { struct channel_program cp; struct irb irb; union scsw scsw; + + spinlock_t crw_lock; struct list_head crw; struct eventfd_ctx *io_trigger; @@ -125,11 +131,13 @@ struct vfio_ccw_private { struct eventfd_ctx *req_trigger; struct work_struct io_work; struct work_struct crw_work; + struct work_struct notoper_work; } __aligned(8); int vfio_ccw_sch_quiesce(struct subchannel *sch); void vfio_ccw_sch_io_todo(struct work_struct *work); void vfio_ccw_crw_todo(struct work_struct *work); +void vfio_ccw_notoper_todo(struct work_struct *work); extern struct mdev_driver vfio_ccw_mdev_driver; diff --git a/drivers/s390/crypto/zcrypt_ccamisc.c b/drivers/s390/crypto/zcrypt_ccamisc.c index 86d2ee78c9f4..d4ce6352b5b2 100644 --- a/drivers/s390/crypto/zcrypt_ccamisc.c +++ b/drivers/s390/crypto/zcrypt_ccamisc.c @@ -15,6 +15,7 @@ #include #include #include +#include #include #include @@ -267,6 +268,10 @@ EXPORT_SYMBOL(cca_check_sececckeytoken); * block, reply CPRB and reply param block and fill in values * for the common fields. Returns 0 on success or errno value * on failure. + * It is guaranteed that request and a possible param block + * are aligned to a 4 byte boundary. Furthermore if a param + * block is used, the memory allocated for this is rounded up to + * the next multiple of 4 bytes. */ static int alloc_and_prep_cprbmem(size_t paramblen, u8 **p_cprb_mem, @@ -275,7 +280,8 @@ static int alloc_and_prep_cprbmem(size_t paramblen, u32 xflags) { u8 *cprbmem = NULL; - size_t cprbplusparamblen = sizeof(struct CPRBX) + paramblen; + size_t cprbplusparamblen = + ALIGN(sizeof(struct CPRBX), 4) + ALIGN(paramblen, 4); size_t len = 2 * cprbplusparamblen; struct CPRBX *preqcblk, *prepcblk; @@ -302,10 +308,10 @@ static int alloc_and_prep_cprbmem(size_t paramblen, memcpy(preqcblk->func_id, "T2", 2); preqcblk->rpl_msgbl = cprbplusparamblen; if (paramblen) { - preqcblk->req_parmb = - ((u8 __user *)preqcblk) + sizeof(struct CPRBX); - preqcblk->rpl_parmb = - ((u8 __user *)prepcblk) + sizeof(struct CPRBX); + preqcblk->req_parmb = ((u8 __user *)preqcblk) + + ALIGN(sizeof(struct CPRBX), 4); + preqcblk->rpl_parmb = ((u8 __user *)prepcblk) + + ALIGN(sizeof(struct CPRBX), 4); } *p_cprb_mem = cprbmem; @@ -323,8 +329,10 @@ static int alloc_and_prep_cprbmem(size_t paramblen, */ static void free_cprbmem(void *mem, size_t paramblen, bool scrub, u32 xflags) { + size_t cprblen = ALIGN(sizeof(struct CPRBX), 4) + ALIGN(paramblen, 4); + if (mem && scrub) - memzero_explicit(mem, 2 * (sizeof(struct CPRBX) + paramblen)); + memzero_explicit(mem, 2 * cprblen); if (xflags & ZCRYPT_XFLAG_NOMEMALLOC) mempool_free(mem, cprb_mempool); diff --git a/drivers/s390/crypto/zcrypt_ep11misc.c b/drivers/s390/crypto/zcrypt_ep11misc.c index 3dda9589f2b9..2d900ffc5068 100644 --- a/drivers/s390/crypto/zcrypt_ep11misc.c +++ b/drivers/s390/crypto/zcrypt_ep11misc.c @@ -14,6 +14,7 @@ #include #include #include +#include #include #include #include @@ -355,21 +356,24 @@ EXPORT_SYMBOL(ep11_check_aes_key); /* * Allocate and prepare ep11 cprb plus additional payload. + * It is guaranteed that the memory is aligned to a 4 byte boundary. + * Furthermore the memory allocation is rounded up to the next + * multiple of 4 bytes (with taking the payload_len into account). */ static void *alloc_cprbmem(size_t payload_len, u32 xflags) { - size_t len = sizeof(struct ep11_cprb) + payload_len; + size_t memlen = ALIGN(sizeof(struct ep11_cprb) + payload_len, 4); struct ep11_cprb *cprb = NULL; if (xflags & ZCRYPT_XFLAG_NOMEMALLOC) { - if (len <= CPRB_MEMPOOL_ITEM_SIZE) + if (memlen <= CPRB_MEMPOOL_ITEM_SIZE) cprb = mempool_alloc_preallocated(cprb_mempool); } else { - cprb = kmalloc(len, GFP_KERNEL); + cprb = kmalloc(memlen, GFP_KERNEL); } if (!cprb) return NULL; - memset(cprb, 0, len); + memset(cprb, 0, memlen); cprb->cprb_len = sizeof(struct ep11_cprb); cprb->cprb_ver_id = 0x04; @@ -385,8 +389,10 @@ static void *alloc_cprbmem(size_t payload_len, u32 xflags) */ static void free_cprbmem(void *mem, size_t payload_len, bool scrub, u32 xflags) { + size_t memlen = ALIGN(sizeof(struct ep11_cprb) + payload_len, 4); + if (mem && scrub) - memzero_explicit(mem, sizeof(struct ep11_cprb) + payload_len); + memzero_explicit(mem, memlen); if (xflags & ZCRYPT_XFLAG_NOMEMALLOC) mempool_free(mem, cprb_mempool); diff --git a/drivers/s390/crypto/zcrypt_msgtype6.c b/drivers/s390/crypto/zcrypt_msgtype6.c index 40f72cdf284d..3df1d676de5d 100644 --- a/drivers/s390/crypto/zcrypt_msgtype6.c +++ b/drivers/s390/crypto/zcrypt_msgtype6.c @@ -19,6 +19,7 @@ #include #include #include +#include #include "ap_bus.h" #include "zcrypt_api.h" @@ -34,6 +35,9 @@ #define CEXXC_RESPONSE_TYPE_XCRB 1 #define CEXXC_RESPONSE_TYPE_EP11 2 +/* smallest possible EP11 payload size */ +#define MIN_EP11_PAYLOAD_SIZE 5 + MODULE_AUTHOR("IBM Corporation"); MODULE_DESCRIPTION("Cryptographic Coprocessor (message type 6), " \ "Copyright IBM Corp. 2001, 2023"); @@ -342,49 +346,39 @@ static int xcrb_msg_to_type6cprb_msgx(bool userspace, struct ap_message *ap_msg, }; } __packed * msg = ap_msg->msg; - int rcblen = CEIL4(xcrb->request_control_blk_length); - int req_sumlen, resp_sumlen; - char *req_data = ap_msg->msg + sizeof(struct type6_hdr) + rcblen; - char *function_code; + size_t req_cblen, rep_cblen, req_sumlen, rep_sumlen; + char *function_code, *req_data; - if (CEIL4(xcrb->request_control_blk_length) < - xcrb->request_control_blk_length) - return -EINVAL; /* overflow after alignment*/ - - /* length checks */ - ap_msg->len = sizeof(struct type6_hdr) + - CEIL4(xcrb->request_control_blk_length) + - xcrb->request_data_length; + /* request length and overflow checks */ + if (xcrb->request_control_blk_length < sizeof(struct CPRBX)) + return -EINVAL; + req_cblen = CEIL4((size_t)xcrb->request_control_blk_length); + if (req_cblen > U32_MAX) + return -EINVAL; + req_sumlen = req_cblen + xcrb->request_data_length; + if (req_sumlen > U32_MAX) + return -EINVAL; + ap_msg->len = sizeof(struct type6_hdr) + req_sumlen; if (ap_msg->len > ap_msg->bufsize) return -EINVAL; - - /* - * Overflow check - * sum must be greater (or equal) than the largest operand - */ - req_sumlen = CEIL4(xcrb->request_control_blk_length) + - xcrb->request_data_length; - if ((CEIL4(xcrb->request_control_blk_length) <= - xcrb->request_data_length) ? + if (req_cblen <= xcrb->request_data_length ? req_sumlen < xcrb->request_data_length : - req_sumlen < CEIL4(xcrb->request_control_blk_length)) { + req_sumlen < req_cblen) { return -EINVAL; } - if (CEIL4(xcrb->reply_control_blk_length) < - xcrb->reply_control_blk_length) - return -EINVAL; /* overflow after alignment*/ - - /* - * Overflow check - * sum must be greater (or equal) than the largest operand - */ - resp_sumlen = CEIL4(xcrb->reply_control_blk_length) + - xcrb->reply_data_length; - if ((CEIL4(xcrb->reply_control_blk_length) <= - xcrb->reply_data_length) ? - resp_sumlen < xcrb->reply_data_length : - resp_sumlen < CEIL4(xcrb->reply_control_blk_length)) { + /* reply length and overflow checks */ + if (xcrb->reply_control_blk_length < sizeof(struct CPRBX)) + return -EINVAL; + rep_cblen = CEIL4((size_t)xcrb->reply_control_blk_length); + if (rep_cblen > U32_MAX) + return -EINVAL; + rep_sumlen = rep_cblen + xcrb->reply_data_length; + if (rep_sumlen > U32_MAX) + return -EINVAL; + if (rep_cblen <= xcrb->reply_data_length ? + rep_sumlen < xcrb->reply_data_length : + rep_sumlen < rep_cblen) { return -EINVAL; } @@ -393,7 +387,7 @@ static int xcrb_msg_to_type6cprb_msgx(bool userspace, struct ap_message *ap_msg, memcpy(msg->hdr.agent_id, &xcrb->agent_ID, sizeof(xcrb->agent_ID)); msg->hdr.tocardlen1 = xcrb->request_control_blk_length; if (xcrb->request_data_length) { - msg->hdr.offset2 = msg->hdr.offset1 + rcblen; + msg->hdr.offset2 = msg->hdr.offset1 + req_cblen; msg->hdr.tocardlen2 = xcrb->request_data_length; } msg->hdr.fromcardlen1 = xcrb->reply_control_blk_length; @@ -404,8 +398,12 @@ static int xcrb_msg_to_type6cprb_msgx(bool userspace, struct ap_message *ap_msg, xcrb->request_control_blk_addr, xcrb->request_control_blk_length)) return -EFAULT; - if (msg->cprbx.cprb_len + sizeof(msg->hdr.function_code) > - xcrb->request_control_blk_length) + /* pad tail with 0 up to req_cblen */ + if (xcrb->request_control_blk_length < req_cblen) + memset(msg->userdata + xcrb->request_control_blk_length, + 0, req_cblen - xcrb->request_control_blk_length); + /* copy subfunction code into AP msg type 6 function code field */ + if (msg->cprbx.cprb_len > req_cblen - sizeof(msg->hdr.function_code)) return -EINVAL; function_code = ((unsigned char *)&msg->cprbx) + msg->cprbx.cprb_len; memcpy(msg->hdr.function_code, function_code, @@ -437,20 +435,70 @@ static int xcrb_msg_to_type6cprb_msgx(bool userspace, struct ap_message *ap_msg, } /* copy data block */ - if (xcrb->request_data_length && - z_copy_from_user(userspace, req_data, xcrb->request_data_address, - xcrb->request_data_length)) - return -EFAULT; + if (xcrb->request_data_length) { + req_data = ap_msg->msg + sizeof(struct type6_hdr) + req_cblen; + if (z_copy_from_user(userspace, req_data, + xcrb->request_data_address, + xcrb->request_data_length)) + return -EFAULT; + } return 0; } +/* + * Simple asn1 int reader/decoder helper function + * Returns number of bytes processed or < 0 on failure + * Only accepts int length values of 1, 2 or 4. + */ +static inline int asn1_int_decode(const u8 *buf, size_t intlen, u32 *u) +{ + switch (intlen) { + case 1: + *u = (u32)(*buf); + return 1; + case 2: + *u = (u32)get_unaligned_be16(buf); + return 2; + case 4: + *u = (u32)get_unaligned_be32(buf); + return 4; + default: + return -EINVAL; + } +} + +/* + * Simple asn1 length parse helper function + * Returns number of bytes processed or < 0 on failure + * Only accepts length encoded within the length octet + * or for long form 1, 2 or 4 octet length bytes. + */ +static inline int asn1_length_decode(const u8 *buf, size_t buflen, u32 *u) +{ + int i; + + if (buflen < 1) + return -EINVAL; + + if (*buf < 128) { + *u = (u32)(*buf & 0x7F); + return 1; + } + + i = *buf & 0x7F; + if (--buflen < i) + return -EINVAL; + i = asn1_int_decode(++buf, i, u); + + return i < 0 ? i : i + 1; +} + static int xcrb_msg_to_type6_ep11cprb_msgx(bool userspace, struct ap_message *ap_msg, struct ep11_urb *xcrb, unsigned int *fcode, unsigned int *domain) { - unsigned int lfmt; static struct type6_hdr static_type6_ep11_hdr = { .type = 0x06, .rqid = {0x00, 0x01}, @@ -462,34 +510,32 @@ static int xcrb_msg_to_type6_ep11cprb_msgx(bool userspace, struct ap_message *ap struct { struct type6_hdr hdr; union { - struct { - struct ep11_cprb cprbx; - unsigned char pld_tag; /* fixed value 0x30 */ - unsigned char pld_lenfmt; /* length format */ - } __packed; + struct ep11_cprb cprbx; DECLARE_FLEX_ARRAY(u8, userdata); }; } __packed * msg = ap_msg->msg; - struct pld_hdr { - unsigned char func_tag; /* fixed value 0x4 */ - unsigned char func_len; /* fixed value 0x4 */ - unsigned int func_val; /* function ID */ - unsigned char dom_tag; /* fixed value 0x4 */ - unsigned char dom_len; /* fixed value 0x4 */ - unsigned int dom_val; /* domain id */ - } __packed * payload_hdr = NULL; + size_t req_len, rep_len, pld_len; + unsigned char *pld; + int offs = 0, i; + unsigned int u; - if (CEIL4(xcrb->req_len) < xcrb->req_len) - return -EINVAL; /* overflow after alignment*/ - - /* length checks */ - ap_msg->len = sizeof(struct type6_hdr) + CEIL4(xcrb->req_len); + /* request length and overflow checks */ + if (xcrb->req_len < sizeof(struct ep11_cprb) + MIN_EP11_PAYLOAD_SIZE) + return -EINVAL; + req_len = CEIL4(xcrb->req_len); + if (req_len < xcrb->req_len || req_len > U32_MAX) + return -EINVAL; + ap_msg->len = sizeof(struct type6_hdr) + req_len; if (ap_msg->len > ap_msg->bufsize) return -EINVAL; - if (CEIL4(xcrb->resp_len) < xcrb->resp_len) - return -EINVAL; /* overflow after alignment*/ + /* reply length and overflow checks */ + if (xcrb->resp_len < sizeof(struct ep11_cprb)) + return -EINVAL; + rep_len = CEIL4(xcrb->resp_len); + if (rep_len < xcrb->resp_len || rep_len > U32_MAX) + return -EINVAL; /* prepare type6 header */ msg->hdr = static_type6_ep11_hdr; @@ -498,26 +544,55 @@ static int xcrb_msg_to_type6_ep11cprb_msgx(bool userspace, struct ap_message *ap /* Import CPRB data from the ioctl input parameter */ if (z_copy_from_user(userspace, msg->userdata, - (char __force __user *)xcrb->req, xcrb->req_len)) { + (char __force __user *)xcrb->req, xcrb->req_len)) return -EFAULT; - } + /* pad tail with 0 up to req_len */ + if (xcrb->req_len < req_len) + memset(msg->userdata + xcrb->req_len, 0, + req_len - xcrb->req_len); - if ((msg->pld_lenfmt & 0x80) == 0x80) { /*ext.len.fmt 2 or 3*/ - switch (msg->pld_lenfmt & 0x03) { - case 1: - lfmt = 2; - break; - case 2: - lfmt = 3; - break; - default: - return -EINVAL; - } - } else { - lfmt = 1; /* length format #1 */ - } - payload_hdr = (struct pld_hdr *)((&msg->pld_lenfmt) + lfmt); - *fcode = payload_hdr->func_val & 0xFFFF; + pld = msg->userdata + sizeof(struct ep11_cprb); + pld_len = msg->cprbx.payload_len; + if (pld_len != xcrb->req_len - sizeof(struct ep11_cprb)) + return -EINVAL; + /* + * At this point pld_len is always >= MIN_EP11_PAYLOAD_SIZE + * and the smallest supported asn1 payload is: + * payload tag (1 octet) + * payload length (1-5 octets) + * function tag (1 octet) + * function length (1-5 octets) + * function value (1-4 octets) + */ + + /* payload tag */ + if (pld[offs++] != 0x30) + return -EINVAL; + /* payload length field */ + i = asn1_length_decode(pld + offs, pld_len - offs, &u); + if (i < 0) + return -EINVAL; + offs += i; + if (offs >= pld_len || u > pld_len - offs) + return -EINVAL; + /* function tag */ + if (pld[offs++] != 0x04) + return -EINVAL; + /* function length */ + if (offs >= pld_len) + return -EINVAL; + i = asn1_length_decode(pld + offs, pld_len - offs, &u); + if (i < 0) + return -EINVAL; + offs += i; + if (offs >= pld_len || u > pld_len - offs) + return -EINVAL; + /* function value */ + i = asn1_int_decode(pld + offs, u, &u); + if (i < 0) + return -EINVAL; + offs += i; + *fcode = 0xFFFF & u; /* enable special processing based on the cprbs flags special bit */ if (msg->cprbx.flags & 0x20) @@ -1159,6 +1234,28 @@ int prep_ep11_ap_msg(bool userspace, struct ep11_urb *xcrb, func_code, domain); } +/* + * Simple asn1 int writer/encoder helper function + * Returns number of bytes processed or < 0 on failure + * Only accepts int length values of 1, 2 or 4. + */ +static inline int asn1_int_encode(u8 *buf, size_t intlen, u32 u) +{ + switch (intlen) { + case 1: + *buf = (u8)u; + return 1; + case 2: + put_unaligned_be16((u16)u, buf); + return 2; + case 4: + put_unaligned_be32((u32)u, buf); + return 4; + default: + return -EINVAL; + } +} + /* * The request distributor calls this function if it picked the CEX4P * device to handle a send_ep11_cprb request. @@ -1171,51 +1268,94 @@ static long zcrypt_msgtype6_send_ep11_cprb(bool userspace, struct zcrypt_queue * struct ap_message *ap_msg) { int rc; - unsigned int lfmt; struct ap_response_type *resp_type = &ap_msg->response; struct { struct type6_hdr hdr; struct ep11_cprb cprbx; - unsigned char pld_tag; /* fixed value 0x30 */ - unsigned char pld_lenfmt; /* payload length format */ } __packed * msg = ap_msg->msg; - struct pld_hdr { - unsigned char func_tag; /* fixed value 0x4 */ - unsigned char func_len; /* fixed value 0x4 */ - unsigned int func_val; /* function ID */ - unsigned char dom_tag; /* fixed value 0x4 */ - unsigned char dom_len; /* fixed value 0x4 */ - unsigned int dom_val; /* domain id */ - } __packed * payload_hdr = NULL; /* * The target domain field within the cprb body/payload block will be * replaced by the usage domain for non-management commands only. * Therefore we check the first bit of the 'flags' parameter for * management command indication. - * 0 - non management command - * 1 - management command */ - if (!((msg->cprbx.flags & 0x80) == 0x80)) { - msg->cprbx.target_id = (unsigned int) - AP_QID_QUEUE(zq->queue->qid); + if (!(msg->cprbx.flags & 0x80)) { + int i, offs = 0; + size_t pld_len; + u8 *pld; + u32 u; - if ((msg->pld_lenfmt & 0x80) == 0x80) { /*ext.len.fmt 2 or 3*/ - switch (msg->pld_lenfmt & 0x03) { - case 1: - lfmt = 2; - break; - case 2: - lfmt = 3; - break; - default: + /* update target field in ep11_cprb */ + msg->cprbx.target_id = (u32)AP_QID_QUEUE(zq->queue->qid); + + /* ptr and length to payload */ + pld = ap_msg->msg + + sizeof(struct type6_hdr) + sizeof(struct ep11_cprb); + pld_len = msg->cprbx.payload_len; + if (pld_len < MIN_EP11_PAYLOAD_SIZE) + return -EINVAL; + + /* + * Parse the asn1 payload, at least we have + * pld tag (1 octet) + * payload length (1-5 octets) + * function tag (1 octet) + * function length (1-5 octets) + * function value (1-4 octets) + * ----- optional fields ----- + * domain tag (1 octet) + * domain length (1-5 octets) + * domain value (1-4 octets) + * ... maybe much more data ... + */ + + /* payload tag */ + if (pld[offs++] != 0x30) + return -EINVAL; + /* payload length field */ + i = asn1_length_decode(pld + offs, pld_len - offs, &u); + if (i < 0) + return -EINVAL; + offs += i; + if (offs >= pld_len || u > pld_len - offs) + return -EINVAL; + /* function tag */ + if (pld[offs++] != 0x04) + return -EINVAL; + /* function length */ + if (offs >= pld_len) + return -EINVAL; + i = asn1_length_decode(pld + offs, pld_len - offs, &u); + if (i < 0) + return -EINVAL; + offs += i; + if (u > pld_len - offs) + return -EINVAL; + /* skip over the function value */ + offs += u; + /* is there some payload left which could hold a domain value ? */ + if (offs < pld_len && pld_len - offs >= 3) { + /* domain tag */ + if (pld[offs++] != 0x04) return -EINVAL; - } - } else { - lfmt = 1; /* length format #1 */ + /* domain length */ + i = asn1_length_decode(pld + offs, pld_len - offs, &u); + if (i < 0) + return -EINVAL; + offs += i; + if (offs >= pld_len || u > pld_len - offs) + return -EINVAL; + /* + * pld[offs] is now at the start of the domain value + * with the value sprawled in u octets. + */ + i = asn1_int_encode(pld + offs, u, + AP_QID_QUEUE(zq->queue->qid)); + if (i < 0) + return -EINVAL; + offs += i; } - payload_hdr = (struct pld_hdr *)((&msg->pld_lenfmt) + lfmt); - payload_hdr->dom_val = AP_QID_QUEUE(zq->queue->qid); } /* diff --git a/drivers/s390/net/ism_drv.c b/drivers/s390/net/ism_drv.c index d99c588d3e00..242da20f27e0 100644 --- a/drivers/s390/net/ism_drv.c +++ b/drivers/s390/net/ism_drv.c @@ -148,13 +148,16 @@ static int unregister_sba(struct ism_dev *ism) if (ret && ret != ISM_ERROR) return -EIO; + return 0; +} + +static void ism_free_sba(struct ism_dev *ism) +{ dma_free_coherent(&ism->pdev->dev, PAGE_SIZE, ism->sba, ism->sba_dma_addr); ism->sba = NULL; ism->sba_dma_addr = 0; - - return 0; } static int unregister_ieq(struct ism_dev *ism) @@ -168,13 +171,16 @@ static int unregister_ieq(struct ism_dev *ism) if (ret && ret != ISM_ERROR) return -EIO; + return 0; +} + +static void ism_free_ieq(struct ism_dev *ism) +{ dma_free_coherent(&ism->pdev->dev, PAGE_SIZE, ism->ieq, ism->ieq_dma_addr); ism->ieq = NULL; ism->ieq_dma_addr = 0; - - return 0; } static int ism_read_local_gid(struct dibs_dev *dibs) @@ -573,6 +579,7 @@ static int ism_dev_init(struct ism_dev *ism) unreg_sba: unregister_sba(ism); + ism_free_sba(ism); free_irq: free_irq(pci_irq_vector(pdev, 0), ism); free_vectors: @@ -585,9 +592,13 @@ static void ism_dev_exit(struct ism_dev *ism) { struct pci_dev *pdev = ism->pdev; + /* ism will only generate new IRQs while ieq & sba are registered */ unregister_ieq(ism); unregister_sba(ism); + /* drain ongoing irpt handlers */ free_irq(pci_irq_vector(pdev, 0), ism); + ism_free_ieq(ism); + ism_free_sba(ism); pci_free_irq_vectors(pdev); } diff --git a/drivers/s390/net/qeth_core_main.c b/drivers/s390/net/qeth_core_main.c index f18eed9df3c7..c3257b213360 100644 --- a/drivers/s390/net/qeth_core_main.c +++ b/drivers/s390/net/qeth_core_main.c @@ -4710,6 +4710,9 @@ static int qeth_snmp_command(struct qeth_card *card, char __user *udata) if (req_len > QETH_BUFSIZE) return -EINVAL; + if (qinfo.udata_len < sizeof(struct qeth_snmp_ureq_hdr)) + return -EINVAL; + iob = qeth_get_adapter_cmd(card, IPA_SETADP_SET_SNMP_CONTROL, req_len); if (!iob) return -ENOMEM; diff --git a/drivers/s390/net/qeth_l3_main.c b/drivers/s390/net/qeth_l3_main.c index 1542bfc9f561..f1ac9950dcb4 100644 --- a/drivers/s390/net/qeth_l3_main.c +++ b/drivers/s390/net/qeth_l3_main.c @@ -1415,6 +1415,11 @@ static int qeth_l3_arp_query(struct qeth_card *card, char __user *udata) rc = -EFAULT; goto out; } + + if (qinfo.udata_len < QETH_QARP_ENTRIES_OFFSET) { + rc = -EINVAL; + goto out; + } qinfo.udata = kzalloc(qinfo.udata_len, GFP_KERNEL); if (!qinfo.udata) { rc = -ENOMEM; diff --git a/drivers/soc/aspeed/Makefile b/drivers/soc/aspeed/Makefile index b35d74592964..b5188dcde37a 100644 --- a/drivers/soc/aspeed/Makefile +++ b/drivers/soc/aspeed/Makefile @@ -4,3 +4,4 @@ obj-$(CONFIG_ASPEED_LPC_SNOOP) += aspeed-lpc-snoop.o obj-$(CONFIG_ASPEED_UART_ROUTING) += aspeed-uart-routing.o obj-$(CONFIG_ASPEED_P2A_CTRL) += aspeed-p2a-ctrl.o obj-$(CONFIG_ASPEED_SOCINFO) += aspeed-socinfo.o +CONTEXT_ANALYSIS_aspeed-lpc-snoop.o := y diff --git a/drivers/soc/aspeed/aspeed-lpc-snoop.c b/drivers/soc/aspeed/aspeed-lpc-snoop.c index 28e491fffc7d..b09dca34099f 100644 --- a/drivers/soc/aspeed/aspeed-lpc-snoop.c +++ b/drivers/soc/aspeed/aspeed-lpc-snoop.c @@ -11,6 +11,7 @@ */ #include +#include #include #include #include @@ -74,7 +75,8 @@ struct aspeed_lpc_snoop_channel_cfg { struct aspeed_lpc_snoop_channel { const struct aspeed_lpc_snoop_channel_cfg *cfg; bool enabled; - struct kfifo fifo; + spinlock_t lock; + struct kfifo fifo __guarded_by(&lock); wait_queue_head_t wq; struct miscdevice miscdev; }; @@ -114,6 +116,7 @@ static ssize_t snoop_file_read(struct file *file, char __user *buffer, size_t count, loff_t *ppos) { struct aspeed_lpc_snoop_channel *chan = snoop_file_to_chan(file); + u8 *buf __free(kfree) = NULL; unsigned int copied; int ret = 0; @@ -125,9 +128,16 @@ static ssize_t snoop_file_read(struct file *file, char __user *buffer, if (ret == -ERESTARTSYS) return -EINTR; } - ret = kfifo_to_user(&chan->fifo, buffer, count, &copied); - if (ret) - return ret; + + count = min_t(size_t, count, SNOOP_FIFO_SIZE); + + buf = kmalloc(count, GFP_KERNEL); + if (!buf) + return -ENOMEM; + + copied = kfifo_out_spinlocked(&chan->fifo, buf, count, &chan->lock); + if (copied && copy_to_user(buffer, buf, copied)) + return -EFAULT; return copied; } @@ -151,11 +161,13 @@ static const struct file_operations snoop_fops = { /* Save a byte to a FIFO and discard the oldest byte if FIFO is full */ static void put_fifo_with_discard(struct aspeed_lpc_snoop_channel *chan, u8 val) { - if (!kfifo_initialized(&chan->fifo)) - return; - if (kfifo_is_full(&chan->fifo)) - kfifo_skip(&chan->fifo); - kfifo_put(&chan->fifo, val); + scoped_guard(spinlock, &chan->lock) { + if (!kfifo_initialized(&chan->fifo)) + return; + if (kfifo_is_full(&chan->fifo)) + kfifo_skip(&chan->fifo); + kfifo_put(&chan->fifo, val); + } wake_up_interruptible(&chan->wq); } @@ -239,9 +251,11 @@ static int aspeed_lpc_enable_snoop(struct device *dev, if (!channel->miscdev.name) return -ENOMEM; - rc = kfifo_alloc(&channel->fifo, SNOOP_FIFO_SIZE, GFP_KERNEL); - if (rc) - return rc; + scoped_guard(spinlock_init, &channel->lock) { + rc = kfifo_alloc(&channel->fifo, SNOOP_FIFO_SIZE, GFP_KERNEL); + if (rc) + return rc; + } rc = misc_register(&channel->miscdev); if (rc) diff --git a/drivers/staging/rtl8723bs/core/rtw_ieee80211.c b/drivers/staging/rtl8723bs/core/rtw_ieee80211.c index 863ddf846218..e60ed3359866 100644 --- a/drivers/staging/rtl8723bs/core/rtw_ieee80211.c +++ b/drivers/staging/rtl8723bs/core/rtw_ieee80211.c @@ -371,6 +371,9 @@ unsigned char *rtw_get_wpa_ie(unsigned char *pie, int *wpa_ie_len, int limit) pbuf = rtw_get_ie(pbuf, WLAN_EID_VENDOR_SPECIFIC, &len, limit_new); if (pbuf) { + if (len < 6) + goto check_next_ie; + /* check if oui matches... */ if (memcmp((pbuf + 2), wpa_oui_type, sizeof(wpa_oui_type))) goto check_next_ie; diff --git a/drivers/staging/rtl8723bs/core/rtw_mlme_ext.c b/drivers/staging/rtl8723bs/core/rtw_mlme_ext.c index a443b3530fb9..a9382dc1294b 100644 --- a/drivers/staging/rtl8723bs/core/rtw_mlme_ext.c +++ b/drivers/staging/rtl8723bs/core/rtw_mlme_ext.c @@ -879,7 +879,7 @@ unsigned int OnAuthClient(struct adapter *padapter, union recv_frame *precv_fram p = rtw_get_ie(pframe + WLAN_HDR_A3_LEN + _AUTH_IE_OFFSET_, WLAN_EID_CHALLENGE, (int *)&len, pkt_len - WLAN_HDR_A3_LEN - _AUTH_IE_OFFSET_); - if (!p) + if (!p || len != WLAN_AUTH_CHALLENGE_LEN) goto authclnt_fail; memcpy(pmlmeinfo->chg_txt, p + 2, len); diff --git a/drivers/staging/rtl8723bs/core/rtw_wlan_util.c b/drivers/staging/rtl8723bs/core/rtw_wlan_util.c index a4de538722b5..c614c0b4c792 100644 --- a/drivers/staging/rtl8723bs/core/rtw_wlan_util.c +++ b/drivers/staging/rtl8723bs/core/rtw_wlan_util.c @@ -666,6 +666,9 @@ int WMM_param_handler(struct adapter *padapter, struct ndis_80211_var_ie *pIE) return false; } + if (pIE->length != WLAN_WMM_LEN) + return false; + if (!memcmp(&(pmlmeinfo->WMM_param), (pIE->data + 6), sizeof(struct WMM_para_element))) return false; diff --git a/drivers/staging/rtl8723bs/os_dep/ioctl_cfg80211.c b/drivers/staging/rtl8723bs/os_dep/ioctl_cfg80211.c index 967cd1b34aed..9fa3131feb66 100644 --- a/drivers/staging/rtl8723bs/os_dep/ioctl_cfg80211.c +++ b/drivers/staging/rtl8723bs/os_dep/ioctl_cfg80211.c @@ -2033,6 +2033,8 @@ static netdev_tx_t rtw_cfg80211_monitor_if_xmit_entry(struct sk_buff *skb, struc /* Skip the ratio tap header */ skb_pull(skb, rtap_len); + if (skb->len < dot11_hdr_len) + goto fail; dot11_hdr = (struct ieee80211_hdr *)skb->data; frame_control = le16_to_cpu(dot11_hdr->frame_control); @@ -2045,6 +2047,8 @@ static netdev_tx_t rtw_cfg80211_monitor_if_xmit_entry(struct sk_buff *skb, struc qos_len = 2; if ((frame_control & 0x0300) == 0x0300) dot11_hdr_len += 6; + if (skb->len < dot11_hdr_len + qos_len + snap_len) + goto fail; memcpy(dst_mac_addr, dot11_hdr->addr1, sizeof(dst_mac_addr)); memcpy(src_mac_addr, dot11_hdr->addr2, sizeof(src_mac_addr)); diff --git a/drivers/thermal/thermal_hwmon.c b/drivers/thermal/thermal_hwmon.c index 386dfb9f559e..ec73d03a1e60 100644 --- a/drivers/thermal/thermal_hwmon.c +++ b/drivers/thermal/thermal_hwmon.c @@ -19,19 +19,28 @@ #include "thermal_hwmon.h" #include "thermal_core.h" -/* - * Needs to be large enough to hold a thermal zone type string followed by an - * underline character and a 32-bit integer in decimal representation. - */ -#define THERMAL_HWMON_NAME_LENGTH (THERMAL_NAME_LENGTH + 11) - /* hwmon sys I/F */ /* thermal zone devices with the same type share one hwmon device */ struct thermal_hwmon_device { - char name[THERMAL_HWMON_NAME_LENGTH]; + char type[THERMAL_NAME_LENGTH]; struct device *device; + int count; + struct list_head tz_list; struct list_head node; +}; + +struct thermal_hwmon_attr { + struct device_attribute attr; + char name[16]; +}; + +/* one temperature input for each thermal zone */ +struct thermal_hwmon_temp { + struct list_head hwmon_node; struct thermal_zone_device *tz; + struct thermal_hwmon_attr temp_input; /* hwmon sys attr */ + struct thermal_hwmon_attr temp_crit; /* hwmon sys attr */ + bool temp_crit_present; }; static LIST_HEAD(thermal_hwmon_list); @@ -39,14 +48,19 @@ static LIST_HEAD(thermal_hwmon_list); static DEFINE_MUTEX(thermal_hwmon_list_lock); static ssize_t -temp1_input_show(struct device *dev, struct device_attribute *attr, char *buf) +temp_input_show(struct device *dev, struct device_attribute *attr, char *buf) { - struct thermal_hwmon_device *hwmon = dev_get_drvdata(dev); - struct thermal_zone_device *tz = hwmon->tz; int temperature; int ret; + struct thermal_hwmon_attr *hwmon_attr + = container_of(attr, struct thermal_hwmon_attr, attr); + struct thermal_hwmon_temp *temp + = container_of(hwmon_attr, struct thermal_hwmon_temp, + temp_input); + struct thermal_zone_device *tz = temp->tz; ret = thermal_zone_get_temp(tz, &temperature); + if (ret) return ret; @@ -54,10 +68,14 @@ temp1_input_show(struct device *dev, struct device_attribute *attr, char *buf) } static ssize_t -temp1_crit_show(struct device *dev, struct device_attribute *attr, char *buf) +temp_crit_show(struct device *dev, struct device_attribute *attr, char *buf) { - struct thermal_hwmon_device *hwmon = dev_get_drvdata(dev); - struct thermal_zone_device *tz = hwmon->tz; + struct thermal_hwmon_attr *hwmon_attr + = container_of(attr, struct thermal_hwmon_attr, attr); + struct thermal_hwmon_temp *temp + = container_of(hwmon_attr, struct thermal_hwmon_temp, + temp_crit); + struct thermal_zone_device *tz = temp->tz; int temperature; int ret; @@ -70,98 +88,170 @@ temp1_crit_show(struct device *dev, struct device_attribute *attr, char *buf) return sysfs_emit(buf, "%d\n", temperature); } -static DEVICE_ATTR_RO(temp1_input); -static DEVICE_ATTR_RO(temp1_crit); -static struct attribute *thermal_hwmon_attrs[] = { - &dev_attr_temp1_input.attr, - &dev_attr_temp1_crit.attr, - NULL, -}; - -static umode_t thermal_hwmon_attr_is_visible(struct kobject *kobj, - struct attribute *a, int n) +static struct thermal_hwmon_device * +thermal_hwmon_lookup_by_type(const struct thermal_zone_device *tz) { - if (a == &dev_attr_temp1_input.attr) - return a->mode; + struct thermal_hwmon_device *hwmon; + char type[THERMAL_NAME_LENGTH]; - if (a == &dev_attr_temp1_crit.attr) { - struct thermal_hwmon_device *hwmon = dev_get_drvdata(kobj_to_dev(kobj)); - struct thermal_zone_device *tz = hwmon->tz; - int dummy; - - if (tz->ops.get_crit_temp && !tz->ops.get_crit_temp(tz, &dummy)) - return a->mode; + mutex_lock(&thermal_hwmon_list_lock); + list_for_each_entry(hwmon, &thermal_hwmon_list, node) { + strscpy(type, tz->type); + strreplace(type, '-', '_'); + if (!strcmp(hwmon->type, type)) { + mutex_unlock(&thermal_hwmon_list_lock); + return hwmon; + } } + mutex_unlock(&thermal_hwmon_list_lock); - return 0; + return NULL; } -static const struct attribute_group thermal_hwmon_group = { - .attrs = thermal_hwmon_attrs, - .is_visible = thermal_hwmon_attr_is_visible, -}; +/* Find the temperature input matching a given thermal zone */ +static struct thermal_hwmon_temp * +thermal_hwmon_lookup_temp(const struct thermal_hwmon_device *hwmon, + const struct thermal_zone_device *tz) +{ + struct thermal_hwmon_temp *temp; -__ATTRIBUTE_GROUPS(thermal_hwmon); + mutex_lock(&thermal_hwmon_list_lock); + list_for_each_entry(temp, &hwmon->tz_list, hwmon_node) + if (temp->tz == tz) { + mutex_unlock(&thermal_hwmon_list_lock); + return temp; + } + mutex_unlock(&thermal_hwmon_list_lock); + + return NULL; +} + +static bool thermal_zone_crit_temp_valid(struct thermal_zone_device *tz) +{ + int temp; + return tz->ops.get_crit_temp && !tz->ops.get_crit_temp(tz, &temp); +} int thermal_add_hwmon_sysfs(struct thermal_zone_device *tz) { struct thermal_hwmon_device *hwmon; + struct thermal_hwmon_temp *temp; + int new_hwmon_device = 1; + int result; + + hwmon = thermal_hwmon_lookup_by_type(tz); + if (hwmon) { + new_hwmon_device = 0; + goto register_sys_interface; + } hwmon = kzalloc_obj(*hwmon); if (!hwmon) return -ENOMEM; - hwmon->tz = tz; - /* - * Append the thermal zone ID preceded by an underline character to the - * type to disambiguate the sensors command output. - */ - scnprintf(hwmon->name, THERMAL_HWMON_NAME_LENGTH, "%s_%d", tz->type, tz->id); - strreplace(hwmon->name, '-', '_'); + INIT_LIST_HEAD(&hwmon->tz_list); + strscpy(hwmon->type, tz->type, THERMAL_NAME_LENGTH); + strreplace(hwmon->type, '-', '_'); hwmon->device = hwmon_device_register_for_thermal(&tz->device, - hwmon->name, hwmon, - thermal_hwmon_groups); + hwmon->type, hwmon); if (IS_ERR(hwmon->device)) { - int result = PTR_ERR(hwmon->device); - - kfree(hwmon); - return result; + result = PTR_ERR(hwmon->device); + goto free_mem; + } + + register_sys_interface: + temp = kzalloc_obj(*temp); + if (!temp) { + result = -ENOMEM; + goto unregister_name; + } + + temp->tz = tz; + hwmon->count++; + + snprintf(temp->temp_input.name, sizeof(temp->temp_input.name), + "temp%d_input", hwmon->count); + temp->temp_input.attr.attr.name = temp->temp_input.name; + temp->temp_input.attr.attr.mode = 0444; + temp->temp_input.attr.show = temp_input_show; + sysfs_attr_init(&temp->temp_input.attr.attr); + result = device_create_file(hwmon->device, &temp->temp_input.attr); + if (result) + goto free_temp_mem; + + if (thermal_zone_crit_temp_valid(tz)) { + snprintf(temp->temp_crit.name, + sizeof(temp->temp_crit.name), + "temp%d_crit", hwmon->count); + temp->temp_crit.attr.attr.name = temp->temp_crit.name; + temp->temp_crit.attr.attr.mode = 0444; + temp->temp_crit.attr.show = temp_crit_show; + sysfs_attr_init(&temp->temp_crit.attr.attr); + result = device_create_file(hwmon->device, + &temp->temp_crit.attr); + if (result) + goto unregister_input; + + temp->temp_crit_present = true; } - /* The list is needed for hwmon lookup during removal. */ mutex_lock(&thermal_hwmon_list_lock); - list_add_tail(&hwmon->node, &thermal_hwmon_list); + if (new_hwmon_device) + list_add_tail(&hwmon->node, &thermal_hwmon_list); + list_add_tail(&temp->hwmon_node, &hwmon->tz_list); mutex_unlock(&thermal_hwmon_list_lock); return 0; + + unregister_input: + device_remove_file(hwmon->device, &temp->temp_input.attr); + free_temp_mem: + kfree(temp); + unregister_name: + if (new_hwmon_device) + hwmon_device_unregister(hwmon->device); + free_mem: + if (new_hwmon_device) + kfree(hwmon); + + return result; } EXPORT_SYMBOL_GPL(thermal_add_hwmon_sysfs); -static struct thermal_hwmon_device * -thermal_hwmon_lookup(const struct thermal_zone_device *tz) -{ - struct thermal_hwmon_device *hwmon; - - list_for_each_entry(hwmon, &thermal_hwmon_list, node) { - if (hwmon->tz == tz) - return hwmon; - } - return NULL; -} - void thermal_remove_hwmon_sysfs(struct thermal_zone_device *tz) { struct thermal_hwmon_device *hwmon; + struct thermal_hwmon_temp *temp; - scoped_guard(mutex, &thermal_hwmon_list_lock) { - hwmon = thermal_hwmon_lookup(tz); - if (!hwmon) - return; - - list_del(&hwmon->node); + hwmon = thermal_hwmon_lookup_by_type(tz); + if (unlikely(!hwmon)) { + /* Should never happen... */ + dev_dbg(&tz->device, "hwmon device lookup failed!\n"); + return; } + temp = thermal_hwmon_lookup_temp(hwmon, tz); + if (unlikely(!temp)) { + /* Should never happen... */ + dev_dbg(&tz->device, "temperature input lookup failed!\n"); + return; + } + + device_remove_file(hwmon->device, &temp->temp_input.attr); + if (temp->temp_crit_present) + device_remove_file(hwmon->device, &temp->temp_crit.attr); + + mutex_lock(&thermal_hwmon_list_lock); + list_del(&temp->hwmon_node); + kfree(temp); + if (!list_empty(&hwmon->tz_list)) { + mutex_unlock(&thermal_hwmon_list_lock); + return; + } + list_del(&hwmon->node); + mutex_unlock(&thermal_hwmon_list_lock); + hwmon_device_unregister(hwmon->device); kfree(hwmon); } diff --git a/drivers/thunderbolt/eeprom.c b/drivers/thunderbolt/eeprom.c index 5681c17f82ec..2a13fa6888ba 100644 --- a/drivers/thunderbolt/eeprom.c +++ b/drivers/thunderbolt/eeprom.c @@ -394,9 +394,16 @@ static int tb_drom_parse_entry_port(struct tb_switch *sw, return -EIO; } port->link_nr = entry->link_nr; - if (entry->has_dual_link_port) + if (entry->has_dual_link_port) { + if (entry->dual_link_port_nr > sw->config.max_port_number) { + tb_sw_warn(sw, + "port entry has invalid dual link port number %u\n", + entry->dual_link_port_nr); + return -EIO; + } port->dual_link_port = &port->sw->ports[entry->dual_link_port_nr]; + } } return 0; } diff --git a/drivers/thunderbolt/icm.c b/drivers/thunderbolt/icm.c index 10fefac3b1d9..669807f0eaf8 100644 --- a/drivers/thunderbolt/icm.c +++ b/drivers/thunderbolt/icm.c @@ -2341,7 +2341,7 @@ static int icm_usb4_switch_op(struct tb_switch *sw, u16 opcode, u32 *metadata, if (tx_data_len) { request.data_len_valid |= ICM_USB4_SWITCH_DATA_VALID; if (tx_data_len < ARRAY_SIZE(request.data)) - request.data_len_valid = + request.data_len_valid |= tx_data_len & ICM_USB4_SWITCH_DATA_LEN_MASK; memcpy(request.data, tx_data, tx_data_len * sizeof(u32)); } diff --git a/drivers/thunderbolt/nhi.c b/drivers/thunderbolt/nhi.c index 0f795ea58756..35e3c119d5ee 100644 --- a/drivers/thunderbolt/nhi.c +++ b/drivers/thunderbolt/nhi.c @@ -1226,6 +1226,8 @@ int nhi_probe(struct tb_nhi *nhi) return dev_err_probe(dev, res, "NHI specific init failed\n"); } + init_completion(&nhi->domain_released); + tb = nhi_select_cm(nhi); if (!tb) return dev_err_probe(dev, -ENODEV, @@ -1233,8 +1235,6 @@ int nhi_probe(struct tb_nhi *nhi) dev_dbg(dev, "NHI initialized, starting thunderbolt\n"); - init_completion(&nhi->domain_released); - res = tb_domain_add(tb, host_reset); if (res) { /* diff --git a/drivers/thunderbolt/stream.c b/drivers/thunderbolt/stream.c index c1f5c55583d0..4cc86d8d6491 100644 --- a/drivers/thunderbolt/stream.c +++ b/drivers/thunderbolt/stream.c @@ -257,7 +257,7 @@ static void tbstream_ring_free(struct tbstream_ring *ring) if (sf->frame.buffer_phy) dma_unmap_page(dma_dev, sf->frame.buffer_phy, - tb_ring_frame_size(&sf->frame), dir); + TB_MAX_FRAME_SIZE, dir); sf->frame.buffer_phy = 0; if (sf->page) __free_page(sf->page); diff --git a/drivers/thunderbolt/tb.c b/drivers/thunderbolt/tb.c index 76323255439a..f43f2d952372 100644 --- a/drivers/thunderbolt/tb.c +++ b/drivers/thunderbolt/tb.c @@ -609,7 +609,7 @@ static int tb_consumed_dp_bandwidth(struct tb *tb, int *consumed_up, int *consumed_down) { - int group_reserved[MAX_GROUPS] = {}; + int group_reserved[MAX_GROUPS + 1] = {}; struct tb_cm *tcm = tb_priv(tb); struct tb_tunnel *tunnel; bool downstream; diff --git a/drivers/tty/serial/8250/8250_dma.c b/drivers/tty/serial/8250/8250_dma.c index 3b6452e759d5..5a83e5269b41 100644 --- a/drivers/tty/serial/8250/8250_dma.c +++ b/drivers/tty/serial/8250/8250_dma.c @@ -211,11 +211,12 @@ void serial8250_rx_dma_flush(struct uart_8250_port *p) { struct uart_8250_dma *dma = p->dma; - if (dma->rx_running) { - dmaengine_pause(dma->rxchan); - __dma_rx_complete(p); - dmaengine_terminate_async(dma->rxchan); - } + if (!dma || !dma->rxchan || !dma->rx_running) + return; + + dmaengine_pause(dma->rxchan); + __dma_rx_complete(p); + dmaengine_terminate_async(dma->rxchan); } EXPORT_SYMBOL_GPL(serial8250_rx_dma_flush); @@ -324,6 +325,7 @@ void serial8250_release_dma(struct uart_8250_port *p) /* Release RX resources */ dmaengine_terminate_sync(dma->rxchan); + dma->rx_running = 0; dma_free_coherent(dma->rxchan->device->dev, dma->rx_size, dma->rx_buf, dma->rx_addr); dma_release_channel(dma->rxchan); diff --git a/drivers/tty/serial/8250/8250_of.c b/drivers/tty/serial/8250/8250_of.c index 81644d40b09a..160f4c71de06 100644 --- a/drivers/tty/serial/8250/8250_of.c +++ b/drivers/tty/serial/8250/8250_of.c @@ -81,6 +81,40 @@ static int of_platform_serial_clk_notifier_cb(struct notifier_block *nb, unsigne return NOTIFY_DONE; } +static int lpc32xx_handle_irq(struct uart_port *port) +{ + struct uart_8250_port *up = up_to_u8250p(port); + unsigned int iir; + u16 status; + + guard(serial8250_rpm)(up); + + iir = serial_port_in(port, UART_IIR); + if (iir & UART_IIR_NO_INT) + return 0; + + guard(uart_port_lock_check_sysrq_irqsave)(port); + + /* + * The LPC32xx UART can assert an RX character-timeout interrupt while + * the RX FIFO is empty: IIR reports UART_IIR_RX_TIMEOUT but LSR.DR is + * clear. The timeout is only cleared by reading RHR, but the core RX + * path skips that read when the FIFO is empty, so the level-triggered + * IRQ re-fires forever and livelocks this single-core SoC. Do one + * throwaway RHR read to clear it; a healthy UART never reports a + * timeout with DR/BI clear, so no received data is ever discarded. + */ + if ((iir & 0x3f) == UART_IIR_RX_TIMEOUT) { + status = serial_lsr_in(up); + if (!(status & (UART_LSR_DR | UART_LSR_BI))) + serial_port_in(port, UART_RX); + } + + serial8250_handle_irq_locked(port, iir); + + return 1; +} + /* * Fill a struct uart_port for a given device node */ @@ -173,6 +207,9 @@ static int of_platform_serial_setup(struct platform_device *ofdev, case PORT_NPCM: ret = npcm_setup(port); break; + case PORT_LPC3220: + port->handle_irq = lpc32xx_handle_irq; + break; default: /* Nothing to do */ ret = 0; @@ -369,6 +406,7 @@ static struct platform_driver of_platform_serial_driver = { module_platform_driver(of_platform_serial_driver); +MODULE_IMPORT_NS("SERIAL_8250"); MODULE_AUTHOR("Arnd Bergmann "); MODULE_LICENSE("GPL"); MODULE_DESCRIPTION("Serial Port driver for Open Firmware platform devices"); diff --git a/drivers/tty/serial/amba-pl011.c b/drivers/tty/serial/amba-pl011.c index 8ed91e1da22b..9abaeecd05fc 100644 --- a/drivers/tty/serial/amba-pl011.c +++ b/drivers/tty/serial/amba-pl011.c @@ -309,6 +309,7 @@ enum pl011_rs485_tx_state { WAIT_AFTER_RTS, SEND, WAIT_AFTER_SEND, + WAIT_AFTER_SEND_DELAY, }; /* @@ -1246,7 +1247,7 @@ static void pl011_dma_shutdown(struct uart_amba_port *uap) if (uap->using_tx_dma) { /* In theory, this should already be done by pl011_dma_flush_buffer */ - dmaengine_terminate_all(uap->dmatx.chan); + dmaengine_terminate_sync(uap->dmatx.chan); if (uap->dmatx.queued) { dma_unmap_single(uap->dmatx.chan->device->dev, uap->dmatx.dma, uap->dmatx.len, @@ -1259,12 +1260,12 @@ static void pl011_dma_shutdown(struct uart_amba_port *uap) } if (uap->using_rx_dma) { - dmaengine_terminate_all(uap->dmarx.chan); + if (uap->dmarx.poll_rate) + timer_delete_sync(&uap->dmarx.timer); + dmaengine_terminate_sync(uap->dmarx.chan); /* Clean up the RX DMA */ pl011_dmabuf_free(uap->dmarx.chan, &uap->dmarx.dbuf_a, DMA_FROM_DEVICE); pl011_dmabuf_free(uap->dmarx.chan, &uap->dmarx.dbuf_b, DMA_FROM_DEVICE); - if (uap->dmarx.poll_rate) - timer_delete_sync(&uap->dmarx.timer); uap->using_rx_dma = false; } } @@ -1333,33 +1334,11 @@ static inline bool pl011_dma_rx_running(struct uart_amba_port *uap) #define pl011_dma_flush_buffer NULL #endif -static void pl011_rs485_tx_stop(struct uart_amba_port *uap) +static void pl011_rs485_tx_stop_now(struct uart_amba_port *uap) { struct uart_port *port = &uap->port; u32 cr; - if (uap->rs485_tx_state == SEND) - uap->rs485_tx_state = WAIT_AFTER_SEND; - - if (uap->rs485_tx_state == WAIT_AFTER_SEND) { - /* Schedule hrtimer if tx queue not empty */ - if (!pl011_tx_empty(port)) { - hrtimer_start(&uap->trigger_stop_tx, - uap->rs485_tx_drain_interval, - HRTIMER_MODE_REL); - return; - } - if (port->rs485.delay_rts_after_send > 0) { - hrtimer_start(&uap->trigger_stop_tx, - ms_to_ktime(port->rs485.delay_rts_after_send), - HRTIMER_MODE_REL); - return; - } - /* Continue without any delay */ - } else if (uap->rs485_tx_state == WAIT_AFTER_RTS) { - hrtimer_try_to_cancel(&uap->trigger_start_tx); - } - cr = pl011_read(uap, REG_CR); if (port->rs485.flags & SER_RS485_RTS_AFTER_SEND) @@ -1375,6 +1354,36 @@ static void pl011_rs485_tx_stop(struct uart_amba_port *uap) uap->rs485_tx_state = OFF; } +static void pl011_rs485_tx_stop(struct uart_amba_port *uap) +{ + struct uart_port *port = &uap->port; + + if (uap->rs485_tx_state == SEND) + uap->rs485_tx_state = WAIT_AFTER_SEND; + + if (uap->rs485_tx_state == WAIT_AFTER_SEND) { + /* Schedule hrtimer if tx queue not empty */ + if (!pl011_tx_empty(port)) { + hrtimer_start(&uap->trigger_stop_tx, + uap->rs485_tx_drain_interval, + HRTIMER_MODE_REL); + return; + } + if (port->rs485.delay_rts_after_send > 0) { + uap->rs485_tx_state = WAIT_AFTER_SEND_DELAY; + hrtimer_start(&uap->trigger_stop_tx, + ms_to_ktime(port->rs485.delay_rts_after_send), + HRTIMER_MODE_REL); + return; + } + /* Continue without any delay */ + } else if (uap->rs485_tx_state == WAIT_AFTER_RTS) { + hrtimer_try_to_cancel(&uap->trigger_start_tx); + } + + pl011_rs485_tx_stop_now(uap); +} + static void pl011_stop_tx(struct uart_port *port) { struct uart_amba_port *uap = @@ -1415,7 +1424,8 @@ static void pl011_rs485_tx_start(struct uart_amba_port *uap) uap->rs485_tx_state = SEND; return; } - if (uap->rs485_tx_state == WAIT_AFTER_SEND) { + if (uap->rs485_tx_state == WAIT_AFTER_SEND || + uap->rs485_tx_state == WAIT_AFTER_SEND_DELAY) { hrtimer_try_to_cancel(&uap->trigger_stop_tx); uap->rs485_tx_state = SEND; return; @@ -1482,7 +1492,8 @@ static enum hrtimer_restart pl011_trigger_stop_tx(struct hrtimer *t) unsigned long flags; uart_port_lock_irqsave(&uap->port, &flags); - if (uap->rs485_tx_state == WAIT_AFTER_SEND) + if (uap->rs485_tx_state == WAIT_AFTER_SEND || + uap->rs485_tx_state == WAIT_AFTER_SEND_DELAY) pl011_rs485_tx_stop(uap); uart_port_unlock_irqrestore(&uap->port, flags); @@ -2080,11 +2091,20 @@ static void pl011_shutdown(struct uart_port *port) pl011_dma_shutdown(uap); - if ((port->rs485.flags & SER_RS485_ENABLED && uap->rs485_tx_state != OFF)) - pl011_rs485_tx_stop(uap); - free_irq(uap->port.irq, uap); + /* + * free_irq() drains the UART interrupt handler, which can arm either + * timer. Cancel the timers afterwards to drain their callbacks too. + */ + hrtimer_cancel(&uap->trigger_start_tx); + hrtimer_cancel(&uap->trigger_stop_tx); + + uart_port_lock_irq(port); + if (uap->rs485_tx_state != OFF) + pl011_rs485_tx_stop_now(uap); + uart_port_unlock_irq(port); + pl011_disable_uart(uap); /* @@ -3063,6 +3083,8 @@ static void pl011_remove(struct amba_device *dev) struct uart_amba_port *uap = amba_get_drvdata(dev); uart_remove_one_port(&amba_reg, &uap->port); + hrtimer_cancel(&uap->trigger_start_tx); + hrtimer_cancel(&uap->trigger_stop_tx); pl011_unregister_port(uap); } diff --git a/drivers/tty/serial/qcom_geni_serial.c b/drivers/tty/serial/qcom_geni_serial.c index 7ead87b4eb65..1e39122ec09f 100644 --- a/drivers/tty/serial/qcom_geni_serial.c +++ b/drivers/tty/serial/qcom_geni_serial.c @@ -158,6 +158,7 @@ static const struct uart_ops qcom_geni_uart_pops; static struct uart_driver qcom_geni_console_driver; static struct uart_driver qcom_geni_uart_driver; +static void qcom_geni_serial_stop_tx_dma(struct uart_port *uport); static void __qcom_geni_serial_cancel_tx_cmd(struct uart_port *uport); static void qcom_geni_serial_cancel_tx_cmd(struct uart_port *uport); static int qcom_geni_serial_port_setup(struct uart_port *uport); @@ -636,35 +637,34 @@ static unsigned int qcom_geni_serial_tx_empty(struct uart_port *uport) return !readl(uport->membase + SE_GENI_TX_FIFO_STATUS); } +static void qcom_geni_serial_flush_buffer_dma(struct uart_port *uport) +{ + struct qcom_geni_serial_port *port = to_dev_port(uport); + + qcom_geni_serial_stop_tx_dma(uport); + port->tx_remaining = 0; + port->tx_queued = 0; +} + static void qcom_geni_serial_stop_tx_dma(struct uart_port *uport) { struct qcom_geni_serial_port *port = to_dev_port(uport); - bool done; - if (!qcom_geni_serial_main_active(uport)) - return; + if (qcom_geni_serial_main_active(uport)) + __qcom_geni_serial_cancel_tx_cmd(uport); if (port->tx_dma_addr) { + writel(1, uport->membase + SE_DMA_TX_FSM_RST); + if (!qcom_geni_serial_poll_bit(uport, SE_DMA_TX_IRQ_STAT, + TX_RESET_DONE, true)) + dev_err_ratelimited(uport->dev, "TX DMA reset failed"); + writel(TX_RESET_DONE | TX_DMA_DONE, + uport->membase + SE_DMA_TX_IRQ_CLR); + geni_se_tx_dma_unprep(&port->se, port->tx_dma_addr, port->tx_remaining); port->tx_dma_addr = 0; - port->tx_remaining = 0; } - - geni_se_cancel_m_cmd(&port->se); - - done = qcom_geni_serial_poll_bit(uport, SE_GENI_M_IRQ_STATUS, - M_CMD_CANCEL_EN, true); - if (!done) { - geni_se_abort_m_cmd(&port->se); - done = qcom_geni_serial_poll_bit(uport, SE_GENI_M_IRQ_STATUS, - M_CMD_ABORT_EN, true); - if (!done) - dev_err_ratelimited(uport->dev, "M_CMD_ABORT_EN not set"); - writel(M_CMD_ABORT_EN, uport->membase + SE_GENI_M_IRQ_CLEAR); - } - - writel(M_CMD_CANCEL_EN, uport->membase + SE_GENI_M_IRQ_CLEAR); } static void qcom_geni_serial_start_tx_dma(struct uart_port *uport) @@ -1180,7 +1180,7 @@ static void qcom_geni_serial_shutdown(struct uart_port *uport) uart_port_unlock_irq(uport); } -static void qcom_geni_serial_flush_buffer(struct uart_port *uport) +static void qcom_geni_serial_flush_buffer_fifo(struct uart_port *uport) { qcom_geni_serial_cancel_tx_cmd(uport); } @@ -1769,7 +1769,7 @@ static const struct uart_ops qcom_geni_console_pops = { .request_port = qcom_geni_serial_request_port, .config_port = qcom_geni_serial_config_port, .shutdown = qcom_geni_serial_shutdown, - .flush_buffer = qcom_geni_serial_flush_buffer, + .flush_buffer = qcom_geni_serial_flush_buffer_fifo, .type = qcom_geni_serial_get_type, .set_mctrl = qcom_geni_serial_set_mctrl, .get_mctrl = qcom_geni_serial_get_mctrl, @@ -1792,6 +1792,7 @@ static const struct uart_ops qcom_geni_uart_pops = { .request_port = qcom_geni_serial_request_port, .config_port = qcom_geni_serial_config_port, .shutdown = qcom_geni_serial_shutdown, + .flush_buffer = qcom_geni_serial_flush_buffer_dma, .type = qcom_geni_serial_get_type, .set_mctrl = qcom_geni_serial_set_mctrl, .get_mctrl = qcom_geni_serial_get_mctrl, diff --git a/drivers/tty/serial/sc16is7xx.c b/drivers/tty/serial/sc16is7xx.c index 6c97953d593b..9b152ead050f 100644 --- a/drivers/tty/serial/sc16is7xx.c +++ b/drivers/tty/serial/sc16is7xx.c @@ -827,6 +827,9 @@ static void sc16is7xx_tx_proc(struct kthread_work *ws) msleep(port->rs485.delay_rts_before_send); guard(mutex)(&one->lock); + sc16is7xx_port_update(port, SC16IS7XX_IER_REG, + SC16IS7XX_IER_THRI_BIT, + SC16IS7XX_IER_THRI_BIT); sc16is7xx_handle_tx(port); } diff --git a/drivers/tty/vt/keyboard.c b/drivers/tty/vt/keyboard.c index 763a3f1b7be0..c41d850b29c6 100644 --- a/drivers/tty/vt/keyboard.c +++ b/drivers/tty/vt/keyboard.c @@ -1437,7 +1437,7 @@ static void kbd_keycode(unsigned int keycode, int down, bool hw_raw) struct keyboard_notifier_param param = { .vc = vc, .value = keycode, .down = down }; int rc; - tty = vc->port.tty; + tty = tty_port_tty_get(&vc->port); if (tty && (!tty->driver_data)) { /* No driver data? Strange. Okay we fix it then. */ @@ -1497,9 +1497,12 @@ static void kbd_keycode(unsigned int keycode, int down, bool hw_raw) * characters get aren't echoed locally. This makes key repeat * usable with slow applications and under heavy loads. */ + tty_kref_put(tty); return; } + tty_kref_put(tty); + param.shift = shift_final = (shift_state | kbd->slockstate) ^ kbd->lockstate; param.ledstate = kbd->ledflagstate; key_map = key_maps[shift_final]; diff --git a/drivers/tty/vt/vt_ioctl.c b/drivers/tty/vt/vt_ioctl.c index 28993a3d0acb..deb3b3f93461 100644 --- a/drivers/tty/vt/vt_ioctl.c +++ b/drivers/tty/vt/vt_ioctl.c @@ -406,6 +406,8 @@ static int vt_k_ioctl(struct tty_struct *tty, unsigned int cmd, /* this could be folded into KDSKBMODE, but for compatibility reasons it is not so easy to fold KDGKBMETA into KDGKBMODE */ case KDSKBMETA: + if (!perm) + return -EPERM; return vt_do_kdskbmeta(console, arg); case KDGKBMETA: diff --git a/drivers/usb/atm/cxacru.c b/drivers/usb/atm/cxacru.c index f1900c567ba4..429ac20a8999 100644 --- a/drivers/usb/atm/cxacru.c +++ b/drivers/usb/atm/cxacru.c @@ -700,6 +700,8 @@ static int cxacru_cm(struct cxacru_data *instance, enum cxacru_cm_request cm, ret = offd; usb_dbg(instance->usbatm, "cm %#x\n", cm); fail: + if (ret < 0) + usb_kill_urb(instance->rcv_urb); mutex_unlock(&instance->cm_serialize); err: return ret; diff --git a/drivers/usb/cdns3/cdnsp-gadget.c b/drivers/usb/cdns3/cdnsp-gadget.c index a5275c2fb43b..7a516e509198 100644 --- a/drivers/usb/cdns3/cdnsp-gadget.c +++ b/drivers/usb/cdns3/cdnsp-gadget.c @@ -163,9 +163,9 @@ static void cdnsp_set_apb_timeout_value(struct cdnsp_device *pdev) offset = cdnsp_find_next_ext_cap(base, offset, D_XEC_PRE_REGS_CAP); reg = base + offset + REG_CHICKEN_BITS_3_OFFSET; - val = le32_to_cpu(readl(reg)); + val = readl(reg); val = CHICKEN_APB_TIMEOUT_SET(val, cdns->override_apb_timeout); - writel(cpu_to_le32(val), reg); + writel(val, reg); } static void cdnsp_set_chicken_bits_2(struct cdnsp_device *pdev, u32 bit) diff --git a/drivers/usb/core/config.c b/drivers/usb/core/config.c index 45e20c6d76c0..346a2faa9bb8 100644 --- a/drivers/usb/core/config.c +++ b/drivers/usb/core/config.c @@ -912,6 +912,18 @@ int usb_get_configuration(struct usb_device *dev) unsigned char *bigbuffer; struct usb_config_descriptor *desc; int result; + size_t usb_config_req_size; + + /* + * We usually start by grabbing the first 9-bytes descriptor so we know + * how long the whole configuration is. Some devices with quirky + * firmware will fail enumeration, so if the quirk is set, use 255 instead, + * mirroring the behavior of Windows. + */ + if (dev->quirks & USB_QUIRK_WINDOWS_CONFIG_REQ_SIZE) + usb_config_req_size = 255; + else + usb_config_req_size = USB_DT_CONFIG_SIZE; if (ncfg > USB_MAXCONFIG) { dev_notice(ddev, "too many configurations: %d, " @@ -938,15 +950,13 @@ int usb_get_configuration(struct usb_device *dev) if (!dev->rawdescriptors) return -ENOMEM; - desc = kmalloc(USB_DT_CONFIG_SIZE, GFP_KERNEL); + desc = kmalloc(usb_config_req_size, GFP_KERNEL); if (!desc) return -ENOMEM; for (cfgno = 0; cfgno < ncfg; cfgno++) { - /* We grab just the first descriptor so we know how long - * the whole configuration is */ result = usb_get_descriptor(dev, USB_DT_CONFIG, cfgno, - desc, USB_DT_CONFIG_SIZE); + desc, usb_config_req_size); if (result < 0) { dev_err(ddev, "unable to read config index %d " "descriptor/%s: %d\n", cfgno, "start", result); @@ -956,16 +966,14 @@ int usb_get_configuration(struct usb_device *dev) dev->descriptor.bNumConfigurations = cfgno; break; } else if (result < 4) { - dev_err(ddev, "config index %d descriptor too short " - "(expected %i, got %i)\n", cfgno, - USB_DT_CONFIG_SIZE, result); + dev_err(ddev, "config index %d descriptor too short (asked for %zu, got %i)\n", + cfgno, usb_config_req_size, result); result = -EINVAL; goto err; } length = max_t(int, le16_to_cpu(desc->wTotalLength), USB_DT_CONFIG_SIZE); - /* Now that we know the length, get the whole thing */ bigbuffer = kmalloc(length, GFP_KERNEL); if (!bigbuffer) { result = -ENOMEM; @@ -975,6 +983,13 @@ int usb_get_configuration(struct usb_device *dev) if (dev->quirks & USB_QUIRK_DELAY_INIT) msleep(200); + /* Skip the second read if we already got everything */ + if (result >= length) { + memcpy(bigbuffer, desc, length); + goto store_and_parse; + } + + /* Get the whole thing */ result = usb_get_descriptor(dev, USB_DT_CONFIG, cfgno, bigbuffer, length); if (result < 0) { @@ -989,6 +1004,7 @@ int usb_get_configuration(struct usb_device *dev) length = result; } +store_and_parse: dev->rawdescriptors[cfgno] = bigbuffer; result = usb_parse_configuration(dev, cfgno, diff --git a/drivers/usb/core/hub.c b/drivers/usb/core/hub.c index 5262e11c12cd..d92bf887739d 100644 --- a/drivers/usb/core/hub.c +++ b/drivers/usb/core/hub.c @@ -2401,7 +2401,7 @@ static void show_string(struct usb_device *udev, char *id, char *string) dev_info(&udev->dev, "%s: %s\n", id, string); } -static void announce_device(struct usb_device *udev) +static void announce_device_ids(struct usb_device *udev) { u16 bcdDevice = le16_to_cpu(udev->descriptor.bcdDevice); @@ -2410,6 +2410,10 @@ static void announce_device(struct usb_device *udev) le16_to_cpu(udev->descriptor.idVendor), le16_to_cpu(udev->descriptor.idProduct), bcdDevice >> 8, bcdDevice & 0xff); +} + +static void announce_device_strings(struct usb_device *udev) +{ dev_info(&udev->dev, "New USB device strings: Mfr=%d, Product=%d, SerialNumber=%d\n", udev->descriptor.iManufacturer, @@ -2420,7 +2424,8 @@ static void announce_device(struct usb_device *udev) show_string(udev, "SerialNumber", udev->serial); } #else -static inline void announce_device(struct usb_device *udev) { } +static inline void announce_device_ids(struct usb_device *udev) { } +static inline void announce_device_strings(struct usb_device *udev) { } #endif @@ -2651,6 +2656,9 @@ int usb_new_device(struct usb_device *udev) device_init_wakeup(&udev->dev, 0); } + /* Announce the device identity */ + announce_device_ids(udev); + /* Tell the runtime-PM framework the device is active */ pm_runtime_set_active(&udev->dev); pm_runtime_get_noresume(&udev->dev); @@ -2672,8 +2680,8 @@ int usb_new_device(struct usb_device *udev) udev->dev.devt = MKDEV(USB_DEVICE_MAJOR, (((udev->bus->busnum-1) * 128) + (udev->devnum-1))); - /* Tell the world! */ - announce_device(udev); + /* Announce the device's product, manufacturer and serial number */ + announce_device_strings(udev); if (udev->serial) add_device_randomness(udev->serial, strlen(udev->serial)); diff --git a/drivers/usb/core/quirks.c b/drivers/usb/core/quirks.c index 87ee2d938bc0..b5b577f0b931 100644 --- a/drivers/usb/core/quirks.c +++ b/drivers/usb/core/quirks.c @@ -142,6 +142,10 @@ static int quirks_param_set(const char *value, const struct kernel_param *kp) break; case 'q': flags |= USB_QUIRK_FORCE_ONE_CONFIG; + break; + case 'r': + flags |= USB_QUIRK_WINDOWS_CONFIG_REQ_SIZE; + break; /* Ignore unrecognized flag characters */ } } @@ -589,6 +593,9 @@ static const struct usb_device_id usb_quirk_list[] = { { USB_DEVICE(0x2386, 0x350e), .driver_info = USB_QUIRK_NO_LPM }, + /* ShanWan Wireless Gamepad */ + { USB_DEVICE(0x2563, 0x0575), .driver_info = USB_QUIRK_WINDOWS_CONFIG_REQ_SIZE }, + /* UGREEN 35871 - BOS descriptor fetch hangs at SuperSpeed Plus */ { USB_DEVICE(0x2b89, 0x5871), .driver_info = USB_QUIRK_NO_BOS }, diff --git a/drivers/usb/gadget/function/f_ncm.c b/drivers/usb/gadget/function/f_ncm.c index 64eabda2f546..bf02545b37a2 100644 --- a/drivers/usb/gadget/function/f_ncm.c +++ b/drivers/usb/gadget/function/f_ncm.c @@ -1171,7 +1171,7 @@ static int ncm_unwrap_ntb(struct gether *port, unsigned char *ntb_ptr = skb->data; __le16 *tmp; unsigned index, index2; - int ndp_index; + unsigned int ndp_index; unsigned dg_len, dg_len2; unsigned ndp_len; unsigned block_len; diff --git a/drivers/usb/host/xhci.h b/drivers/usb/host/xhci.h index 2d3941b5e1e3..708e3ccc5d87 100644 --- a/drivers/usb/host/xhci.h +++ b/drivers/usb/host/xhci.h @@ -190,13 +190,13 @@ struct xhci_op_regs { /* CRCR - Command Ring Control Register - cmd_ring bitmasks */ /* bit 0 - Cycle bit indicates the ownership of the command ring */ -#define CMD_RING_CYCLE BIT(0) +#define CMD_RING_CYCLE BIT_ULL(0) /* stop ring operation after completion of the currently executing command */ -#define CMD_RING_PAUSE BIT(1) +#define CMD_RING_PAUSE BIT_ULL(1) /* stop ring immediately - abort the currently executing command */ -#define CMD_RING_ABORT BIT(2) +#define CMD_RING_ABORT BIT_ULL(2) /* true: command ring is running */ -#define CMD_RING_RUNNING BIT(3) +#define CMD_RING_RUNNING BIT_ULL(3) /* bits 63:6 - Command Ring pointer */ #define CMD_RING_PTR_MASK GENMASK_ULL(63, 6) @@ -271,7 +271,7 @@ struct xhci_intr_reg { * bit 3 - Event Handler Busy (EHB), whether the event ring is scheduled to be serviced by * a work queue (or delayed service routine)? */ -#define ERST_EHB BIT(3) +#define ERST_EHB BIT_ULL(3) /* bits 63:4 - Event Ring Dequeue Pointer */ #define ERST_PTR_MASK GENMASK_ULL(63, 4) @@ -502,7 +502,7 @@ struct xhci_ep_ctx { #define CTX_TO_MAX_ESIT_PAYLOAD(p) (((p) >> 16) & 0xffff) /* deq bitmasks */ -#define EP_CTX_CYCLE_MASK BIT(0) +#define EP_CTX_CYCLE_MASK BIT_ULL(0) /* bits 63:4 - TR Dequeue Pointer */ #define TR_DEQ_PTR_MASK GENMASK_ULL(63, 4) diff --git a/drivers/usb/misc/usbio.c b/drivers/usb/misc/usbio.c index 3c2474dca810..fe093e7760d5 100644 --- a/drivers/usb/misc/usbio.c +++ b/drivers/usb/misc/usbio.c @@ -265,7 +265,7 @@ int usbio_bulk_msg(struct auxiliary_device *adev, u8 type, u8 cmd, bool last, lockdep_assert_held(&usbio->bulk_mutex); if ((obuf_len > (usbio->txbuf_len - sizeof(*bpkt))) || - (ibuf_len > (usbio->txbuf_len - sizeof(*bpkt)))) + (ibuf_len > (usbio->rxbuf_len - sizeof(*bpkt)))) return -EMSGSIZE; if (ibuf_len) diff --git a/drivers/vdpa/mlx5/core/mlx5_vdpa.h b/drivers/vdpa/mlx5/core/mlx5_vdpa.h index 2cedf7e2dbc4..42f2f44b383c 100644 --- a/drivers/vdpa/mlx5/core/mlx5_vdpa.h +++ b/drivers/vdpa/mlx5/core/mlx5_vdpa.h @@ -11,6 +11,8 @@ #define MLX5V_ETH_HARD_MTU (ETH_HLEN + VLAN_HLEN + ETH_FCS_LEN) +extern int mlx5_vdpa_max_iotlb_entries; + struct mlx5_vdpa_direct_mr { u64 start; u64 end; diff --git a/drivers/vdpa/mlx5/core/mr.c b/drivers/vdpa/mlx5/core/mr.c index 6d02ccf9eb91..77a479aeaa85 100644 --- a/drivers/vdpa/mlx5/core/mr.c +++ b/drivers/vdpa/mlx5/core/mr.c @@ -233,7 +233,8 @@ static int create_direct_keys(struct mlx5_vdpa_dev *mvdev, struct mlx5_vdpa_mr * cmds[i].out = cmd_mem->out; cmds[i].outlen = sizeof(cmd_mem->out); cmds[i].in = cmd_mem->in; - cmds[i].inlen = struct_size(cmd_mem, mtt, mttcount); + cmds[i].inlen = struct_size(cmd_mem, mtt, mttcount) - + offsetof(struct mlx5_create_mkey_mem, in); fill_create_direct_mr(mvdev, dmr, cmd_mem); @@ -776,6 +777,9 @@ static int _mlx5_vdpa_create_mr(struct mlx5_vdpa_dev *mvdev, { int err; + if (mlx5_vdpa_max_iotlb_entries < 2) + return -EINVAL; + if (iotlb) err = create_user_mr(mvdev, mr, iotlb); else @@ -784,7 +788,7 @@ static int _mlx5_vdpa_create_mr(struct mlx5_vdpa_dev *mvdev, if (err) return err; - mr->iotlb = vhost_iotlb_alloc(0, 0); + mr->iotlb = vhost_iotlb_alloc(mlx5_vdpa_max_iotlb_entries, 0); if (!mr->iotlb) { err = -ENOMEM; goto err_mr; diff --git a/drivers/vdpa/mlx5/core/resources.c b/drivers/vdpa/mlx5/core/resources.c index aeae31d0cefa..28a4d7a35bf4 100644 --- a/drivers/vdpa/mlx5/core/resources.c +++ b/drivers/vdpa/mlx5/core/resources.c @@ -3,8 +3,14 @@ #include #include +#include #include "mlx5_vdpa.h" +int mlx5_vdpa_max_iotlb_entries = 2048; +module_param_named(max_iotlb_entries, mlx5_vdpa_max_iotlb_entries, int, 0444); +MODULE_PARM_DESC(max_iotlb_entries, + "Maximum number of iotlb entries. (default: 2048)"); + static int alloc_pd(struct mlx5_vdpa_dev *dev, u32 *pdn, u16 uid) { struct mlx5_core_dev *mdev = dev->mdev; @@ -229,7 +235,10 @@ int mlx5_vdpa_destroy_mkey(struct mlx5_vdpa_dev *mvdev, u32 mkey) static int init_ctrl_vq(struct mlx5_vdpa_dev *mvdev) { - mvdev->cvq.iotlb = vhost_iotlb_alloc(0, 0); + if (mlx5_vdpa_max_iotlb_entries < 2) + return -EINVAL; + + mvdev->cvq.iotlb = vhost_iotlb_alloc(mlx5_vdpa_max_iotlb_entries, 0); if (!mvdev->cvq.iotlb) return -ENOMEM; diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim.c b/drivers/vdpa/vdpa_sim/vdpa_sim.c index 8cb1cc2ea139..4d116644851d 100644 --- a/drivers/vdpa/vdpa_sim/vdpa_sim.c +++ b/drivers/vdpa/vdpa_sim/vdpa_sim.c @@ -34,7 +34,7 @@ MODULE_PARM_DESC(batch_mapping, "Batched mapping 1 -Enable; 0 - Disable"); static int max_iotlb_entries = 2048; module_param(max_iotlb_entries, int, 0444); MODULE_PARM_DESC(max_iotlb_entries, - "Maximum number of iotlb entries for each address space. 0 means unlimited. (default: 2048)"); + "Maximum number of iotlb entries for each address space. (default: 2048)"); static bool use_va = true; module_param(use_va, bool, 0444); @@ -201,6 +201,8 @@ struct vdpasim *vdpasim_create(struct vdpasim_dev_attr *dev_attr, if (!dev_attr->alloc_size) return ERR_PTR(-EINVAL); + if (max_iotlb_entries < 2) + return ERR_PTR(-EINVAL); if (config->mask & BIT_ULL(VDPA_ATTR_DEV_FEATURES)) { if (config->device_features & @@ -261,8 +263,10 @@ struct vdpasim *vdpasim_create(struct vdpasim_dev_attr *dev_attr, for (i = 0; i < vdpasim->dev_attr.nas; i++) { vhost_iotlb_init(&vdpasim->iommu[i], max_iotlb_entries, 0); - vhost_iotlb_add_range(&vdpasim->iommu[i], 0, ULONG_MAX, 0, - VHOST_MAP_RW); + ret = vhost_iotlb_add_range(&vdpasim->iommu[i], 0, ULONG_MAX, + 0, VHOST_MAP_RW); + if (ret) + goto err_iommu; vdpasim->iommu_pt[i] = true; } diff --git a/drivers/vdpa/vdpa_user/iova_domain.c b/drivers/vdpa/vdpa_user/iova_domain.c index 4dc76c0d0d13..b6c958224b7c 100644 --- a/drivers/vdpa/vdpa_user/iova_domain.c +++ b/drivers/vdpa/vdpa_user/iova_domain.c @@ -12,11 +12,17 @@ #include #include #include +#include #include #include #include "iova_domain.h" +static int max_iotlb_entries = 2048; +module_param(max_iotlb_entries, int, 0444); +MODULE_PARM_DESC(max_iotlb_entries, + "Maximum number of iotlb entries. (default: 2048)"); + static int vduse_iotlb_add_range(struct vduse_iova_domain *domain, u64 start, u64 last, u64 addr, unsigned int perm, @@ -622,11 +628,14 @@ vduse_domain_create(unsigned long iova_limit, size_t bounce_size) if (iova_limit <= bounce_size) return NULL; + if (max_iotlb_entries <= 0) + return NULL; + domain = kzalloc_obj(*domain); if (!domain) return NULL; - domain->iotlb = vhost_iotlb_alloc(0, 0); + domain->iotlb = vhost_iotlb_alloc(max_iotlb_entries, 0); if (!domain->iotlb) goto err_iotlb; diff --git a/drivers/vhost/iotlb.c b/drivers/vhost/iotlb.c index e1414c774c34..a1d4376a5b87 100644 --- a/drivers/vhost/iotlb.c +++ b/drivers/vhost/iotlb.c @@ -20,6 +20,14 @@ INTERVAL_TREE_DEFINE(struct vhost_iotlb_map, rb, __u64, __subtree_last, START, LAST, static inline, vhost_iotlb_itree); +static void vhost_iotlb_map_unlink(struct vhost_iotlb *iotlb, + struct vhost_iotlb_map *map) +{ + vhost_iotlb_itree_remove(map, &iotlb->root); + list_del(&map->link); + iotlb->nmaps--; +} + /** * vhost_iotlb_map_free - remove a map node and free it * @iotlb: the IOTLB @@ -28,10 +36,8 @@ INTERVAL_TREE_DEFINE(struct vhost_iotlb_map, void vhost_iotlb_map_free(struct vhost_iotlb *iotlb, struct vhost_iotlb_map *map) { - vhost_iotlb_itree_remove(map, &iotlb->root); - list_del(&map->link); + vhost_iotlb_map_unlink(iotlb, map); kfree(map); - iotlb->nmaps--; } EXPORT_SYMBOL_GPL(vhost_iotlb_map_free); @@ -57,14 +63,25 @@ int vhost_iotlb_add_range_ctx(struct vhost_iotlb *iotlb, if (last < start) return -EFAULT; + if (!iotlb->limit) + return -EINVAL; + /* If the range being mapped is [0, ULONG_MAX], split it into two entries * otherwise its size would overflow u64. */ if (start == 0 && last == ULONG_MAX) { u64 mid = last / 2; - int err = vhost_iotlb_add_range_ctx(iotlb, start, mid, addr, - perm, opaque); + int err; + if (iotlb->limit < 2) + return -ENOSPC; + + if (!(iotlb->flags & VHOST_IOTLB_FLAG_RETIRE) && + iotlb->nmaps > iotlb->limit - 2) + return -ENOSPC; + + err = vhost_iotlb_add_range_ctx(iotlb, start, mid, addr, + perm, opaque); if (err) return err; @@ -72,17 +89,19 @@ int vhost_iotlb_add_range_ctx(struct vhost_iotlb *iotlb, start = mid + 1; } - if (iotlb->limit && - iotlb->nmaps == iotlb->limit && - iotlb->flags & VHOST_IOTLB_FLAG_RETIRE) { - map = list_first_entry(&iotlb->list, typeof(*map), link); - vhost_iotlb_map_free(iotlb, map); + if (iotlb->nmaps >= iotlb->limit) { + if (iotlb->flags & VHOST_IOTLB_FLAG_RETIRE) { + map = list_first_entry(&iotlb->list, typeof(*map), link); + vhost_iotlb_map_unlink(iotlb, map); + } else { + return -ENOSPC; + } + } else { + map = kmalloc_obj(*map, GFP_ATOMIC); + if (!map) + return -ENOMEM; } - map = kmalloc_obj(*map, GFP_ATOMIC); - if (!map) - return -ENOMEM; - map->start = start; map->size = last - start + 1; map->last = last; diff --git a/drivers/vhost/scsi.c b/drivers/vhost/scsi.c index 9a1253b9d8c5..7a1f39a327da 100644 --- a/drivers/vhost/scsi.c +++ b/drivers/vhost/scsi.c @@ -972,6 +972,9 @@ vhost_scsi_mapal(struct vhost_scsi *vs, struct vhost_scsi_cmd *cmd, if (prot_bytes) { sgl_count = vhost_scsi_calc_sgls(prot_iter, prot_bytes, VHOST_SCSI_PREALLOC_PROT_SGLS); + if (sgl_count < 0) + return sgl_count; + cmd->prot_table.sgl = cmd->prot_sgl; ret = sg_alloc_table_chained(&cmd->prot_table, sgl_count, cmd->prot_table.sgl, @@ -1416,6 +1419,11 @@ vhost_scsi_handle_vq(struct vhost_scsi *vs, struct vhost_virtqueue *vq) * actual data payload length. */ if (prot_bytes) { + if (prot_bytes >= exp_data_len) { + vq_err(vq, "Protection data exceeds payload length\n"); + goto err; + } + exp_data_len -= prot_bytes; prot_iter = data_iter; iov_iter_truncate(&prot_iter, prot_bytes); @@ -2219,6 +2227,7 @@ static int vhost_scsi_set_features(struct vhost_scsi *vs, u64 features) { struct vhost_virtqueue *vq; bool is_log, was_log; + u64 old_features; int i; if (features & ~VHOST_SCSI_FEATURES) @@ -2234,6 +2243,14 @@ static int vhost_scsi_set_features(struct vhost_scsi *vs, u64 features) if (!vs->dev.nvqs) goto out; + old_features = vs->vqs[0].vq.acked_features; + if (vs->vs_tpg && + ((features ^ old_features) & + ~(1ULL << VHOST_F_LOG_ALL))) { + mutex_unlock(&vs->dev.mutex); + return -EBUSY; + } + is_log = features & (1 << VHOST_F_LOG_ALL); /* * All VQs should have same feature. @@ -2426,9 +2443,10 @@ vhost_scsi_ioctl(struct file *f, default: mutex_lock(&vs->dev.mutex); r = vhost_dev_ioctl(&vs->dev, ioctl, argp); - /* TODO: flush backend after dev ioctl. */ if (r == -ENOIOCTLCMD) r = vhost_vring_ioctl(&vs->dev, ioctl, argp); + else + vhost_scsi_flush(vs); mutex_unlock(&vs->dev.mutex); return r; } diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c index ac55275fa0d0..c3d913bd7cac 100644 --- a/drivers/vhost/vdpa.c +++ b/drivers/vhost/vdpa.c @@ -34,6 +34,11 @@ enum { #define VHOST_VDPA_DEV_MAX (1U << MINORBITS) +static int max_iotlb_entries = 2048; +module_param(max_iotlb_entries, int, 0444); +MODULE_PARM_DESC(max_iotlb_entries, + "Maximum number of iotlb entries. (default: 2048)"); + #define VHOST_VDPA_IOTLB_BUCKETS 16 struct vhost_vdpa_as { @@ -109,12 +114,14 @@ static struct vhost_vdpa_as *vhost_vdpa_alloc_as(struct vhost_vdpa *v, u32 asid) if (asid >= v->vdpa->nas) return NULL; + if (max_iotlb_entries <= 0) + return NULL; as = kmalloc_obj(*as); if (!as) return NULL; - vhost_iotlb_init(&as->iotlb, 0, 0); + vhost_iotlb_init(&as->iotlb, max_iotlb_entries, 0); as->id = asid; hlist_add_head(&as->hash_link, head); @@ -1102,6 +1109,7 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v, unsigned int gup_flags = FOLL_LONGTERM; unsigned long npages, cur_base, map_pfn, last_pfn = 0; unsigned long lock_limit, sz2pin, nchunks, i; + unsigned long page_offset; u64 start = iova; long pinned; int ret = 0; @@ -1114,7 +1122,13 @@ static int vhost_vdpa_pa_map(struct vhost_vdpa *v, if (perm & VHOST_ACCESS_WO) gup_flags |= FOLL_WRITE; - npages = PFN_UP(size + (iova & ~PAGE_MASK)); + page_offset = iova & ~PAGE_MASK; + if (size > ULONG_MAX - page_offset) { + ret = -EINVAL; + goto free; + } + + npages = PFN_UP(size + page_offset); if (!npages) { ret = -EINVAL; goto free; diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c index 4c525b3e16ea..269efad90369 100644 --- a/drivers/vhost/vhost.c +++ b/drivers/vhost/vhost.c @@ -1137,6 +1137,9 @@ EXPORT_SYMBOL_GPL(vhost_dev_set_owner); static struct vhost_iotlb *iotlb_alloc(void) { + if (max_iotlb_entries <= 0) + return NULL; + return vhost_iotlb_alloc(max_iotlb_entries, VHOST_IOTLB_FLAG_RETIRE); } @@ -1981,6 +1984,8 @@ static long vhost_set_memory(struct vhost_dev *d, struct vhost_memory __user *m) return -EOPNOTSUPP; if (mem.nregions > max_mem_regions) return -E2BIG; + if (max_iotlb_entries <= 0) + return -EINVAL; newmem = kvzalloc_flex(*newmem, regions, mem.nregions); if (!newmem) return -ENOMEM; @@ -2126,6 +2131,14 @@ static long vhost_vring_set_num_addr(struct vhost_dev *d, BUG(); } + /* + * The metadata cache holds the IOTLB mapping that backed the previous + * desc/avail/used addresses and vring size, both of which are being + * replaced here. iotlb_access_ok() takes a cache hit as proof that the + * region was validated, so the stale entries have to go. + */ + __vhost_vq_meta_reset(vq); + mutex_unlock(&vq->mutex); return r; @@ -2275,6 +2288,9 @@ int vhost_init_device_iotlb(struct vhost_dev *d) struct vhost_iotlb *niotlb, *oiotlb; int i; + if (max_iotlb_entries <= 0) + return -EINVAL; + niotlb = iotlb_alloc(); if (!niotlb) return -ENOMEM; diff --git a/drivers/video/fbdev/core/bitblit.c b/drivers/video/fbdev/core/bitblit.c index 65681dcc5930..39f44258d793 100644 --- a/drivers/video/fbdev/core/bitblit.c +++ b/drivers/video/fbdev/core/bitblit.c @@ -273,9 +273,14 @@ static void bit_cursor(struct vc_data *vc, struct fb_info *info, bool enable, if (!vc->vc_font.data) return; - c = scr_readw((u16 *) vc->vc_pos); + c = scr_readw((u16 *) vc->vc_pos); attribute = get_attribute(info, c); - src = vc->vc_font.data + ((c & charmask) * (w * vc->vc_font.height)); + c &= charmask; + + /* Clamp to font size, same as bit_putcs_aligned() */ + if (c >= vc->vc_font.charcount) + c = 0; + src = vc->vc_font.data + (c * (w * vc->vc_font.height)); if (par->cursor_state.image.data != (const char *)src || par->cursor_reset) { diff --git a/drivers/video/fbdev/core/fb_io_fops.c b/drivers/video/fbdev/core/fb_io_fops.c index 6ab60fcd0050..0798e88799eb 100644 --- a/drivers/video/fbdev/core/fb_io_fops.c +++ b/drivers/video/fbdev/core/fb_io_fops.c @@ -61,6 +61,14 @@ ssize_t fb_io_read(struct fb_info *info, char __user *buf, size_t count, loff_t buf += c; cnt += c; count -= c; + + /* + * If there was a partial copy, the user buffer is faulty. + * Break out to avoid over-advancing the src pointer and + * reading out of bounds in the next iteration. + */ + if (trailing) + break; } kfree(buffer); diff --git a/drivers/video/fbdev/core/fbcon.c b/drivers/video/fbdev/core/fbcon.c index 9f5c4c101581..23b3c536d53d 100644 --- a/drivers/video/fbdev/core/fbcon.c +++ b/drivers/video/fbdev/core/fbcon.c @@ -2641,9 +2641,31 @@ static void fbcon_modechanged(struct fb_info *info) fbcon_info_from_console(par->currcon) != info) return; + /* + * Clear the selection before switching bitops. Without this, the + * clear_selection() inside vc_resize() below repaints the highlighted + * cells through the new bitops while the console geometry(vc_rows/vc_cols) + * has not been updated to match, so the repaint is computed from a + * half-switched geometry and overflows the framebuffer address. + * Pre-clearing makes that repaint a no-op. + */ + clear_selection(); + p = &fb_display[vc->vc_num]; set_blitting_type(vc, info); + /* + * Rebuild par->rotated.buf for the new rotation now that bitops have + * switched. The new putcs/cursor ops read this buffer; if it is still + * sized for the old rotation, fbcon_putcs() and the cursor path reached + * via update_screen() below overflow it. Mirrors fbcon_switch(); fall + * back to unrotated rendering on allocation failure. + */ + if (par->bitops->rotate_font && par->bitops->rotate_font(info, vc)) { + par->rotate = FB_ROTATE_UR; + set_blitting_type(vc, info); + } + if (con_is_visible(vc)) { var_to_display(p, &info->var, info); cols = FBCON_SWAP(par->rotate, info->var.xres, info->var.yres); @@ -2675,6 +2697,9 @@ static void fbcon_set_all_vcs(struct fb_info *info) if (!par || par->currcon < 0) return; + /* See the comment in fbcon_modechanged(). */ + clear_selection(); + for (i = first_fb_vc; i <= last_fb_vc; i++) { vc = vc_cons[i].d; if (!vc || vc->vc_mode != KD_TEXT || diff --git a/drivers/video/fbdev/core/fbmem.c b/drivers/video/fbdev/core/fbmem.c index 2f1c56e5a7a2..c8aa163b0ecf 100644 --- a/drivers/video/fbdev/core/fbmem.c +++ b/drivers/video/fbdev/core/fbmem.c @@ -246,8 +246,11 @@ fb_set_var(struct fb_info *info, struct fb_var_screeninfo *var) ret = fb_mode_is_equal(&mode1, &mode2); if (!ret) { ret = fbcon_mode_deleted(info, &mode1); - if (!ret) + if (!ret) { + if (info->mode && fb_mode_is_equal(info->mode, &mode1)) + info->mode = NULL; fb_delete_videomode(&mode1, &info->modelist); + } } return ret ? -EINVAL : 0; diff --git a/drivers/video/fbdev/core/fbsysfs.c b/drivers/video/fbdev/core/fbsysfs.c index ea196603c7a8..d3d60c555bb8 100644 --- a/drivers/video/fbdev/core/fbsysfs.c +++ b/drivers/video/fbdev/core/fbsysfs.c @@ -12,27 +12,35 @@ #include "fb_internal.h" #include "fbcon.h" +static int activate_locked(struct fb_info *fb_info, + struct fb_var_screeninfo *var) +{ + var->activate |= FB_ACTIVATE_FORCE; + return fb_set_var_from_user(fb_info, var); +} + static int activate(struct fb_info *fb_info, struct fb_var_screeninfo *var) { int err; - var->activate |= FB_ACTIVATE_FORCE; console_lock(); lock_fb_info(fb_info); - err = fb_set_var_from_user(fb_info, var); + err = activate_locked(fb_info, var); unlock_fb_info(fb_info); console_unlock(); - if (err) - return err; - return 0; + + return err; } -static int mode_string(char *buf, unsigned int offset, +static int mode_string(char *buf, size_t size, unsigned int offset, const struct fb_videomode *mode) { char m = 'U'; char v = 'p'; + if (offset >= size) + return 0; + if (mode->flag & FB_MODE_IS_DETAILED) m = 'D'; if (mode->flag & FB_MODE_IS_VESA) @@ -45,7 +53,7 @@ static int mode_string(char *buf, unsigned int offset, if (mode->vmode & FB_VMODE_DOUBLE) v = 'd'; - return snprintf(&buf[offset], PAGE_SIZE - offset, "%c:%dx%d%c-%d\n", + return scnprintf(&buf[offset], size - offset, "%c:%dx%d%c-%d\n", m, mode->xres, mode->yres, v, mode->refresh); } @@ -62,19 +70,32 @@ static ssize_t store_mode(struct device *device, struct device_attribute *attr, memset(&var, 0, sizeof(var)); + console_lock(); + lock_fb_info(fb_info); + list_for_each_entry(modelist, &fb_info->modelist, list) { mode = &modelist->mode; - i = mode_string(mstr, 0, mode); + i = mode_string(mstr, sizeof(mstr), 0, mode); if (strncmp(mstr, buf, max(count, i)) == 0) { var = fb_info->var; fb_videomode_to_var(&var, mode); - if ((err = activate(fb_info, &var))) + err = activate_locked(fb_info, &var); + if (err) { + unlock_fb_info(fb_info); + console_unlock(); return err; + } fb_info->mode = mode; + unlock_fb_info(fb_info); + console_unlock(); return count; } } + + unlock_fb_info(fb_info); + console_unlock(); + return -EINVAL; } @@ -82,11 +103,20 @@ static ssize_t show_mode(struct device *device, struct device_attribute *attr, char *buf) { struct fb_info *fb_info = dev_get_drvdata(device); + struct fb_videomode mode; + bool have_mode = false; - if (!fb_info->mode) + lock_fb_info(fb_info); + if (fb_info->mode) { + mode = *fb_info->mode; + have_mode = true; + } + unlock_fb_info(fb_info); + + if (!have_mode) return 0; - return mode_string(buf, 0, fb_info->mode); + return mode_string(buf, PAGE_SIZE, 0, &mode); } static ssize_t store_modes(struct device *device, @@ -134,10 +164,15 @@ static ssize_t show_modes(struct device *device, struct device_attribute *attr, const struct fb_videomode *mode; i = 0; + lock_fb_info(fb_info); list_for_each_entry(modelist, &fb_info->modelist, list) { mode = &modelist->mode; - i += mode_string(buf, i, mode); + i += mode_string(buf, PAGE_SIZE, i, mode); + if (i >= PAGE_SIZE - 1) + break; } + unlock_fb_info(fb_info); + return i; } diff --git a/drivers/watchdog/at91sam9_wdt.c b/drivers/watchdog/at91sam9_wdt.c index aba66b8e9d03..80ba04df54ad 100644 --- a/drivers/watchdog/at91sam9_wdt.c +++ b/drivers/watchdog/at91sam9_wdt.c @@ -242,7 +242,7 @@ static int at91_wdt_init(struct platform_device *pdev, struct at91wdt *wdt) return 0; out_stop_timer: - timer_delete(&wdt->timer); + timer_shutdown_sync(&wdt->timer); return err; } @@ -378,7 +378,7 @@ static void at91wdt_remove(struct platform_device *pdev) watchdog_unregister_device(&wdt->wdd); pr_warn("I quit now, hardware will probably reboot!\n"); - timer_delete(&wdt->timer); + timer_shutdown_sync(&wdt->timer); } #if defined(CONFIG_OF) diff --git a/drivers/watchdog/atcwdt200_wdt.c b/drivers/watchdog/atcwdt200_wdt.c index 8e3b18aea368..7e757c217b0b 100644 --- a/drivers/watchdog/atcwdt200_wdt.c +++ b/drivers/watchdog/atcwdt200_wdt.c @@ -260,9 +260,9 @@ static void atcwdt_get_timeout_params(struct atcwdt_drv *drv_data, * register to determine the interrupt timer type supported by the hardware. * * Note: This function must only be called when the ATCWDT200 watchdog is - * disabled. If the watchdog is enabled, this function returns TMR_UNKNOWN. + * disabled. If the watchdog is enabled, this function returns -EBUSY. * - * Returns: The interrupt timer type supported by the hardware. + * Returns: 0 on success or negative error code on failure. */ static int atcwdt_get_int_timer_type(struct atcwdt_drv *drv_data) { @@ -274,7 +274,8 @@ static int atcwdt_get_int_timer_type(struct atcwdt_drv *drv_data) regmap_read(drv_data->regmap, REG_CTRL, &val); if (val & CTRL_WDT_EN) { spin_unlock(&drv_data->lock); - return TMR_UNKNOWN; + return dev_err_probe(dev, -EBUSY, + "Watchdog is enabled, cannot detect timer type\n"); } /* diff --git a/drivers/watchdog/bd96801_wdt.c b/drivers/watchdog/bd96801_wdt.c index 12b74fd2bc05..a25b7cf1488b 100644 --- a/drivers/watchdog/bd96801_wdt.c +++ b/drivers/watchdog/bd96801_wdt.c @@ -169,7 +169,6 @@ static int bd96801_set_wdt_mode(struct wdtbd96801 *w, unsigned int hw_margin, int fastng, slowng, type, ret, reg, mask; struct device *dev = w->dev; - if (hw_margin_min * 1000 > FASTNG_MAX_US) { dev_err(dev, "Unsupported fast timeout %u uS [max %u]\n", hw_margin_min * 1000, FASTNG_MAX_US); @@ -258,10 +257,10 @@ static int bd96801_set_heartbeat_from_hw(struct wdtbd96801 *w, fast = FASTNG_MIN << sel; sel = (val & BD96801_WD_RATIO_MASK) + 1; - w->wdt.max_hw_heartbeat_ms = (fast << sel) / USEC_PER_MSEC; + w->wdt.max_hw_heartbeat_ms = (fast << sel) / 10; if ((conf_reg & BD96801_WD_TYPE_MASK) == BD96801_WD_TYPE_WIN) - w->wdt.min_hw_heartbeat_ms = fast / USEC_PER_MSEC; + w->wdt.min_hw_heartbeat_ms = fast / 10; return 0; } diff --git a/fs/attr.c b/fs/attr.c index 4f437fabb7f0..71888ac903c2 100644 --- a/fs/attr.c +++ b/fs/attr.c @@ -176,7 +176,7 @@ int setattr_prepare(struct mnt_idmap *idmap, struct dentry *dentry, * covered by the open-time check because sys_truncate() takes a * path, not an open file. */ - if (IS_ENABLED(CONFIG_FS_VERITY) && IS_VERITY(inode)) + if (IS_VERITY(inode)) return -EPERM; error = inode_newsize_ok(inode, attr->ia_size); diff --git a/fs/btrfs/Kconfig b/fs/btrfs/Kconfig index 9de04c37e11a..4b10d78ed99b 100644 --- a/fs/btrfs/Kconfig +++ b/fs/btrfs/Kconfig @@ -106,7 +106,8 @@ config BTRFS_EXPERIMENTAL - extent tree v2 - complex rework of extent tracking - - block size > page size support + - block size > page size support - needs transparent huge page and + non-HIGHMEM system - huge folios for data - folios can be as large as 2MiB now diff --git a/fs/btrfs/btrfs_inode.h b/fs/btrfs/btrfs_inode.h index 7fdc6c3fd066..1082fa92c145 100644 --- a/fs/btrfs/btrfs_inode.h +++ b/fs/btrfs/btrfs_inode.h @@ -600,6 +600,7 @@ int btrfs_prealloc_file_range_trans(struct inode *inode, loff_t actual_len, u64 *alloc_hint); int btrfs_run_delalloc_range(struct btrfs_inode *inode, struct folio *locked_folio, u64 start, u64 end, struct writeback_control *wbc); +void btrfs_queue_writepage_fixup(struct btrfs_inode *inode, struct folio *folio); int btrfs_encoded_io_compression_from_extent(struct btrfs_fs_info *fs_info, int compress_type); int btrfs_encoded_read_regular_fill_pages(struct btrfs_inode *inode, diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 22c321719e4f..2f1666d9544e 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -1760,6 +1760,8 @@ static int read_backup_root(struct btrfs_fs_info *fs_info, u8 priority) /* helper to cleanup workers */ static void btrfs_stop_all_workers(struct btrfs_fs_info *fs_info) { + if (fs_info->fixup_workers) + destroy_workqueue(fs_info->fixup_workers); btrfs_destroy_workqueue(fs_info->delalloc_workers); btrfs_destroy_workqueue(fs_info->workers); if (fs_info->endio_workers) @@ -1967,6 +1969,9 @@ static int btrfs_init_workqueues(struct btrfs_fs_info *fs_info) fs_info->caching_workers = btrfs_alloc_workqueue(fs_info, "cache", flags, max_active, 0); + fs_info->fixup_workers = + alloc_ordered_workqueue("btrfs-fixup", ordered_flags); + fs_info->endio_workers = alloc_workqueue("btrfs-endio", flags, max_active); fs_info->endio_meta_workers = @@ -1992,7 +1997,7 @@ static int btrfs_init_workqueues(struct btrfs_fs_info *fs_info) fs_info->endio_workers && fs_info->endio_meta_workers && fs_info->endio_write_workers && fs_info->endio_freespace_worker && fs_info->rmw_workers && - fs_info->caching_workers && + fs_info->caching_workers && fs_info->fixup_workers && fs_info->delayed_workers && fs_info->qgroup_rescan_workers && fs_info->discard_ctl.discard_workers)) { return -ENOMEM; @@ -3468,7 +3473,15 @@ int __cold open_ctree(struct super_block *sb, struct btrfs_fs_devices *fs_device fs_info->sectorsize = sectorsize; fs_info->sectorsize_bits = ilog2(sectorsize); fs_info->block_min_order = ilog2(round_up(sectorsize, PAGE_SIZE) >> PAGE_SHIFT); - fs_info->block_max_order = calc_block_max_order(fs_info->sectorsize_bits); + /* + * For HIGHMEM, a large folio cannot be mapped in one go, breaking a lot + * of basic assumptions for btrfs IOs. + * Disable large folios for such 32-bit systems. + */ + if (IS_ENABLED(CONFIG_HIGHMEM)) + fs_info->block_max_order = fs_info->block_min_order; + else + fs_info->block_max_order = calc_block_max_order(fs_info->sectorsize_bits); fs_info->csums_per_leaf = BTRFS_MAX_ITEM_SIZE(fs_info) / fs_info->csum_size; fs_info->stripesize = stripesize; fs_info->fs_devices->fs_info = fs_info; @@ -4356,6 +4369,18 @@ void __cold close_ctree(struct btrfs_fs_info *fs_info) /* clear out the rbtree of defraggable inodes */ btrfs_cleanup_defrag_inodes(fs_info); + /* + * Before the unmount, we sync down all the writeback which can + * generate fixup work. We are about to run delalloc for autodefrag so + * piggy back on that by also flushing the fixup work which can also + * generate delalloc we would like to get run. + * + * After this, it is still possible that some thread doing writeback is + * in btrfs_queue_writepage_fixup() and might finish queueing some final + * work, racing the btrfs_fs_closing() check there. + */ + flush_workqueue(fs_info->fixup_workers); + /* * Handle the error fs first, as it will flush and wait for all ordered * extents. This will generate delayed iputs, thus we want to handle @@ -4433,6 +4458,15 @@ void __cold close_ctree(struct btrfs_fs_info *fs_info) cancel_work_sync(&fs_info->preempt_reclaim_work); cancel_work_sync(&fs_info->em_shrinker_work); + /* + * Reclaim workers can run writeback which can queue fixup. + * After the above cancel_work_sync() calls, any such queueing attempts are + * guaranteed to see btrfs_fs_closing(), so at this point we can genuinely fully + * flush the fixup workqueue. This relies on the belief that *now* no thread can + * still be sitting in btrfs_queue_writepage_fixup(). + */ + flush_workqueue(fs_info->fixup_workers); + /* * Run delayed iputs again because an async reclaim worker may have * added new ones if it was flushing delalloc: diff --git a/fs/btrfs/extent_io.c b/fs/btrfs/extent_io.c index de5785117a47..f032f0858f40 100644 --- a/fs/btrfs/extent_io.c +++ b/fs/btrfs/extent_io.c @@ -1440,6 +1440,115 @@ static bool find_next_delalloc_bitmap(struct folio *folio, return true; } +/* + * Debug checks for fixup selection logic to help ensure the invariants + * we expect for fixup marking hold in practice. + * + * - A dirty block without a fixup bit is covered by delalloc or a running + * ordered extent (it was dirtied by a reserving write path). + * - A block with a fixup bit is never covered by delalloc: every delalloc + * setter holds the folio lock and cancels the fixup state of the blocks + * it covers (btrfs_folio_set_dirty()) before releasing it. + */ +static void debug_check_writepage_fixup(struct btrfs_inode *inode, u64 start, + u32 len, bool needs_fixup) +{ + struct btrfs_ordered_extent *ordered; + bool delalloc; + + if (!IS_ENABLED(CONFIG_BTRFS_DEBUG)) + return; + + delalloc = btrfs_test_range_bit_exists(&inode->io_tree, start, + start + len - 1, EXTENT_DELALLOC); + if (needs_fixup) { + if (unlikely(delalloc)) + DEBUG_WARN("writeback: delalloc and fixup conflict. ino %llu start %llu", + btrfs_ino(inode), start); + } else { + if (delalloc) + return; + + ordered = btrfs_lookup_ordered_range(inode, start, len); + if (unlikely(!ordered)) + DEBUG_WARN("dirty block, no delalloc, fixup, ordered. ino %llu start %llu", + btrfs_ino(inode), start); + else + btrfs_put_ordered_extent(ordered); + } +} + +/* + * Handle folios dirtied without a delalloc reservation, e.g. + * O_DIRECT read into a MAP_SHARED mapping dirtying via set_page_dirty_lock(). + * + * btrfs_data_dirty_folio() records the affected blocks in the fixup bitmap + * and the folio fixup flag and we check them here in writeback. + * + * Don't submit such blocks and queue work for the fixup worker to reserve + * space for them so that they can be submitted properly by writeback. + * + * Return 1 if the folio needed fixup, 0 if not, and a negative error code + * on error. + */ +static noinline_for_stack int writepage_fixup(struct btrfs_inode *inode, + struct folio *folio, + struct btrfs_bio_ctrl *bio_ctrl) +{ + struct btrfs_fs_info *fs_info = inode_to_fs_info(&inode->vfs_inode); + const unsigned int blocks_per_folio = btrfs_blocks_per_folio(fs_info, folio); + const u32 sectorsize = fs_info->sectorsize; + const u64 page_start = folio_pos(folio); + bool found_fixup = false; + unsigned int bit; + + /* + * A folio was dirtied without calling aops->dirty_folio() which we + * explicitly assert is not allowed. + */ + if (unlikely(bitmap_empty(bio_ctrl->submit_bitmap, blocks_per_folio))) { + DEBUG_WARN(); + btrfs_err_rl(fs_info, + "root %lld ino %llu folio %llu is dirty with an empty dirty bitmap", + btrfs_root_id(inode->root), btrfs_ino(inode), + folio_pos(folio)); + return -EUCLEAN; + } + + /* Cheap check on the folio flag. Set iff the fixup bitmap is non-empty. */ + if (likely(!folio_test_fixup_pending(folio))) + return 0; + + for_each_set_bit(bit, bio_ctrl->submit_bitmap, blocks_per_folio) { + const u64 start = page_start + (bit << fs_info->sectorsize_bits); + const bool needs_fixup = btrfs_folio_test_fixup(fs_info, folio, + start, sectorsize); + + debug_check_writepage_fixup(inode, start, sectorsize, needs_fixup); + if (needs_fixup) { + bitmap_clear(bio_ctrl->submit_bitmap, bit, 1); + found_fixup = true; + } + } + if (likely(found_fixup)) { + btrfs_queue_writepage_fixup(inode, folio); + folio_redirty_for_writepage(bio_ctrl->wbc, folio); + if (bitmap_empty(bio_ctrl->submit_bitmap, blocks_per_folio)) { + folio_unlock(folio); + return 1; + } + return 0; + } + /* We should always find fixup if the folio fixup flag was set. */ + DEBUG_WARN(); + btrfs_err_rl(fs_info, + "root %lld ino %llu folio %llu is fixup with an empty fixup bitmap", + btrfs_root_id(inode->root), btrfs_ino(inode), + folio_pos(folio)); + + return -EUCLEAN; +} + /* * Do all of the delayed allocation setup. * @@ -1492,6 +1601,10 @@ static noinline_for_stack int writepage_delalloc(struct btrfs_inode *inode, /* Save the dirty bitmap as our submission bitmap will be a subset of it. */ btrfs_copy_subpage_dirty_bitmap(fs_info, folio, bio_ctrl->submit_bitmap); + ret = writepage_fixup(inode, folio, bio_ctrl); + if (ret) + return ret; + for_each_set_bitrange(start_bit, end_bit, bio_ctrl->submit_bitmap, blocks_per_folio) { u64 start = page_start + (start_bit << fs_info->sectorsize_bits); diff --git a/fs/btrfs/fs.c b/fs/btrfs/fs.c index 14d83565cdee..dcf12979af33 100644 --- a/fs/btrfs/fs.c +++ b/fs/btrfs/fs.c @@ -166,9 +166,17 @@ bool __attribute_const__ btrfs_supported_blocksize(u32 blocksize) * * Considering HIGHMEM is such a pain to deal with and it's going * to be deprecated eventually, just reject HIGHMEM && bs > ps cases. + * + * Finally, for bs > ps cases, we need to set the minimal folio order, + * which requires transparent hugepage. */ - if (IS_ENABLED(CONFIG_HIGHMEM) && blocksize > PAGE_SIZE) - return false; + if (blocksize > PAGE_SIZE) { + if (IS_ENABLED(CONFIG_HIGHMEM)) + return false; + + if (!IS_ENABLED(CONFIG_TRANSPARENT_HUGEPAGE)) + return false; + } return true; #endif return false; diff --git a/fs/btrfs/fs.h b/fs/btrfs/fs.h index 7ee9ec2b0efb..f7f343fbe732 100644 --- a/fs/btrfs/fs.h +++ b/fs/btrfs/fs.h @@ -713,6 +713,8 @@ struct btrfs_fs_info { struct btrfs_workqueue *endio_write_workers; struct btrfs_workqueue *endio_freespace_worker; struct btrfs_workqueue *caching_workers; + + struct workqueue_struct *fixup_workers; struct btrfs_workqueue *delayed_workers; struct task_struct *transaction_kthread; @@ -1200,6 +1202,16 @@ static inline void btrfs_wake_unfinished_drop(struct btrfs_fs_info *fs_info) clear_and_wake_up_bit(BTRFS_FS_UNFINISHED_DROPS, &fs_info->flags); } +/* + * We use the folio owner_2 flag to indicate the folio has blocks that were + * dirtied without a space reservation and need the writepage fixup before + * writeback. For bs < folio_size the fixup bitmap tracks the affected + * blocks. + */ +#define folio_test_fixup_pending(folio) folio_test_owner_2(folio) +#define folio_set_fixup_pending(folio) folio_set_owner_2(folio) +#define folio_clear_fixup_pending(folio) folio_clear_owner_2(folio) + #define BTRFS_FS_ERROR(fs_info) (READ_ONCE((fs_info)->fs_error)) #define BTRFS_FS_LOG_CLEANUP_ERROR(fs_info) \ diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index b446c3014b24..2534cd9284d5 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c @@ -2812,6 +2812,180 @@ int btrfs_set_extent_delalloc(struct btrfs_inode *inode, u64 start, u64 end, EXTENT_DELALLOC | extra_bits, cached_state); } +struct btrfs_writepage_fixup { + struct folio *folio; + struct btrfs_inode *inode; + struct work_struct work; +}; + +/* + * Do the real fixup work of reserving space for the blocks a folio's fixup + * state records. Queued by writepage_fixup() when writeback found the bits set. + * + * Since the fixup can be cancelled by a task dirtying with a reservation, we must + * re-check the state of fixup under the folio lock. + */ +static void btrfs_writepage_fixup_worker(struct work_struct *work) +{ + struct btrfs_writepage_fixup *fixup = + container_of(work, struct btrfs_writepage_fixup, work); + struct extent_state *cached_state = NULL; + struct extent_changeset *data_reserved = NULL; + unsigned long delalloc_bitmap[BITS_TO_LONGS(BTRFS_MAX_BLOCKS_PER_FOLIO)] = { 0 }; + struct folio *folio = fixup->folio; + struct btrfs_inode *inode = fixup->inode; + struct btrfs_fs_info *fs_info = inode->root->fs_info; + const unsigned int blocks_per_folio = btrfs_blocks_per_folio(fs_info, folio); + const u32 sectorsize = fs_info->sectorsize; + const u64 page_start = folio_pos(folio); + const u64 page_end = folio_next_pos(folio) - 1; + unsigned int start_bit; + unsigned int end_bit; + unsigned int bit; + bool reserved; + int ret; + + /* + * We would prefer to reserve under the folio lock when we know exactly + * which blocks need a reservation. Unfortunately, since the reservation + * can go into flushers which can go into writeback, which takes folio + * locks, that is not possible. Therefore, we have to reserve for the + * whole folio here, then release what we didn't end up needing once we + * figure it out. + * + * Also note the slightly strange error checking. If fixup is actually + * not set, we don't need to mark an error on the mapping. So hang on to + * ret until after we lock and find out if we actually care. + */ + ret = btrfs_delalloc_reserve_space(inode, &data_reserved, page_start, + folio_size(folio)); + reserved = (ret == 0); +again: + folio_lock(folio); + + if (!folio->mapping || !folio_test_fixup_pending(folio)) { + ret = 0; + goto out; + } + if (ret) + goto out; + + btrfs_lock_extent(&inode->io_tree, page_start, page_end, &cached_state); + + for (bit = 0; bit < blocks_per_folio; bit++) { + struct btrfs_ordered_extent *ordered; + const u64 start = page_start + (bit << fs_info->sectorsize_bits); + + if (test_bit(bit, delalloc_bitmap)) + continue; + if (!btrfs_folio_test_fixup(fs_info, folio, start, sectorsize)) + continue; + /* + * Any task that sets EXTENT_DELALLOC clears the fixup bits + * under the folio lock, so it should be impossible to observe + * both under the lock. Setting delalloc twice would wrongly + * double account the space. + */ + if (IS_ENABLED(CONFIG_BTRFS_DEBUG) && + unlikely(btrfs_test_range_bit_exists(&inode->io_tree, start, + start + sectorsize - 1, + EXTENT_DELALLOC))) { + DEBUG_WARN("fixup worker: delalloc and fixup conflict. ino %llu start %llu", + btrfs_ino(inode), start); + btrfs_folio_clear_fixup(fs_info, folio, start, sectorsize); + continue; + } + ordered = btrfs_lookup_ordered_range(inode, start, sectorsize); + if (ordered) { + trace_btrfs_writepage_fixup_defer(inode, ordered); + btrfs_unlock_extent(&inode->io_tree, page_start, + page_end, &cached_state); + folio_unlock(folio); + btrfs_start_ordered_extent(ordered); + btrfs_put_ordered_extent(ordered); + goto again; + } + ret = btrfs_set_extent_delalloc(inode, start, + start + sectorsize - 1, 0, + &cached_state); + if (ret) + break; + trace_btrfs_writepage_fixup_reserve(inode, start, sectorsize); + btrfs_folio_clear_fixup(fs_info, folio, start, sectorsize); + set_bit(bit, delalloc_bitmap); + } + + btrfs_unlock_extent(&inode->io_tree, page_start, page_end, &cached_state); +out: + if (ret < 0) { + /* Failure here is analogous to failure in writeback. */ + mapping_set_error(folio->mapping, ret); + btrfs_folio_clear_fixup_dirty(fs_info, folio, page_start, + folio_size(folio)); + } + if (reserved) { + btrfs_delalloc_release_extents(inode, folio_size(folio)); + for_each_clear_bitrange(start_bit, end_bit, delalloc_bitmap, + blocks_per_folio) + btrfs_delalloc_release_space(inode, data_reserved, + page_start + (start_bit << fs_info->sectorsize_bits), + (end_bit - start_bit) << fs_info->sectorsize_bits, + true); + } + folio_unlock(folio); + folio_put(folio); + kfree(fixup); + extent_changeset_free(data_reserved); + btrfs_add_delayed_iput(inode); +} + +/* + * Queue space reservation fixup work for blocks dirtied without a space reservation. + * + * Should be used by writeback while holding the folio locked. + * + * If we fail to queue fixup, then the folio state is unchanged and a future + * writeback pass will still see it. + */ +void btrfs_queue_writepage_fixup(struct btrfs_inode *inode, struct folio *folio) +{ + struct btrfs_fs_info *fs_info = inode->root->fs_info; + struct btrfs_writepage_fixup *fixup; + + /* + * Disallow queueing more fixup during unmount to break the cycle + * of writeback queuing fixup queuing writeback etc. + * + * If it actually hit, then something which was fixup wasn't written + * which we should warn about. + */ + if (btrfs_fs_closing(fs_info)) { + btrfs_warn_rl(fs_info, + "dropping unqueued fixup blocks at unmount. root %lld ino %llu folio %llu", + btrfs_root_id(inode->root), btrfs_ino(inode), + folio_pos(folio)); + btrfs_folio_clear_fixup_dirty(fs_info, folio, + folio_pos(folio), folio_size(folio)); + return; + } + + fixup = kzalloc_obj(*fixup, GFP_NOFS); + if (!fixup) + return; + + /* + * This is called from within extent_write_cache_pages() which + * has successfully done an igrab(). But that will be released at the + * end of the writeback pass. We need to extend it for the worker as well. + */ + ihold(&inode->vfs_inode); + folio_get(folio); + INIT_WORK(&fixup->work, btrfs_writepage_fixup_worker); + fixup->folio = folio; + fixup->inode = inode; + queue_work(fs_info->fixup_workers, &fixup->work); +} + /* * Clear the old accounting flags and set EXTENT_DELALLOC for the range. * @@ -3938,10 +4112,11 @@ static int btrfs_read_locked_inode(struct btrfs_inode *inode, struct btrfs_path btrfs_inode_split_flags(btrfs_inode_flags(leaf, inode_item), &inode->flags, &inode->ro_flags); + +cache_index: btrfs_update_inode_mapping_flags(inode); btrfs_set_inode_mapping_order(inode); -cache_index: /* * If we were modified in the current generation and evicted from memory * and then re-read we need to do a full sync since we don't have any @@ -7507,6 +7682,12 @@ static void btrfs_invalidate_folio(struct folio *folio, size_t offset, folio_wait_writeback(folio); wait_subpage_spinlock(folio); + /* + * The invalidated blocks are going away; drop any fixup blocks among + * them, data included, as they have no space reservation. + */ + btrfs_folio_clear_fixup_dirty(fs_info, folio, page_start + offset, length); + /* * For subpage case, we have call sites like * btrfs_punch_hole_lock_range() which passes range not aligned to @@ -9954,6 +10135,7 @@ ssize_t btrfs_do_encoded_write(struct kiocb *iocb, struct iov_iter *from, if (cb) cleanup_compressed_bio(cb); out: + extent_changeset_free(data_reserved); if (ret >= 0) iocb->ki_pos += encoded->len; return ret; @@ -10548,6 +10730,41 @@ static const struct file_operations btrfs_dir_file_operations = { .setlease = generic_setlease, }; +/* + * The folio is going dirty without a btrfs delalloc space reservation. + * This requires a fixup before writeback which we might sleep so cannot + * run in this context, so we merely set state on the folio indicating it + * needs fixup before writeback. + * + * Note that there is no range in the input, so the whole folio is marked + * dirty and fixup. + * + * We believe that all callers of dirty_folio either: + * - take the folio lock (e.g. pinned folio release notification). + * - take the pte lock but must be running on a dirty pte which means + * page_mkwrite() ran on it and reserved the space. zap_pte_range() cannot + * race with writeback cleaning the folio because writeback runs + * folio_mkclean() which also uses the pte lock and revokes outstanding + * writable mappings. + * Therefore, an additional folio private lock (a la bfs->lock for all cases, + * not just subpage) is not necessary. + */ +static bool btrfs_data_dirty_folio(struct address_space *mapping, + struct folio *folio) +{ + struct btrfs_inode *inode = BTRFS_I(mapping->host); + struct btrfs_fs_info *fs_info = inode->root->fs_info; + const u64 page_start = folio_pos(folio); + const u64 range_end = min_t(u64, folio_next_pos(folio), + round_up(i_size_read(&inode->vfs_inode), + fs_info->sectorsize)); + + if (range_end > page_start) + btrfs_folio_set_fixup_dirty(fs_info, folio, page_start, + range_end - page_start); + return filemap_dirty_folio(mapping, folio); +} + /* * btrfs doesn't support the bmap operation because swapfiles * use bmap to make a mapping of extents in the file. They assume @@ -10568,7 +10785,7 @@ static const struct address_space_operations btrfs_aops = { .launder_folio = btrfs_launder_folio, .release_folio = btrfs_release_folio, .migrate_folio = btrfs_migrate_folio, - .dirty_folio = filemap_dirty_folio, + .dirty_folio = btrfs_data_dirty_folio, .error_remove_folio = generic_error_remove_folio, .swap_activate = btrfs_swap_activate, .swap_deactivate = btrfs_swap_deactivate, diff --git a/fs/btrfs/lzo.c b/fs/btrfs/lzo.c index 1531adb117d1..2f0996692da0 100644 --- a/fs/btrfs/lzo.c +++ b/fs/btrfs/lzo.c @@ -552,9 +552,10 @@ int lzo_decompress(struct list_head *ws, const u8 *data_in, size_t max_segment_len = workspace_buf_length(fs_info); int ret; - if (unlikely(srclen < LZO_LEN || srclen > max_segment_len + LZO_LEN * 2)) { + if (unlikely(srclen <= LZO_LEN * 2 || + srclen > max_segment_len + LZO_LEN * 2)) { btrfs_err(fs_info, "invalid lzo header length, has %zu expect (%u, %zu)", - srclen, LZO_LEN, max_segment_len + LZO_LEN * 2); + srclen, LZO_LEN * 2, max_segment_len + LZO_LEN * 2); return -EUCLEAN; } diff --git a/fs/btrfs/subpage.c b/fs/btrfs/subpage.c index 2a9397be8116..27dd677ca687 100644 --- a/fs/btrfs/subpage.c +++ b/fs/btrfs/subpage.c @@ -345,18 +345,57 @@ void btrfs_subpage_clear_uptodate(const struct btrfs_fs_info *fs_info, spin_unlock_irqrestore(&bfs->lock, flags); } +/* + * folio_mark_dirty() for a folio we are dirtying with a space reservation. + * + * Dirtiers without a reservation use btrfs_data_dirty_folio(). + */ +static void btrfs_folio_mark_dirty(struct folio *folio) +{ + struct address_space *mapping = folio_mapping(folio); + + if (!mapping || !mapping->host || !is_data_inode(BTRFS_I(mapping->host))) { + folio_mark_dirty(folio); + return; + } + if (folio_test_reclaim(folio)) + folio_clear_reclaim(folio); + filemap_dirty_folio(mapping, folio); +} + +/* + * The set helper of the dirty ops, so it only runs for folios without a + * fixup bitmap: for those the folio flag is the whole fixup state, and this + * reserving write covers the block, so retire it. Metadata never has the + * flag set and only pays the test. + */ +static void btrfs_folio_mark_dirty_reserved(struct folio *folio) +{ + if (folio_test_fixup_pending(folio)) + folio_clear_fixup_pending(folio); + btrfs_folio_mark_dirty(folio); +} + void btrfs_subpage_set_dirty(const struct btrfs_fs_info *fs_info, struct folio *folio, u64 start, u32 len) { struct btrfs_folio_state *bfs = folio_get_private(folio); - unsigned int start_bit = subpage_calc_start_bit(fs_info, folio, + unsigned int dirty_bit = subpage_calc_start_bit(fs_info, folio, dirty, start, len); + unsigned int fixup_bit = subpage_calc_start_bit(fs_info, folio, + fixup, start, len); + const unsigned int nbits = len >> fs_info->sectorsize_bits; unsigned long flags; spin_lock_irqsave(&bfs->lock, flags); - bitmap_set(bfs->bitmaps, start_bit, len >> fs_info->sectorsize_bits); + bitmap_set(bfs->bitmaps, dirty_bit, nbits); + /* Proper dirtying obviates the need for fixup. */ + bitmap_clear(bfs->bitmaps, fixup_bit, nbits); + if (folio_test_fixup_pending(folio) && + subpage_test_bitmap_all_zero(fs_info, folio, fixup)) + folio_clear_fixup_pending(folio); spin_unlock_irqrestore(&bfs->lock, flags); - folio_mark_dirty(folio); + btrfs_folio_mark_dirty(folio); } static void folio_clear_tags(struct folio *folio) @@ -457,6 +496,172 @@ void btrfs_subpage_clear_writeback(const struct btrfs_fs_info *fs_info, spin_unlock_irqrestore(&bfs->lock, flags); } +void btrfs_subpage_clear_fixup(const struct btrfs_fs_info *fs_info, + struct folio *folio, u64 start, u32 len) +{ + struct btrfs_folio_state *bfs = folio_get_private(folio); + unsigned int start_bit = subpage_calc_start_bit(fs_info, folio, + fixup, start, len); + unsigned long flags; + + spin_lock_irqsave(&bfs->lock, flags); + bitmap_clear(bfs->bitmaps, start_bit, len >> fs_info->sectorsize_bits); + if (subpage_test_bitmap_all_zero(fs_info, folio, fixup)) + folio_clear_fixup_pending(folio); + spin_unlock_irqrestore(&bfs->lock, flags); +} + +/* + * In one pass under bfs->lock, mark every block with a clear dirty bit in the + * range both dirty and needing fixup. + * + * Only called from the dirty_folio callback, which owns the folio-level + * dirty flag; calling folio_mark_dirty() here would recurse. + * + * The folio fixup flag and bits are both set under bfs->lock so that a + * writeback pass observing the new bits also observes the flag. + */ +static void btrfs_subpage_set_fixup_dirty(const struct btrfs_fs_info *fs_info, + struct folio *folio, u64 start, u32 len) +{ + struct btrfs_folio_state *bfs = folio_get_private(folio); + unsigned int dirty_bit = subpage_calc_start_bit(fs_info, folio, + dirty, start, len); + unsigned int fixup_bit = subpage_calc_start_bit(fs_info, folio, + fixup, start, len); + const unsigned int nbits = len >> fs_info->sectorsize_bits; + unsigned long flags; + bool marked = false; + + spin_lock_irqsave(&bfs->lock, flags); + for (unsigned int i = 0; i < nbits; i++) { + if (test_bit(dirty_bit + i, bfs->bitmaps)) + continue; + set_bit(dirty_bit + i, bfs->bitmaps); + set_bit(fixup_bit + i, bfs->bitmaps); + marked = true; + } + if (marked) + folio_set_fixup_pending(folio); + spin_unlock_irqrestore(&bfs->lock, flags); +} + +/* + * Mark the still-clean blocks of a folio dirty and needing fixup, for + * btrfs_data_dirty_folio(). + * + * A subpage block size folio that is not uptodate is left alone: its clean + * blocks may hold content that was never read in, which must not be marked + * dirty. + */ +void btrfs_folio_set_fixup_dirty(const struct btrfs_fs_info *fs_info, + struct folio *folio, u64 start, u32 len) +{ + if (!btrfs_is_subpage(fs_info, folio)) { + if (!folio_test_dirty(folio)) + folio_set_fixup_pending(folio); + return; + } + if (!folio_test_uptodate(folio)) + return; + btrfs_subpage_set_fixup_dirty(fs_info, folio, start, len); +} + +/* + * Drop the fixup blocks inside the range: clear both their fixup and dirty + * bits. + * + * Fixup blocks carry no space reservation, so their fixup and dirty bits + * must be dropped together. Clearing only the fixup bit would leave a + * dirty block without a reservation which is not a valid state. + * + * Returns true if the folio has no dirty blocks left. + */ +static bool btrfs_subpage_clear_fixup_dirty(const struct btrfs_fs_info *fs_info, + struct folio *folio, u64 start, u32 len) +{ + struct btrfs_folio_state *bfs = folio_get_private(folio); + unsigned int dirty_bit = subpage_calc_start_bit(fs_info, folio, + dirty, start, len); + unsigned int fixup_bit = subpage_calc_start_bit(fs_info, folio, + fixup, start, len); + const unsigned int nbits = len >> fs_info->sectorsize_bits; + unsigned long flags; + bool last; + + spin_lock_irqsave(&bfs->lock, flags); + for (unsigned int i = 0; i < nbits; i++) { + if (!test_bit(fixup_bit + i, bfs->bitmaps)) + continue; + clear_bit(fixup_bit + i, bfs->bitmaps); + clear_bit(dirty_bit + i, bfs->bitmaps); + } + if (subpage_test_bitmap_all_zero(fs_info, folio, fixup)) + folio_clear_fixup_pending(folio); + last = subpage_test_bitmap_all_zero(fs_info, folio, dirty); + spin_unlock_irqrestore(&bfs->lock, flags); + return last; +} + +/* + * Drop the fixup blocks inside the range, for callers discarding their data: + * btrfs_invalidate_folio() and the writepage fixup worker's error path. + * + * Callers that have just reserved space for a block want + * btrfs_folio_clear_fixup() instead - there the block stays dirty and gets + * written. + * + * The range can be byte-granular (an unaligned truncate through + * btrfs_invalidate_folio()); only blocks fully inside it are dropped, as a + * partially covered block still holds live data outside the range. For + * single-block folios the folio flag is the fixup state, so it is dropped + * only when the range covers the whole folio. + */ +void btrfs_folio_clear_fixup_dirty(const struct btrfs_fs_info *fs_info, + struct folio *folio, u64 start, u32 len) +{ + u64 aligned_start; + u64 aligned_end; + + /* The folio flag is set whenever any fixup bitmap bit is. */ + if (!folio_test_fixup_pending(folio)) + return; + if (!btrfs_is_subpage(fs_info, folio)) { + if (start <= folio_pos(folio) && + start + len >= folio_next_pos(folio)) { + folio_clear_fixup_pending(folio); + folio_clear_dirty_for_io(folio); + } + return; + } + btrfs_subpage_clamp_range(folio, &start, &len); + aligned_start = round_up(start, fs_info->sectorsize); + aligned_end = round_down(start + len, fs_info->sectorsize); + if (aligned_end <= aligned_start) + return; + if (btrfs_subpage_clear_fixup_dirty(fs_info, folio, aligned_start, + aligned_end - aligned_start)) + folio_clear_dirty_for_io(folio); +} + +bool btrfs_folio_test_fixup(const struct btrfs_fs_info *fs_info, + struct folio *folio, u64 start, u32 len) +{ + if (!btrfs_is_subpage(fs_info, folio)) + return folio_test_fixup_pending(folio); + return btrfs_subpage_test_fixup(fs_info, folio, start, len); +} + +void btrfs_folio_clear_fixup(const struct btrfs_fs_info *fs_info, + struct folio *folio, u64 start, u32 len) +{ + if (!btrfs_is_subpage(fs_info, folio)) { + folio_clear_fixup_pending(folio); + return; + } + btrfs_subpage_clear_fixup(fs_info, folio, start, len); +} + /* * Unlike set/clear which is dependent on each page status, for test all bits * are tested in the same way. @@ -480,6 +685,7 @@ bool btrfs_subpage_test_##name(const struct btrfs_fs_info *fs_info, \ IMPLEMENT_BTRFS_SUBPAGE_TEST_OP(uptodate); IMPLEMENT_BTRFS_SUBPAGE_TEST_OP(dirty); IMPLEMENT_BTRFS_SUBPAGE_TEST_OP(writeback); +IMPLEMENT_BTRFS_SUBPAGE_TEST_OP(fixup); /* * Note that, in selftests (extent-io-tests), we can have empty fs_info passed @@ -571,8 +777,8 @@ bool btrfs_meta_folio_test_##name(struct folio *folio, const struct extent_buffe } IMPLEMENT_BTRFS_PAGE_OPS(uptodate, folio_mark_uptodate, folio_clear_uptodate, folio_test_uptodate); -IMPLEMENT_BTRFS_PAGE_OPS(dirty, folio_mark_dirty, folio_clear_dirty_for_io, - folio_test_dirty); +IMPLEMENT_BTRFS_PAGE_OPS(dirty, btrfs_folio_mark_dirty_reserved, + folio_clear_dirty_for_io, folio_test_dirty); IMPLEMENT_BTRFS_PAGE_OPS(writeback, folio_start_writeback, folio_end_writeback, folio_test_writeback); diff --git a/fs/btrfs/subpage.h b/fs/btrfs/subpage.h index c6d7394e6418..9aceba93c818 100644 --- a/fs/btrfs/subpage.h +++ b/fs/btrfs/subpage.h @@ -14,15 +14,15 @@ struct folio; /* * Extra info for subpage bitmap. * - * For subpage we pack all uptodate/dirty/writeback bitmaps into + * For subpage we pack all uptodate/dirty/writeback/fixup bitmaps into * one larger bitmap. * * This structure records how they are organized in the bitmap: * - * /- uptodate /- dirty /- writeback - * | | | - * v v v - * |u|u|u|u|........|u|u|d|d|.......|d|d|w|w|.......|w|w| + * /- uptodate /- dirty /- writeback /- fixup + * | | | | + * v v v v + * |u|u|u|u|........|u|u|d|d|.......|d|d|w|w|.....|w|w|f|f|.....|f|f| * |< sectors_per_page >| * * Unlike regular macro-like enums, here we do not go upper-case names, as @@ -40,6 +40,14 @@ enum { */ btrfs_bitmap_nr_writeback, + /* + * Blocks dirtied by the dirty_folio callback instead of a reserving + * write path (e.g. set_page_dirty_lock() on a GUP pin). They have + * no space reservation and need the writepage fixup before they can + * be submitted. + */ + btrfs_bitmap_nr_fixup, + btrfs_bitmap_nr_max }; @@ -165,6 +173,29 @@ DECLARE_BTRFS_SUBPAGE_OPS(uptodate); DECLARE_BTRFS_SUBPAGE_OPS(dirty); DECLARE_BTRFS_SUBPAGE_OPS(writeback); +/* + * Fixup bit helpers. + * + * The fixup bit is data-only and has no plain set helper (setting happens + * together with dirtying in btrfs_subpage_set_fixup_dirty()), so it does not + * go through DECLARE_BTRFS_SUBPAGE_OPS(). For single-block folios the + * folio_*_fixup_pending() flag takes the place of the bitmap. + */ +void btrfs_subpage_clear_fixup(const struct btrfs_fs_info *fs_info, + struct folio *folio, u64 start, u32 len); +bool btrfs_subpage_test_fixup(const struct btrfs_fs_info *fs_info, + struct folio *folio, u64 start, u32 len); +bool btrfs_folio_test_fixup(const struct btrfs_fs_info *fs_info, + struct folio *folio, u64 start, u32 len); +void btrfs_folio_set_fixup_dirty(const struct btrfs_fs_info *fs_info, + struct folio *folio, u64 start, u32 len); +/* For a block that just got its space reserved; it stays dirty. */ +void btrfs_folio_clear_fixup(const struct btrfs_fs_info *fs_info, + struct folio *folio, u64 start, u32 len); +/* For callers discarding the data; clears the dirty bits too. */ +void btrfs_folio_clear_fixup_dirty(const struct btrfs_fs_info *fs_info, + struct folio *folio, u64 start, u32 len); + /* * Helper for error cleanup, where a folio will have its dirty flag cleared, * with writeback started and finished. diff --git a/fs/crypto/policy.c b/fs/crypto/policy.c index 9915e39362db..c80b24a941ad 100644 --- a/fs/crypto/policy.c +++ b/fs/crypto/policy.c @@ -534,7 +534,7 @@ int fscrypt_ioctl_set_policy(struct file *filp, const void __user *arg) return -EFAULT; policy.version = version; - if (!inode_owner_or_capable(&nop_mnt_idmap, inode)) + if (!inode_owner_or_capable(file_mnt_idmap(filp), inode)) return -EACCES; ret = mnt_want_write_file(filp); diff --git a/fs/nfs/nfs4proc.c b/fs/nfs/nfs4proc.c index 1360409d8de9..5709c6fea85b 100644 --- a/fs/nfs/nfs4proc.c +++ b/fs/nfs/nfs4proc.c @@ -10364,6 +10364,7 @@ static void nfs41_free_stateid_release(void *calldata) struct nfs_free_stateid_data *data = calldata; struct nfs_client *clp = data->server->nfs_client; + nfs_sb_deactive(data->server->super); nfs_put_client(clp); kfree(calldata); } @@ -10402,17 +10403,22 @@ static int nfs41_free_stateid(struct nfs_server *server, struct nfs_free_stateid_data *data; struct rpc_task *task; struct nfs_client *clp = server->nfs_client; + int ret = -EIO; if (!refcount_inc_not_zero(&clp->cl_count)) - return -EIO; + return ret; + if (!nfs_sb_active(server->super)) + goto out_put_clp; nfs4_state_protect(clp, NFS_SP4_MACH_CRED_STATEID, &task_setup.rpc_client, &msg); dprintk("NFS call free_stateid %p\n", stateid); data = kmalloc_obj(*data); - if (!data) - return -ENOMEM; + if (!data) { + ret = -ENOMEM; + goto out_put_server; + } data->server = server; nfs4_stateid_copy(&data->args.stateid, stateid); @@ -10428,6 +10434,11 @@ static int nfs41_free_stateid(struct nfs_server *server, rpc_put_task(task); stateid->type = NFS4_FREED_STATEID_TYPE; return 0; +out_put_server: + nfs_sb_deactive(server->super); +out_put_clp: + nfs_put_client(clp); + return ret; } static void @@ -10585,7 +10596,8 @@ const struct nfs4_minor_version_ops *nfs_v4_minor_ops[] = { static ssize_t nfs4_listxattr(struct dentry *dentry, char *list, size_t size) { ssize_t error, error2, error3; - size_t left = size; + ssize_t left = size; + ssize_t left2; error = generic_listxattr(dentry, list, left); if (error < 0) @@ -10595,10 +10607,13 @@ static ssize_t nfs4_listxattr(struct dentry *dentry, char *list, size_t size) left -= error; } - error2 = security_inode_listsecurity(d_inode(dentry), &list, &left); + left2 = left; + error2 = security_inode_listsecurity(d_inode(dentry), &list, &left2); if (error2 < 0) return error2; - error2 = size - error - left; + error2 = left - left2; + if (list) + left -= error2; error3 = nfs4_listxattr_nfs4_user(d_inode(dentry), list, left); if (error3 < 0) diff --git a/fs/smb/client/cifsfs.c b/fs/smb/client/cifsfs.c index 1788d93a2522..a1dacc7d8f74 100644 --- a/fs/smb/client/cifsfs.c +++ b/fs/smb/client/cifsfs.c @@ -692,6 +692,8 @@ cifs_show_options(struct seq_file *s, struct dentry *root) seq_puts(s, ",seal"); else if (tcon->ses->server->ignore_signature) seq_puts(s, ",signloosely"); + if (cifs_sb->ctx->compress) + seq_puts(s, ",compress"); if (tcon->nocase) seq_puts(s, ",nocase"); if (tcon->nodelete) diff --git a/fs/smb/client/sess.c b/fs/smb/client/sess.c index de2012cc9cf3..7cf7dd104f7c 100644 --- a/fs/smb/client/sess.c +++ b/fs/smb/client/sess.c @@ -233,9 +233,9 @@ int cifs_try_adding_channels(struct cifs_ses *ses) cifs_dbg(VFS, "failed to open extra channel on iface:%pIS rc=%d\n", &iface->sockaddr, rc); - kref_put(&iface->refcount, release_iface); /* failure to add chan should increase weight */ iface->weight_fulfilled++; + kref_put(&iface->refcount, release_iface); continue; } diff --git a/fs/smb/client/smb1transport.c b/fs/smb/client/smb1transport.c index 53abb29fe71b..966f2cf83a51 100644 --- a/fs/smb/client/smb1transport.c +++ b/fs/smb/client/smb1transport.c @@ -260,9 +260,23 @@ SendReceive(const unsigned int xid, struct cifs_ses *ses, goto out; if (out_buf) { - *pbytes_returned = resp_iov.iov_len; - if (resp_iov.iov_len) - memcpy(out_buf, resp_iov.iov_base, resp_iov.iov_len); + /* Use smbCalcSize() for both single- and multi-part T2 responses, + * both here and in coalesce_t2(). + */ + unsigned int copy_len; + if (WARN_ON_ONCE(!resp_iov.iov_base)) { + rc = -EIO; + goto out; + } + copy_len = smbCalcSize(resp_iov.iov_base); + if (copy_len > CIFSMaxBufSize + MAX_CIFS_HDR_SIZE) { + cifs_dbg(VFS, "response size %u exceeds buffer\n", + copy_len); + rc = -ENOBUFS; + goto out; + } + *pbytes_returned = copy_len; + memcpy(out_buf, resp_iov.iov_base, copy_len); } out: @@ -386,11 +400,13 @@ coalesce_t2(char *second_buf, struct smb_hdr *target_hdr, unsigned int *pdu_len) } put_bcc(byte_count, target_hdr); - byte_count = *pdu_len; - byte_count += total_in_src; + /* use smbCalcSize() rather than *pdu_len: the demux loop resets + * *pdu_len to each secondary's pdu_length, making it unreliable. + */ + byte_count = smbCalcSize(target_hdr); /* don't allow buffer to overflow */ if (byte_count > CIFSMaxBufSize + MAX_CIFS_HDR_SIZE) { - cifs_dbg(FYI, "coalesced BCC exceeds buffer size (%u)\n", + cifs_dbg(FYI, "coalesced size exceeds buffer size (%u)\n", byte_count); return -ENOBUFS; } diff --git a/fs/smb/common/compress/compress.c b/fs/smb/common/compress/compress.c index b07a317597a4..a4123c8f1c0a 100644 --- a/fs/smb/common/compress/compress.c +++ b/fs/smb/common/compress/compress.c @@ -95,6 +95,7 @@ static int smb_decompress_lz77_payload(const u8 **src, u32 *slen, u8 **dst, } static int smb_decompress_chained(__le16 alg, bool allow_chained, + bool allow_pattern, const struct smb2_compression_hdr *hdr, u32 slen, void *dst, u32 dlen) { @@ -143,6 +144,8 @@ static int smb_decompress_chained(__le16 alg, bool allow_chained, rc = smb_decompress_none(&src, &remaining, &out, &out_remaining, len); } else if (payload_alg == SMB3_COMPRESS_PATTERN) { + if (!allow_pattern) + return -EINVAL; rc = smb_decompress_pattern(&src, &remaining, &out, &out_remaining, len); } else if (payload_alg == alg && alg == SMB3_COMPRESS_LZ77) { @@ -185,6 +188,7 @@ static int smb_decompress_unchained(__le16 alg, * smb_compression_decompress() - decode an SMB2 compression transform * @alg: negotiated general-purpose compression algorithm * @allow_chained: whether chained transforms were negotiated + * @allow_pattern: whether Pattern_V1 payloads were negotiated * @src: transform header followed by compressed payload data * @slen: total number of bytes available at @src * @dst: output buffer for the reconstructed SMB2 message @@ -197,7 +201,8 @@ static int smb_decompress_unchained(__le16 alg, * Return: 0 on success, otherwise a negative errno. */ int smb_compression_decompress(__le16 alg, bool allow_chained, - const void *src, u32 slen, void *dst, u32 dlen) + bool allow_pattern, const void *src, u32 slen, + void *dst, u32 dlen) { const struct smb2_compression_hdr *hdr = src; @@ -207,8 +212,8 @@ int smb_compression_decompress(__le16 alg, bool allow_chained, return -EINVAL; if (hdr->Flags == cpu_to_le16(SMB2_COMPRESSION_FLAG_CHAINED)) - return smb_decompress_chained(alg, allow_chained, hdr, slen, - dst, dlen); + return smb_decompress_chained(alg, allow_chained, allow_pattern, + hdr, slen, dst, dlen); if (hdr->Flags != cpu_to_le16(SMB2_COMPRESSION_FLAG_NONE)) return -EINVAL; diff --git a/fs/smb/common/compress/compress.h b/fs/smb/common/compress/compress.h index 7ace3bf4b664..d6916669f887 100644 --- a/fs/smb/common/compress/compress.h +++ b/fs/smb/common/compress/compress.h @@ -20,7 +20,8 @@ static __always_inline bool smb_compress_alg_valid(__le16 alg, bool valid_none) } int smb_compression_decompress(__le16 alg, bool allow_chained, - const void *src, u32 slen, void *dst, u32 dlen); + bool allow_pattern, const void *src, u32 slen, + void *dst, u32 dlen); int smb_compression_compress_chained(__le16 alg, bool allow_pattern, const void *src, u32 slen, void *dst, u32 *dlen); diff --git a/fs/smb/server/compress.c b/fs/smb/server/compress.c index 95e48fa6b448..01d1771ff663 100644 --- a/fs/smb/server/compress.c +++ b/fs/smb/server/compress.c @@ -46,16 +46,25 @@ int ksmbd_decompress_request(struct ksmbd_conn *conn) return -EINVAL; orig_size = le32_to_cpu(hdr->OriginalCompressedSegmentSize); + /* + * For chained transforms the top-level header is only eight bytes; the + * Flags field overlays the first payload header. Reject unknown Flags + * and unnegotiated chained mode before allocating the output buffer. + */ if (hdr->Flags == cpu_to_le16(SMB2_COMPRESSION_FLAG_CHAINED)) { + if (!conn->compress_chained) + return -EINVAL; out_size = orig_size; - } else { + } else if (hdr->Flags == cpu_to_le16(SMB2_COMPRESSION_FLAG_NONE)) { offset = le32_to_cpu(hdr->Offset); if (offset > pdu_size - sizeof(*hdr) || check_add_overflow(orig_size, offset, &out_size)) return -EINVAL; + } else { + return -EINVAL; } - max_allowed_pdu_size = SMB3_MAX_MSGSIZE + conn->vals->max_write_size; + max_allowed_pdu_size = ksmbd_max_allowed_pdu_size(conn); if (out_size < sizeof(struct smb2_pdu) || out_size > max_allowed_pdu_size || out_size > MAX_STREAM_PROT_LEN) @@ -69,6 +78,7 @@ int ksmbd_decompress_request(struct ksmbd_conn *conn) *(__be32 *)out = cpu_to_be32(out_size); rc = smb_compression_decompress(conn->compress_algorithm, conn->compress_chained, + conn->compress_pattern, buf, pdu_size, out + 4, out_size); if (rc) { kvfree(out); diff --git a/fs/smb/server/connection.c b/fs/smb/server/connection.c index dee8e4aced99..ef6f202f4024 100644 --- a/fs/smb/server/connection.c +++ b/fs/smb/server/connection.c @@ -488,11 +488,7 @@ int ksmbd_conn_handler_loop(void *p) pdu_size = get_rfc1002_len(hdr_buf); ksmbd_debug(CONN, "RFC1002 header %u bytes\n", pdu_size); - if (ksmbd_conn_good(conn)) - max_allowed_pdu_size = - SMB3_MAX_MSGSIZE + conn->vals->max_write_size; - else - max_allowed_pdu_size = SMB3_MAX_MSGSIZE; + max_allowed_pdu_size = ksmbd_max_allowed_pdu_size(conn); if (pdu_size > max_allowed_pdu_size) { pr_err_ratelimited("PDU length(%u) exceeded maximum allowed pdu size(%u) on connection(%d)\n", diff --git a/fs/smb/server/connection.h b/fs/smb/server/connection.h index 2a194ee36fb4..0e4ebfac5558 100644 --- a/fs/smb/server/connection.h +++ b/fs/smb/server/connection.h @@ -210,6 +210,15 @@ static inline bool ksmbd_conn_good(struct ksmbd_conn *conn) return READ_ONCE(conn->status) == KSMBD_SESS_GOOD; } +static inline unsigned int +ksmbd_max_allowed_pdu_size(struct ksmbd_conn *conn) +{ + if (ksmbd_conn_good(conn)) + return SMB3_MAX_MSGSIZE + conn->vals->max_write_size; + + return SMB3_MAX_MSGSIZE; +} + static inline bool ksmbd_conn_need_negotiate(struct ksmbd_conn *conn) { return READ_ONCE(conn->status) == KSMBD_SESS_NEED_NEGOTIATE; diff --git a/fs/tracefs/event_inode.c b/fs/tracefs/event_inode.c index 39c7a34531e8..a52458435327 100644 --- a/fs/tracefs/event_inode.c +++ b/fs/tracefs/event_inode.c @@ -124,7 +124,17 @@ static inline void put_ei(struct eventfs_inode *ei) static inline void free_ei(struct eventfs_inode *ei) { if (ei) { + /* The ei should have no children if it is being freed. */ + WARN_ON_ONCE(!list_empty(&ei->children)); ei->is_freed = 1; + /* + * The SRCU iteration has a smp_rmb() to make sure it + * sees a child (that may have already been freed) + * before it reads is_free. If is_free is set, it must + * not use the child it acquired from ei->children, as + * the list may be used for SRCU. + */ + smp_wmb(); put_ei(ei); } } @@ -627,6 +637,20 @@ static int eventfs_iterate(struct file *file, struct dir_context *ctx) list_for_each_entry_srcu(ei_child, &ei->children, list, srcu_read_lock_held(&eventfs_srcu)) { + /* + * If the ei is being freed, then the ei->children may be + * being used as the rcu list, which means the next element + * may be garbage. The ei->is_free is set before switching + * the ei->children over to ei->rcu. The read memory barrier + * here makes sure the ei_child is read before is_free is + * updated. + * + * Matches the smp_wmb() in free_ei() + */ + smp_rmb(); + if (ei->is_freed) + return -EINVAL; + if (c > 0) { c--; continue; @@ -822,7 +846,7 @@ struct eventfs_inode *eventfs_create_events_dir(const char *name, struct dentry */ static void eventfs_remove_rec(struct eventfs_inode *ei, int level) { - struct eventfs_inode *ei_child; + struct eventfs_inode *ei_child, *tmp; /* * Check recursion depth. It should never be greater than 3: @@ -835,7 +859,7 @@ static void eventfs_remove_rec(struct eventfs_inode *ei, int level) return; /* search for nested folders or files */ - list_for_each_entry(ei_child, &ei->children, list) + list_for_each_entry_safe(ei_child, tmp, &ei->children, list) eventfs_remove_rec(ei_child, level + 1); list_del_rcu(&ei->list); diff --git a/fs/tracefs/internal.h b/fs/tracefs/internal.h index a4a7f8431aff..c61481d04c8e 100644 --- a/fs/tracefs/internal.h +++ b/fs/tracefs/internal.h @@ -46,11 +46,11 @@ struct eventfs_attr { * @ino: The saved inode number */ struct eventfs_inode { + struct list_head list; union { - struct list_head list; + struct list_head children; struct rcu_head rcu; }; - struct list_head children; const struct eventfs_entry *entries; const char *name; struct eventfs_attr *entry_attrs; diff --git a/fs/verity/measure.c b/fs/verity/measure.c index cfe2d5e535f9..68dfccb69772 100644 --- a/fs/verity/measure.c +++ b/fs/verity/measure.c @@ -122,11 +122,11 @@ __bpf_kfunc int bpf_get_fsverity_digest(struct file *file, const struct bpf_dynp { const struct bpf_dynptr_kern *digest_ptr = (struct bpf_dynptr_kern *)digest_p; const struct inode *inode = file_inode(file); - u32 dynptr_sz = __bpf_dynptr_size(digest_ptr); + u64 dynptr_sz = __bpf_dynptr_size(digest_ptr); struct fsverity_digest *arg; const struct fsverity_info *vi; const struct fsverity_hash_alg *hash_alg; - int out_digest_sz; + u64 out_digest_sz; if (dynptr_sz < sizeof(struct fsverity_digest)) return -EINVAL; @@ -144,17 +144,20 @@ __bpf_kfunc int bpf_get_fsverity_digest(struct file *file, const struct bpf_dynp hash_alg = vi->tree_params.hash_alg; + out_digest_sz = dynptr_sz - sizeof(struct fsverity_digest); + if (out_digest_sz < hash_alg->digest_size) + return -EOVERFLOW; + arg->digest_algorithm = hash_alg - fsverity_hash_algs; arg->digest_size = hash_alg->digest_size; - out_digest_sz = dynptr_sz - sizeof(struct fsverity_digest); - /* copy digest */ - memcpy(arg->digest, vi->file_digest, min_t(int, hash_alg->digest_size, out_digest_sz)); + memcpy(arg->digest, vi->file_digest, hash_alg->digest_size); /* fill the extra buffer with zeros */ if (out_digest_sz > hash_alg->digest_size) - memset(arg->digest + arg->digest_size, 0, out_digest_sz - hash_alg->digest_size); + memset(arg->digest + hash_alg->digest_size, 0, + out_digest_sz - hash_alg->digest_size); return 0; } diff --git a/fs/xfs/libxfs/xfs_exchmaps.c b/fs/xfs/libxfs/xfs_exchmaps.c index dcd0bd0b13b4..3efed37cb98a 100644 --- a/fs/xfs/libxfs/xfs_exchmaps.c +++ b/fs/xfs/libxfs/xfs_exchmaps.c @@ -959,6 +959,16 @@ xmi_can_exchange_reflink_flags( { struct xfs_mount *mp = req->ip1->i_mount; + /* + * The INO1_WRITTEN optimization can skip exchanging hole and + * unwritten mappings, which means we cannot guarantee that all + * shared extents actually moved to the other file. Clearing the + * reflink flag of an inode that still holds shared extents breaks + * the CoW write path, so refuse to exchange the flags in that case. + */ + if (req->flags & XFS_EXCHMAPS_INO1_WRITTEN) + return false; + if (hweight32(reflink_state) != 1) return false; if (req->startoff1 != 0 || req->startoff2 != 0) diff --git a/fs/xfs/libxfs/xfs_rtrefcount_btree.c b/fs/xfs/libxfs/xfs_rtrefcount_btree.c index f27b80a199ba..22acc1411aac 100644 --- a/fs/xfs/libxfs/xfs_rtrefcount_btree.c +++ b/fs/xfs/libxfs/xfs_rtrefcount_btree.c @@ -201,7 +201,7 @@ xfs_rtrefcountbt_verify( if (fa) return fa; level = be16_to_cpu(block->bb_level); - if (level > mp->m_rtrefc_maxlevels) + if (level >= mp->m_rtrefc_maxlevels) return __this_address; return xfs_btree_fsblock_verify(bp, mp->m_rtrefc_mxr[level != 0]); @@ -651,7 +651,7 @@ xfs_iformat_rtrefcount( numrecs = be16_to_cpu(dfp->bb_numrecs); level = be16_to_cpu(dfp->bb_level); - if (level > mp->m_rtrefc_maxlevels || + if (level >= mp->m_rtrefc_maxlevels || xfs_rtrefcount_droot_space_calc(level, numrecs) > dsize) { xfs_inode_mark_sick(ip, XFS_SICK_INO_CORE); return -EFSCORRUPTED; diff --git a/fs/xfs/libxfs/xfs_sb.c b/fs/xfs/libxfs/xfs_sb.c index 47322adb7690..75f2a021ee6d 100644 --- a/fs/xfs/libxfs/xfs_sb.c +++ b/fs/xfs/libxfs/xfs_sb.c @@ -1118,10 +1118,10 @@ xfs_sb_read_verify( * because _verify_common checks the on-disk values. */ __xfs_sb_from_disk(&sb, dsb, false); - error = xfs_validate_sb_common(mp, bp, &sb); + error = xfs_validate_sb_read(mp, &sb); if (error) goto out_error; - error = xfs_validate_sb_read(mp, &sb); + error = xfs_validate_sb_common(mp, bp, &sb); out_error: if (error == -EFSCORRUPTED || error == -EFSBADCRC) diff --git a/fs/xfs/scrub/agheader.c b/fs/xfs/scrub/agheader.c index 9ed053b5f061..1fa66aa68e16 100644 --- a/fs/xfs/scrub/agheader.c +++ b/fs/xfs/scrub/agheader.c @@ -18,6 +18,8 @@ #include "xfs_inode.h" #include "scrub/scrub.h" #include "scrub/common.h" +#include "scrub/bitmap.h" +#include "scrub/agino_bitmap.h" int xchk_setup_agheader( @@ -266,7 +268,7 @@ xchk_superblock( xchk_block_set_corrupt(sc, bp); if (sb->sb_gquotino != cpu_to_be64(0)) - xchk_block_set_preen(sc, bp); + xchk_block_set_corrupt(sc, bp); } else { if (sb->sb_uquotino != cpu_to_be64(mp->m_sb.sb_uquotino)) xchk_block_set_preen(sc, bp); @@ -932,41 +934,85 @@ xchk_agi_xref( /* scrub teardown will take care of sc->sa for us */ } +/* + * Walk the incore unlinked list for a particular AGI bucket to construct + * the unlinked inode bitmap for later reconstruction of the unlinked list. + * Returns 1 if we should keep checking, 0 to stop checking, or a negative + * errno. + */ +static int +xchk_iunlink_bucket( + struct xfs_scrub *sc, + unsigned int bucket, + xfs_agino_t agino) +{ + struct xagino_bitmap seen; + int ret; + + xagino_bitmap_init(&seen); + + while (agino != NULLAGINO) { + struct xfs_inode *ip; + unsigned int len = 1; + + if (agino % XFS_AGI_UNLINKED_BUCKETS != bucket) { + xchk_block_set_corrupt(sc, sc->sa.agi_bp); + goto bad; + } + + if (xagino_bitmap_test(&seen, agino, &len)) { + xchk_block_set_corrupt(sc, sc->sa.agi_bp); + goto bad; + } + + ip = xfs_iunlink_lookup(sc->sa.pag, agino); + if (!ip) { + xchk_block_set_corrupt(sc, sc->sa.agi_bp); + goto bad; + } + + if (!xfs_inode_on_unlinked_list(ip)) { + xchk_block_set_corrupt(sc, sc->sa.agi_bp); + goto bad; + } + + ret = xagino_bitmap_set(&seen, agino, 1); + if (ret) + goto out_bitmap; + + agino = ip->i_next_unlinked; + } + ret = 1; + +out_bitmap: + xagino_bitmap_destroy(&seen); + return ret; +bad: + ret = 0; + goto out_bitmap; +} + /* * Check the unlinked buckets for links to bad inodes. We hold the AGI, so * there cannot be any threads updating unlinked list pointers in this AG. */ -STATIC void +STATIC int xchk_iunlink( struct xfs_scrub *sc, struct xfs_agi *agi) { unsigned int i; - struct xfs_inode *ip; for (i = 0; i < XFS_AGI_UNLINKED_BUCKETS; i++) { - xfs_agino_t agino = be32_to_cpu(agi->agi_unlinked[i]); + int ret; - while (agino != NULLAGINO) { - if (agino % XFS_AGI_UNLINKED_BUCKETS != i) { - xchk_block_set_corrupt(sc, sc->sa.agi_bp); - return; - } - - ip = xfs_iunlink_lookup(sc->sa.pag, agino); - if (!ip) { - xchk_block_set_corrupt(sc, sc->sa.agi_bp); - return; - } - - if (!xfs_inode_on_unlinked_list(ip)) { - xchk_block_set_corrupt(sc, sc->sa.agi_bp); - return; - } - - agino = ip->i_next_unlinked; - } + ret = xchk_iunlink_bucket(sc, i, + be32_to_cpu(agi->agi_unlinked[i])); + if (ret < 1) + return ret; } + + return 0; } /* Scrub the AGI. */ @@ -1053,7 +1099,9 @@ xchk_agi( if (pag->pagi_freecount != be32_to_cpu(agi->agi_freecount)) xchk_block_set_corrupt(sc, sc->sa.agi_bp); - xchk_iunlink(sc, agi); + error = xchk_iunlink(sc, agi); + if (error) + goto out; xchk_agi_xref(sc); out: diff --git a/fs/xfs/scrub/agheader_repair.c b/fs/xfs/scrub/agheader_repair.c index 2554494847ff..2104512f1ee1 100644 --- a/fs/xfs/scrub/agheader_repair.c +++ b/fs/xfs/scrub/agheader_repair.c @@ -979,6 +979,13 @@ xrep_agi_calc_from_btrees( return error; } +/* + * Magic value that means "not unlinked" because xfarrays don't support storing + * totally zeroed elements. There can't be a cluster that starts in daddr 0 so + * there can't be an inode #1 either. + */ +#define LINKED_AGINO (0x1) + /* * Record a forwards unlinked chain pointer from agino -> next_agino in our * staging information. @@ -1034,31 +1041,40 @@ xrep_iunlink_next( * the chain or if we should stop walking the chain due to corruption; or a * per-AG inode number. */ -STATIC xfs_agino_t +STATIC int xrep_iunlink_reload_next( struct xrep_agi *ragi, xfs_agino_t prev_agino, - xfs_agino_t agino) + xfs_agino_t agino, + xfs_agino_t *next_agino) { struct xfs_scrub *sc = ragi->sc; struct xfs_inode *ip; - xfs_agino_t ret = NULLAGINO; int error; + *next_agino = NULLAGINO; + error = xchk_iget(ragi->sc, xfs_agino_to_ino(sc->sa.pag, agino), &ip); if (error) - return ret; + return 0; trace_xrep_iunlink_reload_next(ip, prev_agino); /* If this is a linked inode, stop processing the chain. */ if (VFS_I(ip)->i_nlink != 0) { - xrep_iunlink_store_next(ragi, agino, NULLAGINO); + error = xrep_iunlink_store_next(ragi, agino, NULLAGINO); + if (error) + return error; + + error = xrep_iunlink_store_prev(ragi, agino, LINKED_AGINO); + if (error) + return error; + goto rele; } ip->i_prev_unlinked = prev_agino; - ret = ip->i_next_unlinked; + *next_agino = ip->i_next_unlinked; /* * Drop the inode reference that we just took. We hold the AGI, so @@ -1067,7 +1083,7 @@ xrep_iunlink_reload_next( */ rele: xchk_irele(sc, ip); - return ret; + return 0; } /* @@ -1080,18 +1096,22 @@ xrep_iunlink_walk_ondisk_bucket( struct xrep_agi *ragi, unsigned int bucket) { + struct xagino_bitmap seen; struct xfs_scrub *sc = ragi->sc; - struct xfs_agi *agi = sc->sa.agi_bp->b_addr; + struct xfs_agi *agi = ragi->agi_bp->b_addr; xfs_agino_t prev_agino = NULLAGINO; xfs_agino_t next_agino; int error = 0; + xagino_bitmap_init(&seen); + next_agino = be32_to_cpu(agi->agi_unlinked[bucket]); while (next_agino != NULLAGINO) { xfs_agino_t agino = next_agino; + unsigned int len = 1; if (xchk_should_terminate(ragi->sc, &error)) - return error; + goto out_bitmap; trace_xrep_iunlink_walk_ondisk_bucket(sc->sa.pag, bucket, prev_agino, agino); @@ -1099,15 +1119,27 @@ xrep_iunlink_walk_ondisk_bucket( if (bucket != agino % XFS_AGI_UNLINKED_BUCKETS) break; + if (xagino_bitmap_test(&seen, agino, &len)) + break; + next_agino = xrep_iunlink_next(sc, agino); - if (!next_agino) - next_agino = xrep_iunlink_reload_next(ragi, prev_agino, - agino); + if (!next_agino) { + error = xrep_iunlink_reload_next(ragi, prev_agino, + agino, &next_agino); + if (error) + break; + } + + error = xagino_bitmap_set(&seen, agino, 1); + if (error) + goto out_bitmap; prev_agino = agino; } - return 0; +out_bitmap: + xagino_bitmap_destroy(&seen); + return error; } /* Decide if this is an unlinked inode in this AG. */ @@ -1296,7 +1328,7 @@ xrep_iunlink_mark_ondisk_rec( * iunlink_bmp. We haven't checked the inobt yet, so we don't error out if * the btree is corrupt. */ -STATIC void +STATIC int xrep_iunlink_mark_ondisk( struct xrep_agi *ragi) { @@ -1308,6 +1340,14 @@ xrep_iunlink_mark_ondisk( cur = xfs_inobt_init_cursor(sc->sa.pag, sc->tp, agi_bp); error = xfs_btree_query_all(cur, xrep_iunlink_mark_ondisk_rec, ragi); xfs_btree_del_cursor(cur, error); + + /* + * Don't proceed if we couldn't set a bit in the bitmap. All other + * errors we ignore because we haven't actually checked the inobt yet. + */ + if (error == -ENOMEM) + return -ENOMEM; + return 0; } /* @@ -1320,15 +1360,32 @@ xrep_iunlink_resolve_bucket( struct xrep_agi *ragi, unsigned int bucket) { + struct xagino_bitmap seen; struct xfs_scrub *sc = ragi->sc; struct xfs_inode *ip; xfs_agino_t prev_agino = NULLAGINO; xfs_agino_t next_agino = ragi->iunlink_heads[bucket]; int error = 0; + xagino_bitmap_init(&seen); + while (next_agino != NULLAGINO) { + unsigned int len = 1; + if (xchk_should_terminate(ragi->sc, &error)) - return error; + goto out_bitmap; + + /* Inode already seen? We're stuck in a loop */ + if (xagino_bitmap_test(&seen, next_agino, &len)) { + trace_xrep_iunlink_resolve_infinite_loop(sc->sa.pag, + bucket, prev_agino, next_agino); + next_agino = NULLAGINO; + break; + } + + error = xagino_bitmap_set(&seen, next_agino, 1); + if (error) + goto out_bitmap; /* Find the next inode in the chain. */ ip = xfs_iunlink_lookup(sc->sa.pag, next_agino); @@ -1341,6 +1398,35 @@ xrep_iunlink_resolve_bucket( break; } + if (VFS_I(ip)->i_nlink != 0) { + /* + * Inode is linked somewhere! Blow out both unlinked + * list pointers, advance the list, and pretend we + * didn't see this inode. Clear it from iunlink_bmp + * because it's linked. + */ + trace_xrep_iunlink_resolve_allocated(sc->sa.pag, + bucket, prev_agino, next_agino); + + error = xrep_iunlink_store_next(ragi, next_agino, + NULLAGINO); + if (error) + goto out_bitmap; + + error = xrep_iunlink_store_prev(ragi, next_agino, + LINKED_AGINO); + if (error) + goto out_bitmap; + + error = xagino_bitmap_clear(&ragi->iunlink_bmp, + next_agino, 1); + if (error) + goto out_bitmap; + + next_agino = ip->i_next_unlinked; + continue; + } + if (next_agino % XFS_AGI_UNLINKED_BUCKETS != bucket) { /* * Inode is in the wrong bucket. Advance the list, @@ -1376,20 +1462,20 @@ xrep_iunlink_resolve_bucket( */ error = xagino_bitmap_clear(&ragi->iunlink_bmp, next_agino, 1); if (error) - return error; + goto out_bitmap; /* Remember the previous inode's next pointer. */ if (prev_agino != NULLAGINO) { error = xrep_iunlink_store_next(ragi, prev_agino, next_agino); if (error) - return error; + goto out_bitmap; } /* Remember this inode's previous pointer. */ error = xrep_iunlink_store_prev(ragi, next_agino, prev_agino); if (error) - return error; + goto out_bitmap; /* Advance the list and remember this inode. */ prev_agino = next_agino; @@ -1400,10 +1486,12 @@ xrep_iunlink_resolve_bucket( if (prev_agino != NULLAGINO) { error = xrep_iunlink_store_next(ragi, prev_agino, next_agino); if (error) - return error; + goto out_bitmap; } - return 0; +out_bitmap: + xagino_bitmap_destroy(&seen); + return error; } /* Reinsert this unlinked inode into the head of the staged bucket list. */ @@ -1428,6 +1516,10 @@ xrep_iunlink_add_to_bucket( if (error) return error; + error = xrep_iunlink_store_prev(ragi, agino, NULLAGINO); + if (error) + return error; + /* Remember the head inode's previous pointer. */ if (current_head != NULLAGINO) { error = xrep_iunlink_store_prev(ragi, current_head, agino); @@ -1495,7 +1587,9 @@ xrep_iunlink_rebuild_buckets( * If there are ondisk inodes that are unlinked and are not been loaded * into cache, record them in iunlink_bmp. */ - xrep_iunlink_mark_ondisk(ragi); + error = xrep_iunlink_mark_ondisk(ragi); + if (error) + return error; /* * Walk each iunlink bucket to (re)construct as much of the incore list @@ -1517,6 +1611,24 @@ xrep_iunlink_rebuild_buckets( xrep_iunlink_add_lost_inodes, ragi); } +static inline void +set_inode_prev_unlinked( + struct xfs_inode *ip, + xfs_agino_t prev_agino) +{ + /* + * Magic value that means "not unlinked" because xfarrays don't support + * storing totally zeroed elements. + */ + if (prev_agino == LINKED_AGINO) + prev_agino = 0; + + if (ip->i_prev_unlinked != prev_agino) { + trace_xrep_iunlink_relink_prev(ip, prev_agino); + ip->i_prev_unlinked = prev_agino; + } +} + /* Update i_next_iunlinked for the inode @agino. */ STATIC int xrep_iunlink_relink_next( @@ -1550,8 +1662,7 @@ xrep_iunlink_relink_next( if (error) goto out_rele; - trace_xrep_iunlink_relink_prev(ip, prev_agino); - ip->i_prev_unlinked = prev_agino; + set_inode_prev_unlinked(ip, prev_agino); } /* Update the forward pointer. */ @@ -1606,7 +1717,7 @@ xrep_iunlink_relink_prev( want_rele = true; /* Set the forward pointer since this just came off disk. */ - error = xfarray_load(ragi->iunlink_prev, agino, &next_agino); + error = xfarray_load(ragi->iunlink_next, agino, &next_agino); if (error) goto out_rele; @@ -1618,11 +1729,7 @@ xrep_iunlink_relink_prev( ip->i_next_unlinked = next_agino; } - /* Update the backward pointer. */ - if (ip->i_prev_unlinked != prev_agino) { - trace_xrep_iunlink_relink_prev(ip, prev_agino); - ip->i_prev_unlinked = prev_agino; - } + set_inode_prev_unlinked(ip, prev_agino); out_rele: /* @@ -1652,6 +1759,8 @@ xrep_iunlink_commit( if (error) return error; } + if (error < 0) + return error; /* Fix all the back links */ idx = XFARRAY_CURSOR_INIT; @@ -1660,6 +1769,8 @@ xrep_iunlink_commit( if (error) return error; } + if (error < 0) + return error; /* Copy the staged iunlink buckets to the new AGI. */ for (i = 0; i < XFS_AGI_UNLINKED_BUCKETS; i++) { diff --git a/fs/xfs/scrub/alloc.c b/fs/xfs/scrub/alloc.c index 48edaa2cb1e0..c666be69f164 100644 --- a/fs/xfs/scrub/alloc.c +++ b/fs/xfs/scrub/alloc.c @@ -136,7 +136,7 @@ xchk_allocbt_rec( const union xfs_btree_rec *rec) { struct xfs_alloc_rec_incore irec; - struct xchk_alloc *ca = bs->private; + struct xchk_alloc *ca = bs->private; xfs_alloc_btrec_to_irec(rec, &irec); if (xfs_alloc_check_irec(to_perag(bs->cur->bc_group), &irec) != NULL) { @@ -144,7 +144,8 @@ xchk_allocbt_rec( return 0; } - xchk_allocbt_mergeable(bs, ca, &irec); + if (bs->sc->sm->sm_type == XFS_SCRUB_TYPE_BNOBT) + xchk_allocbt_mergeable(bs, ca, &irec); xchk_allocbt_xref(bs->sc, &irec); return 0; diff --git a/fs/xfs/scrub/attr_repair.c b/fs/xfs/scrub/attr_repair.c index be627ab655ad..6e6af142f1fb 100644 --- a/fs/xfs/scrub/attr_repair.c +++ b/fs/xfs/scrub/attr_repair.c @@ -1427,7 +1427,8 @@ xrep_xattr_rebuild_tree( * If we didn't find any attributes to salvage, repair the file by * zapping its attr fork. */ - if (rx->attrs_found == 0) { + if (rx->attrs_found == 0 && + (!xfs_has_parent(sc->mp) || xfarray_length(rx->pptr_recs) == 0)) { xfs_trans_ijoin(sc->tp, sc->ip, 0); error = xrep_xattr_reset_fork(sc); if (error) diff --git a/fs/xfs/scrub/bmap.c b/fs/xfs/scrub/bmap.c index 70028da1aacc..401c278725d2 100644 --- a/fs/xfs/scrub/bmap.c +++ b/fs/xfs/scrub/bmap.c @@ -1170,6 +1170,11 @@ xchk_bmap_attr( } error = xchk_bmap(sc, XFS_ATTR_FORK); + /* A repaired, empty attr fork no longer has mappings to check. */ + if (error == -ENOENT && (sc->flags & XREP_ALREADY_FIXED)) { + xchk_mark_healthy_if_clean(sc, XFS_SICK_INO_BMBTA_ZAPPED); + return 0; + } if (error) return error; diff --git a/fs/xfs/scrub/dirtree_repair.c b/fs/xfs/scrub/dirtree_repair.c index 1c0d7ea4a5be..bbf6acf6fd40 100644 --- a/fs/xfs/scrub/dirtree_repair.c +++ b/fs/xfs/scrub/dirtree_repair.c @@ -349,6 +349,8 @@ xrep_dirtree_unlink_iolock( ASSERT(sc->ilock_flags & XFS_IOLOCK_EXCL); + if (sc->ip == dp) + return 0; if (xfs_ilock_nowait(dp, XFS_IOLOCK_EXCL)) return 0; @@ -400,8 +402,18 @@ xrep_dirtree_unlink( * directory code can handle a reservationless update. */ resblks = xfs_remove_space_res(mp, step->name_len); - error = xfs_trans_alloc_dir(dp, &M_RES(mp)->tr_remove, sc->ip, - &resblks, &sc->tp, &dontcare); + if (sc->ip == dp) { +again: + error = xfs_trans_alloc_inode(dp, &M_RES(mp)->tr_remove, + resblks, 0, false, &sc->tp); + if ((error == -ENOSPC || error == -EDQUOT) && resblks > 0) { + resblks = 0; + goto again; + } + } else { + error = xfs_trans_alloc_dir(dp, &M_RES(mp)->tr_remove, sc->ip, + &resblks, &sc->tp, &dontcare); + } if (error) goto out_iolock; @@ -489,9 +501,11 @@ xrep_dirtree_unlink( xchk_trans_cancel(sc); out_ilock: xfs_iunlock(sc->ip, XFS_ILOCK_EXCL); - xfs_iunlock(dp, XFS_ILOCK_EXCL); + if (dp != sc->ip) + xfs_iunlock(dp, XFS_ILOCK_EXCL); out_iolock: - xfs_iunlock(dp, XFS_IOLOCK_EXCL); + if (dp != sc->ip) + xfs_iunlock(dp, XFS_IOLOCK_EXCL); return error; } diff --git a/fs/xfs/scrub/inode_repair.c b/fs/xfs/scrub/inode_repair.c index 3ec41c198351..8bc508336aa5 100644 --- a/fs/xfs/scrub/inode_repair.c +++ b/fs/xfs/scrub/inode_repair.c @@ -1797,7 +1797,7 @@ xrep_inode_flags( /* Clear junk flags */ if (sc->ip->i_diflags & ~XFS_DIFLAG_ANY) - sc->ip->i_diflags &= ~XFS_DIFLAG_ANY; + sc->ip->i_diflags &= XFS_DIFLAG_ANY; /* NEWRTBM only applies to realtime bitmaps */ if (I_INO(sc->ip) == sc->mp->m_sb.sb_rbmino) @@ -1828,7 +1828,7 @@ xrep_inode_flags( /* Clear junk flags. */ if (sc->ip->i_diflags2 & ~XFS_DIFLAG2_ANY) - sc->ip->i_diflags2 &= ~XFS_DIFLAG2_ANY; + sc->ip->i_diflags2 &= XFS_DIFLAG2_ANY; /* No reflink flag unless we support it and it's a file. */ if (!xfs_has_reflink(sc->mp) || !S_ISREG(mode)) @@ -1960,7 +1960,7 @@ xrep_inode_cowextsize( /* Fix misaligned CoW extent size hints on a directory. */ if ((sc->ip->i_diflags & XFS_DIFLAG_RTINHERIT) && (sc->ip->i_diflags2 & XFS_DIFLAG2_COWEXTSIZE) && - sc->ip->i_extsize % sc->mp->m_sb.sb_rextsize > 0) { + xfs_extlen_to_rtxmod(sc->mp, sc->ip->i_cowextsize) > 0) { sc->ip->i_cowextsize = 0; sc->ip->i_diflags2 &= ~XFS_DIFLAG2_COWEXTSIZE; } diff --git a/fs/xfs/scrub/nlinks.c b/fs/xfs/scrub/nlinks.c index 355ab6de23ea..bcedb8c3e4e6 100644 --- a/fs/xfs/scrub/nlinks.c +++ b/fs/xfs/scrub/nlinks.c @@ -382,6 +382,12 @@ xchk_nlinks_ilock_dir( { uint lock_mode = XFS_ILOCK_SHARED; + /* + * Take the IOLOCK so that other threads cannot start a directory + * update while we're scanning. + */ + xfs_ilock(ip, XFS_IOLOCK_SHARED); + /* * We're going to scan the directory entries, so we must be ready to * pull the data fork mappings into memory if they aren't already. @@ -397,13 +403,8 @@ xchk_nlinks_ilock_dir( xfs_need_iread_extents(&ip->i_af)) lock_mode = XFS_ILOCK_EXCL; - /* - * Take the IOLOCK so that other threads cannot start a directory - * update while we're scanning. - */ - lock_mode |= XFS_IOLOCK_SHARED; xfs_ilock(ip, lock_mode); - return lock_mode; + return lock_mode | XFS_IOLOCK_SHARED; } /* Walk a directory to bump the observed link counts of the children. */ diff --git a/fs/xfs/scrub/nlinks_repair.c b/fs/xfs/scrub/nlinks_repair.c index fbc2ff809fc0..09e097e16689 100644 --- a/fs/xfs/scrub/nlinks_repair.c +++ b/fs/xfs/scrub/nlinks_repair.c @@ -232,9 +232,14 @@ xrep_nlinks_repair_inode( * unlinked list, put it on the unlinked list. */ if (total_links == 0 && !xfs_inode_on_unlinked_list(ip)) { + if (actual_nlink) + clear_nlink(VFS_I(ip)); error = xfs_iunlink(sc->tp, ip); - if (error) + if (error) { + if (actual_nlink) + set_nlink(VFS_I(ip), actual_nlink); goto out_trans; + } dirty = true; } diff --git a/fs/xfs/scrub/parent.c b/fs/xfs/scrub/parent.c index a8c4807e1d94..99b60773b715 100644 --- a/fs/xfs/scrub/parent.c +++ b/fs/xfs/scrub/parent.c @@ -485,7 +485,7 @@ xchk_parent_scan_attr( valuelen, &parent_ino, NULL); if (error) { xchk_fblock_set_corrupt(sc, XFS_ATTR_FORK, 0); - return error; + return -ECANCELED; } /* No self-referential parent pointers. */ diff --git a/fs/xfs/scrub/rtbitmap_repair.c b/fs/xfs/scrub/rtbitmap_repair.c index dc64902d6c25..442a17bf9720 100644 --- a/fs/xfs/scrub/rtbitmap_repair.c +++ b/fs/xfs/scrub/rtbitmap_repair.c @@ -36,6 +36,24 @@ /* rt bitmap content repairs */ +/* + * Reserve enough blocks to write out a completely new bitmap file, plus twice + * as many blocks as we would need if we can only allocate one block per data + * fork mapping. This should cover the preallocation of the temporary file and + * exchanging the extent mappings. + * + * We cannot use xfs_exchmaps_estimate because we have not yet constructed the + * replacement bitmap and therefore do not know how many extents it will use. + * By the time we do, we will have a dirty transaction (which we cannot drop + * because we cannot drop the rtbitmap ILOCK) and cannot ask for more + * reservation. + */ +static inline unsigned long long +xrep_rtbitmap_calc_blocks(struct xfs_mount *mp, unsigned long long blocks) +{ + return blocks + (xfs_bmbt_calc_size(mp, blocks) * 2); +} + /* Set up to repair the realtime bitmap for this group. */ int xrep_setup_rtbitmap( @@ -56,20 +74,7 @@ xrep_setup_rtbitmap( if (error) return error; - /* - * Reserve enough blocks to write out a completely new bitmap file, - * plus twice as many blocks as we would need if we can only allocate - * one block per data fork mapping. This should cover the - * preallocation of the temporary file and exchanging the extent - * mappings. - * - * We cannot use xfs_exchmaps_estimate because we have not yet - * constructed the replacement bitmap and therefore do not know how - * many extents it will use. By the time we do, we will have a dirty - * transaction (which we cannot drop because we cannot drop the - * rtbitmap ILOCK) and cannot ask for more reservation. - */ - blocks += xfs_bmbt_calc_size(mp, blocks) * 2; + blocks = xrep_rtbitmap_calc_blocks(mp, mp->m_sb.sb_rbmblocks); if (blocks > UINT_MAX) return -EOPNOTSUPP; @@ -512,7 +517,7 @@ xrep_rtbitmap( struct xchk_rtbitmap *rtb = sc->buf; struct xfs_mount *mp = sc->mp; struct xfs_group *xg = rtg_group(sc->sr.rtg); - unsigned long long blocks = 0; + unsigned long long blocks; unsigned int busy_gen; int error; @@ -532,15 +537,20 @@ xrep_rtbitmap( * figure out if we need to adjust the block reservation in the * transaction. */ - blocks = xfs_bmbt_calc_size(mp, rtb->rbmblocks); + blocks = xrep_rtbitmap_calc_blocks(mp, rtb->rbmblocks); if (blocks > UINT_MAX) return -EOPNOTSUPP; if (blocks > rtb->resblks) { - error = xfs_trans_reserve_more(sc->tp, blocks, 0); + uint64_t delta = blocks - rtb->resblks; + + if (delta > UINT_MAX) + return -EOPNOTSUPP; + + error = xfs_trans_reserve_more(sc->tp, delta, 0); if (error) return error; - rtb->resblks += blocks; + rtb->resblks += delta; } /* Fix inode core and forks. */ diff --git a/fs/xfs/scrub/rtsummary.c b/fs/xfs/scrub/rtsummary.c index 78f72a046887..546b335ade13 100644 --- a/fs/xfs/scrub/rtsummary.c +++ b/fs/xfs/scrub/rtsummary.c @@ -358,7 +358,7 @@ xchk_rtsummary( * EFSCORRUPTED means the rtbitmap is corrupt, which is an xref * error since we're checking the summary file. */ - xchk_ip_set_corrupt(sc, rbmip); + xchk_ip_xref_set_corrupt(sc, rbmip); return 0; } if (error) diff --git a/fs/xfs/scrub/tempfile.c b/fs/xfs/scrub/tempfile.c index e0c630f888cf..98820003b929 100644 --- a/fs/xfs/scrub/tempfile.c +++ b/fs/xfs/scrub/tempfile.c @@ -174,6 +174,7 @@ xrep_tempfile_create( xfs_iunlock(sc->tempip, XFS_ILOCK_EXCL); xfs_finish_inode_setup(sc->tempip); xchk_irele(sc, sc->tempip); + sc->tempip = NULL; } out_release_dquots: xfs_qm_dqrele(udqp); diff --git a/fs/xfs/scrub/trace.h b/fs/xfs/scrub/trace.h index d5d39d82749e..14aa0ec1f09e 100644 --- a/fs/xfs/scrub/trace.h +++ b/fs/xfs/scrub/trace.h @@ -3538,10 +3538,12 @@ DEFINE_EVENT(xrep_iunlink_resolve_class, name, \ TP_PROTO(const struct xfs_perag *pag, unsigned int bucket, \ xfs_agino_t prev_agino, xfs_agino_t next_agino), \ TP_ARGS(pag, bucket, prev_agino, next_agino)) +DEFINE_REPAIR_IUNLINK_RESOLVE_EVENT(xrep_iunlink_resolve_infinite_loop); DEFINE_REPAIR_IUNLINK_RESOLVE_EVENT(xrep_iunlink_resolve_uncached); DEFINE_REPAIR_IUNLINK_RESOLVE_EVENT(xrep_iunlink_resolve_wronglist); DEFINE_REPAIR_IUNLINK_RESOLVE_EVENT(xrep_iunlink_resolve_nolist); DEFINE_REPAIR_IUNLINK_RESOLVE_EVENT(xrep_iunlink_resolve_ok); +DEFINE_REPAIR_IUNLINK_RESOLVE_EVENT(xrep_iunlink_resolve_allocated); TRACE_EVENT(xrep_iunlink_relink_next, TP_PROTO(struct xfs_inode *ip, xfs_agino_t next_agino), diff --git a/fs/xfs/xfs_buf.c b/fs/xfs/xfs_buf.c index e1465e950acc..48d7dfd3e15f 100644 --- a/fs/xfs/xfs_buf.c +++ b/fs/xfs/xfs_buf.c @@ -114,7 +114,7 @@ xfs_buf_free( vfree(bp->b_addr); else if (bp->b_flags & _XBF_KMEM) kfree(bp->b_addr); - else + else if (bp->b_addr) folio_put(virt_to_folio(bp->b_addr)); call_rcu(&bp->b_rcu, xfs_buf_free_callback); diff --git a/fs/xfs/xfs_buf_item_recover.c b/fs/xfs/xfs_buf_item_recover.c index 02b95b89d1b5..240deb3f7827 100644 --- a/fs/xfs/xfs_buf_item_recover.c +++ b/fs/xfs/xfs_buf_item_recover.c @@ -461,7 +461,7 @@ xlog_recover_validate_buf_type( * given buffer. The bitmap in the buf log format structure indicates * where to place the logged data. */ -STATIC void +STATIC int xlog_recover_do_reg_buffer( struct xfs_mount *mp, struct xlog_recover_item *item, @@ -489,8 +489,24 @@ xlog_recover_do_reg_buffer( ASSERT(nbits > 0); ASSERT(item->ri_buf[i].iov_base != NULL); ASSERT(item->ri_buf[i].iov_len % XFS_BLF_CHUNK == 0); - ASSERT(BBTOB(bp->b_length) >= - ((uint)bit << XFS_BLF_SHIFT) + (nbits << XFS_BLF_SHIFT)); + /* + * The bitmap is only trustworthy to the extent that it + * describes a region that actually fits inside the buffer we + * read in based on the (attacker-controlled) blf_len. Do not + * rely on an ASSERT() for this -- it compiles away entirely on + * non-DEBUG kernels, which is exactly where this matters, so + * validate it for real and abort recovery of this buffer rather + * than copying past the end of it. + */ + if (XFS_IS_CORRUPT(mp, BBTOB(bp->b_length) < + ((uint)bit << XFS_BLF_SHIFT) + + (nbits << XFS_BLF_SHIFT))) { + xfs_alert(mp, + "Bad buffer log item dirty bitmap (bit %d, nbits %d) for %d-byte buffer at daddr 0x%llx.", + bit, nbits, BBTOB(bp->b_length), + xfs_buf_daddr(bp)); + return -EFSCORRUPTED; + } /* * The dirty regions logged in the buffer, even though @@ -544,6 +560,7 @@ xlog_recover_do_reg_buffer( ASSERT(i == item->ri_total); xlog_recover_validate_buf_type(mp, bp, buf_f, current_lsn); + return 0; } /* @@ -552,10 +569,10 @@ xlog_recover_do_reg_buffer( * (ie. USR or GRP), then just toss this buffer away; don't recover it. * Else, treat it as a regular buffer and do recovery. * - * Return false if the buffer was tossed and true if we recovered the buffer to - * indicate to the caller if the buffer needs writing. + * Return 0 if the buffer was not recovered (tossed), 1 if it was recovered and + * needs writing, or a negative errno if recovery of the buffer failed. */ -STATIC bool +STATIC int xlog_recover_do_dquot_buffer( struct xfs_mount *mp, struct xlog *log, @@ -564,6 +581,7 @@ xlog_recover_do_dquot_buffer( struct xfs_buf_log_format *buf_f) { uint type; + int error; trace_xfs_log_recover_buf_dquot_buf(log, buf_f); @@ -571,7 +589,7 @@ xlog_recover_do_dquot_buffer( * Filesystems are required to send in quota flags at mount time. */ if (!mp->m_qflags) - return false; + return 0; type = 0; if (buf_f->blf_flags & XFS_BLF_UDQUOT_BUF) @@ -584,10 +602,12 @@ xlog_recover_do_dquot_buffer( * This type of quotas was turned off, so ignore this buffer */ if (log->l_quotaoffs_flag & type) - return false; + return 0; - xlog_recover_do_reg_buffer(mp, item, bp, buf_f, NULLCOMMITLSN); - return true; + error = xlog_recover_do_reg_buffer(mp, item, bp, buf_f, NULLCOMMITLSN); + if (error) + return error; + return 1; } /* @@ -724,7 +744,9 @@ xlog_recover_do_primary_sb_buffer( xfs_rgnumber_t orig_rgcount = mp->m_sb.sb_rgcount; int error; - xlog_recover_do_reg_buffer(mp, item, bp, buf_f, current_lsn); + error = xlog_recover_do_reg_buffer(mp, item, bp, buf_f, current_lsn); + if (error) + return error; if (orig_agcount == 0) { xfs_alert(mp, "Trying to grow file system without AGs"); @@ -1081,11 +1103,11 @@ xlog_recover_buf_commit_pass2( goto out_release; } else if (buf_f->blf_flags & (XFS_BLF_UDQUOT_BUF|XFS_BLF_PDQUOT_BUF|XFS_BLF_GDQUOT_BUF)) { - bool dirty; - - dirty = xlog_recover_do_dquot_buffer(mp, log, item, bp, buf_f); - if (!dirty) + error = xlog_recover_do_dquot_buffer(mp, log, item, bp, buf_f); + if (error <= 0) goto out_release; + /* write dirty buffer */ + error = 0; } else if ((xfs_blft_from_flags(buf_f) & XFS_BLFT_SB_BUF) && xfs_buf_daddr(bp) == 0) { error = xlog_recover_do_primary_sb_buffer(mp, item, bp, buf_f, @@ -1105,7 +1127,10 @@ xlog_recover_buf_commit_pass2( xfs_buf_relse(rtsb_bp); } } else { - xlog_recover_do_reg_buffer(mp, item, bp, buf_f, current_lsn); + error = xlog_recover_do_reg_buffer(mp, item, bp, buf_f, + current_lsn); + if (error) + goto out_release; } /* diff --git a/fs/xfs/xfs_dquot.c b/fs/xfs/xfs_dquot.c index c311f61d9554..b4f6c594808c 100644 --- a/fs/xfs/xfs_dquot.c +++ b/fs/xfs/xfs_dquot.c @@ -778,7 +778,7 @@ xfs_dq_get_next_id( lock_flags = xfs_ilock_data_map_shared(quotip); error = xfs_iread_extents(NULL, quotip, XFS_DATA_FORK); if (error) - return error; + goto out_unlock; if (xfs_iext_lookup_extent(quotip, "ip->i_df, start, &cur, &got)) { /* contiguous chunk, bump startoff for the id calculation */ @@ -789,6 +789,7 @@ xfs_dq_get_next_id( error = -ENOENT; } +out_unlock: xfs_iunlock(quotip, lock_flags); return error; diff --git a/fs/xfs/xfs_dquot_item_recover.c b/fs/xfs/xfs_dquot_item_recover.c index fe419b28de22..63bc9ab7d947 100644 --- a/fs/xfs/xfs_dquot_item_recover.c +++ b/fs/xfs/xfs_dquot_item_recover.c @@ -173,7 +173,7 @@ xlog_recover_dquot_commit_pass2( out_release: xfs_buf_relse(bp); - return 0; + return error; } const struct xlog_recover_item_ops xlog_dquot_item_ops = { diff --git a/fs/xfs/xfs_mount.h b/fs/xfs/xfs_mount.h index 66a02d1b9ad7..216a38a354e7 100644 --- a/fs/xfs/xfs_mount.h +++ b/fs/xfs/xfs_mount.h @@ -349,6 +349,13 @@ typedef struct xfs_mount { /* Index of uuid record in the uuid xarray. */ unsigned int m_uuid_table_index; + + /* + * Old io_pages/ra_pages valued in the main bdev BDI, and our initial + * calculated values. + */ + unsigned long m_old_io_pages, m_initial_io_pages; + unsigned long m_old_ra_pages, m_initial_ra_pages; } xfs_mount_t; #define M_IGEO(mp) (&(mp)->m_ino_geo) diff --git a/fs/xfs/xfs_rtalloc.c b/fs/xfs/xfs_rtalloc.c index 7a3f97686989..84efe5a8fb11 100644 --- a/fs/xfs/xfs_rtalloc.c +++ b/fs/xfs/xfs_rtalloc.c @@ -737,7 +737,7 @@ xfs_rtginode_ensure( xfs_trans_cancel(tp); if (error != -ENOENT) - return 0; + return error; return xfs_rtginode_create(rtg, type, true); } diff --git a/fs/xfs/xfs_super.c b/fs/xfs/xfs_super.c index 8531d526fc44..63c4bcbe6c2b 100644 --- a/fs/xfs/xfs_super.c +++ b/fs/xfs/xfs_super.c @@ -548,6 +548,52 @@ xfs_open_devices( return error; } +/* + * When using a RT device some or all data I/O is using the RT device, but + * the BDI is inherited from the main data device. When the underlying block + * device for the RT device has larger I/O sizes, the BDI settings might be + * incorrect, which is especially bad if the main device is a SSD and the + * RT device is a HDD, as the io_opt fixup in blk_apply_bdi_limits is missing + * for this case. + * + * Update the BDI values to the max of the data and RT device to cover our + * bases. + */ +static void +xfs_update_bdi_rahead( + struct xfs_mount *mp) +{ + struct backing_dev_info *rt_bdi = + mp->m_rtdev_targp->bt_bdev->bd_disk->bdi; + struct backing_dev_info *sb_bdi = mp->m_super->s_bdi; + + mp->m_old_io_pages = sb_bdi->io_pages; + mp->m_old_ra_pages = sb_bdi->ra_pages; + + sb_bdi->io_pages = mp->m_initial_io_pages = + max(sb_bdi->io_pages, rt_bdi->io_pages); + sb_bdi->ra_pages = mp->m_initial_ra_pages = + max(sb_bdi->ra_pages, rt_bdi->ra_pages); +} + +static void +xfs_restore_bdi_rahead( + struct xfs_mount *mp) +{ + struct backing_dev_info *sb_bdi = mp->m_super->s_bdi; + + if (sb_bdi->io_pages == mp->m_initial_io_pages) + sb_bdi->io_pages = mp->m_old_io_pages; + else + xfs_info(mp, "io_pages changed from %lu to %lu, not restoring.", + mp->m_initial_io_pages, sb_bdi->io_pages); + if (sb_bdi->ra_pages == mp->m_initial_ra_pages) + sb_bdi->ra_pages = mp->m_old_ra_pages; + else + xfs_info(mp, "ra_pages changed from %lu to %lu, not restoring.", + mp->m_initial_ra_pages, sb_bdi->ra_pages); +} + /* * Setup xfs_mount buffer target pointers based on superblock */ @@ -585,6 +631,7 @@ xfs_setup_devices( mp->m_sb.sb_sectsize, mp->m_sb.sb_rblocks); if (error) return error; + xfs_update_bdi_rahead(mp); } return 0; @@ -2283,8 +2330,12 @@ static void xfs_kill_sb( struct super_block *sb) { + struct xfs_mount *mp = XFS_M(sb); + + if (mp->m_rtdev_targp && mp->m_rtdev_targp != mp->m_ddev_targp) + xfs_restore_bdi_rahead(mp); kill_block_super(sb); - xfs_mount_free(XFS_M(sb)); + xfs_mount_free(mp); } static struct file_system_type xfs_fs_type = { diff --git a/fs/xfs/xfs_zone_gc.c b/fs/xfs/xfs_zone_gc.c index f76a09130852..7ab8f2218c6a 100644 --- a/fs/xfs/xfs_zone_gc.c +++ b/fs/xfs/xfs_zone_gc.c @@ -103,6 +103,7 @@ struct xfs_gc_bio { /* Open Zone being written to */ struct xfs_open_zone *oz; + /* Realtime group currently being reclaimed */ struct xfs_rtgroup *victim_rtg; /* Bio used for reads and writes, including the bvec used by it */ @@ -130,6 +131,9 @@ struct xfs_zone_gc_data { /* bioset used to allocate the gc_bios */ struct bio_set bio_set; + /* bioset used when writes need to be split to hardware limits */ + struct bio_set split_bio_set; + /* * Scratchpad to buffer GC data, organized as a ring buffer over * discontiguous folios. scratch_head is where the buffer is filled, @@ -221,6 +225,9 @@ xfs_zone_gc_data_alloc( if (bioset_init(&data->bio_set, 16, offsetof(struct xfs_gc_bio, bio), BIOSET_NEED_BVECS)) goto out_free_recs; + if (bioset_init(&data->split_bio_set, 16, + offsetof(struct xfs_gc_bio, bio), 0)) + goto out_exit_bio_set; for (i = 0; i < XFS_GC_NR_BUFS; i++) { data->scratch_folios[i] = folio_alloc(GFP_KERNEL, get_order(XFS_GC_BUF_SIZE)); @@ -238,6 +245,8 @@ xfs_zone_gc_data_alloc( out_free_scratch: while (--i >= 0) folio_put(data->scratch_folios[i]); + bioset_exit(&data->split_bio_set); +out_exit_bio_set: bioset_exit(&data->bio_set); out_free_recs: kfree(data->iter.recs); @@ -254,6 +263,7 @@ xfs_zone_gc_data_free( for (i = 0; i < XFS_GC_NR_BUFS; i++) folio_put(data->scratch_folios[i]); + bioset_exit(&data->split_bio_set); bioset_exit(&data->bio_set); kfree(data->iter.recs); kfree(data); @@ -810,7 +820,8 @@ xfs_zone_gc_split_write( data->mp->m_sb.sb_blocksize) >> SECTOR_SHIFT; split_len = split_sectors << SECTOR_SHIFT; - split = bio_split(&chunk->bio, split_sectors, GFP_NOFS, &data->bio_set); + split = bio_split(&chunk->bio, split_sectors, GFP_NOFS, + &data->split_bio_set); split_chunk = container_of(split, struct xfs_gc_bio, bio); split_chunk->data = data; ihold(VFS_I(chunk->ip)); diff --git a/include/linux/device.h b/include/linux/device.h index 7b2baffdd2f5..aee79fd6b32b 100644 --- a/include/linux/device.h +++ b/include/linux/device.h @@ -99,7 +99,9 @@ struct device_type { * struct device_attribute - Interface for exporting device attributes. * @attr: sysfs attribute definition. * @show: Show handler. + * @show_const: Show handler (read-only). * @store: Store handler. + * @store_const: Store handler (read-only). */ struct device_attribute { struct attribute attr; diff --git a/include/linux/dibs.h b/include/linux/dibs.h index c75607f8a5cf..d3e0777f25ae 100644 --- a/include/linux/dibs.h +++ b/include/linux/dibs.h @@ -439,7 +439,7 @@ static inline void *dibs_get_priv(struct dibs_dev *dev, /** * dibs_dev_alloc() - allocate and reference device structure * - * The following fields will be valid upon successful return: dev + * The following fields will be valid upon successful return: dev, lock * NOTE: Use put_device(dibs_get_dev(@dibs)) to give up your reference instead * of freeing @dibs @dev directly once you have successfully called this * function. diff --git a/include/linux/fixp-arith.h b/include/linux/fixp-arith.h index e485fb0c1201..4d4f4a7d4749 100644 --- a/include/linux/fixp-arith.h +++ b/include/linux/fixp-arith.h @@ -40,7 +40,7 @@ static const s32 sin_table[] = { }; /** - * __fixp_sin32() returns the sin of an angle in degrees + * __fixp_sin32() - returns the sin of an angle in degrees * * @degrees: angle, in degrees, from 0 to 360. * @@ -64,7 +64,7 @@ static inline s32 __fixp_sin32(int degrees) } /** - * fixp_sin32() returns the sin of an angle in degrees + * fixp_sin32() - returns the sin of an angle in degrees * * @degrees: angle, in degrees. The angle can be positive or negative * diff --git a/include/linux/hwmon.h b/include/linux/hwmon.h index 77a6f2bffcba..dd713e193d0c 100644 --- a/include/linux/hwmon.h +++ b/include/linux/hwmon.h @@ -480,8 +480,7 @@ hwmon_device_register_with_info(struct device *dev, const struct attribute_group **extra_groups); struct device * hwmon_device_register_for_thermal(struct device *dev, const char *name, - void *drvdata, - const struct attribute_group **extra_groups); + void *drvdata); struct device * devm_hwmon_device_register_with_info(struct device *dev, const char *name, void *drvdata, diff --git a/include/linux/mmap_lock.h b/include/linux/mmap_lock.h index 04b8f61ece5d..6b5c2390cc30 100644 --- a/include/linux/mmap_lock.h +++ b/include/linux/mmap_lock.h @@ -621,6 +621,7 @@ static inline void mmap_read_unlock(struct mm_struct *mm) DEFINE_GUARD(mmap_read_lock, struct mm_struct *, mmap_read_lock(_T), mmap_read_unlock(_T)) +DEFINE_GUARD_COND(mmap_read_lock, _try, mmap_read_trylock(_T)) static inline void mmap_read_unlock_non_owner(struct mm_struct *mm) { diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h index 9981d637f8b5..8840b126979f 100644 --- a/include/linux/netdevice.h +++ b/include/linux/netdevice.h @@ -300,9 +300,11 @@ struct hh_cache { * We could use other alignment values, but we must maintain the * relationship HH alignment <= LL alignment. */ -#define LL_RESERVED_SPACE(dev) \ - ((((dev)->hard_header_len + READ_ONCE((dev)->needed_headroom)) \ +#define LL_RESERVED_SPACE_EX(dev, hlen) \ + ((((hlen) + READ_ONCE((dev)->needed_headroom)) \ & ~(HH_DATA_MOD - 1)) + HH_DATA_MOD) +#define LL_RESERVED_SPACE(dev) \ + LL_RESERVED_SPACE_EX(dev, (dev)->hard_header_len) #define LL_RESERVED_SPACE_EXTRA(dev,extra) \ ((((dev)->hard_header_len + READ_ONCE((dev)->needed_headroom) + (extra)) \ & ~(HH_DATA_MOD - 1)) + HH_DATA_MOD) @@ -3531,11 +3533,6 @@ static inline bool dev_validate_header(const struct net_device *dev, if (len < dev->min_header_len) return false; - if (capable(CAP_SYS_RAWIO)) { - memset(ll_header + len, 0, dev->hard_header_len - len); - return true; - } - if (dev->header_ops && dev->header_ops->validate) return dev->header_ops->validate(ll_header, len); diff --git a/include/linux/netfilter/ipset/ip_set.h b/include/linux/netfilter/ipset/ip_set.h index b98331572ad2..c46864cc6623 100644 --- a/include/linux/netfilter/ipset/ip_set.h +++ b/include/linux/netfilter/ipset/ip_set.h @@ -244,8 +244,8 @@ extern void ip_set_type_unregister(struct ip_set_type *set_type); /* A generic IP set */ struct ip_set { - /* For call_cru in destroy */ - struct rcu_head rcu; + /* for set destruction */ + struct rcu_work rwork; /* The name of the set */ char name[IPSET_MAXNAMELEN]; /* Lock protecting the set data */ @@ -273,7 +273,7 @@ struct ip_set { /* Number of elements (vs timeout) */ u32 elements; /* Size of the dynamic extensions (vs timeout) */ - size_t ext_size; + atomic64_t ext_size; /* Element data size */ size_t dsize; /* Offsets to extensions in elements */ diff --git a/include/linux/nvmem-provider.h b/include/linux/nvmem-provider.h index f3b13da78aac..6063fe5b7784 100644 --- a/include/linux/nvmem-provider.h +++ b/include/linux/nvmem-provider.h @@ -214,6 +214,12 @@ static inline int nvmem_layout_register(struct nvmem_layout *layout) static inline void nvmem_layout_unregister(struct nvmem_layout *layout) {} +static inline int nvmem_add_cells_from_dt(struct nvmem_device *nvmem, + struct device_node *np) +{ + return -EOPNOTSUPP; +} + #endif /* CONFIG_NVMEM */ #if IS_ENABLED(CONFIG_NVMEM) && IS_ENABLED(CONFIG_OF) diff --git a/include/linux/psi.h b/include/linux/psi.h index e0745873e3f2..7966e3ac03b9 100644 --- a/include/linux/psi.h +++ b/include/linux/psi.h @@ -25,7 +25,9 @@ void psi_memstall_leave(unsigned long *flags); int psi_show(struct seq_file *s, struct psi_group *group, enum psi_res res); struct psi_trigger *psi_trigger_create(struct psi_group *group, char *buf, enum psi_res res, struct file *file, - struct kernfs_open_file *of); + struct kernfs_open_file *of, + bool *need_rtpoll_worker); +int psi_trigger_create_rtpoll_worker(struct psi_group *group); void psi_trigger_destroy(struct psi_trigger *t); __poll_t psi_trigger_poll(void **trigger_ptr, struct file *file, diff --git a/include/linux/sched/ext.h b/include/linux/sched/ext.h index 20b2343aa344..87e353f7e011 100644 --- a/include/linux/sched/ext.h +++ b/include/linux/sched/ext.h @@ -244,11 +244,11 @@ struct sched_ext_entity { * to %SCHED_EXT with -%EACCES. * * Can be set from ops.init_task() while the BPF scheduler is being - * loaded (!scx_init_task_args->fork). If set and the task's policy is - * already %SCHED_EXT, the task's policy is rejected and forcefully - * reverted to %SCHED_NORMAL. The number of such events are reported - * through /sys/kernel/debug/sched_ext::nr_rejected. Setting this flag - * during fork is not allowed. + * loaded. If set and the task's policy is already %SCHED_EXT, the + * task's policy is rejected and forcefully reverted to %SCHED_NORMAL. + * The number of such events are reported through + * /sys/kernel/sched_ext/nr_rejected. Setting this flag from any other + * ops.init_task() invocation, such as during fork, fails the scheduler. */ bool disallow; /* reject switching into SCX */ diff --git a/include/linux/usb/quirks.h b/include/linux/usb/quirks.h index b3cc7beab4a3..a4043b33c2c2 100644 --- a/include/linux/usb/quirks.h +++ b/include/linux/usb/quirks.h @@ -81,4 +81,7 @@ /* Device claims zero configurations, forcing to 1 */ #define USB_QUIRK_FORCE_ONE_CONFIG BIT(18) +/* Use a 255 bytes config descriptor request mirroring windows behavior */ +#define USB_QUIRK_WINDOWS_CONFIG_REQ_SIZE BIT(19) + #endif /* __LINUX_USB_QUIRKS_H */ diff --git a/include/net/addrconf.h b/include/net/addrconf.h index 8ced27a8229b..e6764245995f 100644 --- a/include/net/addrconf.h +++ b/include/net/addrconf.h @@ -405,8 +405,8 @@ static inline struct inet6_dev *in6_dev_get(const struct net_device *dev) rcu_read_lock(); idev = rcu_dereference(dev->ip6_ptr); - if (idev) - refcount_inc(&idev->refcnt); + if (idev && !refcount_inc_not_zero(&idev->refcnt)) + idev = NULL; rcu_read_unlock(); return idev; } diff --git a/include/net/ip_vs.h b/include/net/ip_vs.h index e6ca930a3507..d2813eb795be 100644 --- a/include/net/ip_vs.h +++ b/include/net/ip_vs.h @@ -25,9 +25,7 @@ #include /* for union nf_inet_addr */ #include #include /* for struct ipv6hdr */ -#include #include -#include #if IS_ENABLED(CONFIG_NF_CONNTRACK) #include #endif @@ -2062,7 +2060,7 @@ static inline bool ip_vs_conn_use_hash2(struct ip_vs_conn *cp) void ip_vs_nat_icmp(struct sk_buff *skb, struct ip_vs_protocol *pp, struct ip_vs_conn *cp, int dir, unsigned int toff, - bool has_ports); + bool has_ports, struct ip_vs_iphdr *ciph); #ifdef CONFIG_IP_VS_IPV6 void ip_vs_nat_icmp_v6(struct sk_buff *skb, struct ip_vs_protocol *pp, @@ -2095,30 +2093,23 @@ static inline __wsum ip_vs_check_diff2(__be16 old, __be16 new, __wsum oldsum) return csum_partial(diff, sizeof(diff), oldsum); } -static inline bool ip_vs_checksum_needed(struct sk_buff *skb, int af) +static inline bool ip_vs_checksum_needed(struct sk_buff *skb) { /* Checksum unnecessary or already validated? */ if (skb_csum_unnecessary(skb)) return false; - /* LOCAL_OUT ? */ - if (!skb->dev || skb->dev->flags & IFF_LOOPBACK) + /* Locally generated ? */ + if (!skb->dev) return false; - /* !LOCAL_IN (FORWARD) ? */ - if (af == AF_INET6) { - if (!(dst_rt6_info(skb_dst(skb))->rt6i_flags & RTF_LOCAL)) - return false; - } else { - if (!(skb_rtable(skb)->rt_flags & RTCF_LOCAL)) - return false; - } return true; } static inline bool ip_vs_checksum_common_check(struct sk_buff *skb, int offset, int proto, int af) { - if (!ip_vs_checksum_needed(skb, af)) + if (!ip_vs_checksum_needed(skb)) return true; + /* Validate csum even for FORWARD */ return !nf_checksum(skb, NF_INET_LOCAL_IN, offset, proto, af); } diff --git a/include/net/libeth/xsk.h b/include/net/libeth/xsk.h index 82b5d21aae87..5dcc0d7f65b7 100644 --- a/include/net/libeth/xsk.h +++ b/include/net/libeth/xsk.h @@ -205,7 +205,7 @@ __libeth_xsk_xmit_fill_buf_md(const struct xdp_desc *xdesc, BUILD_BUG_ON(!__builtin_constant_p(tmo == libeth_xsktmo)); tmo = tmo == libeth_xsktmo ? &__libeth_xsktmo : tmo; - xsk_tx_metadata_request(ctx.meta, tmo, &desc); + xsk_tx_metadata_request(sq->pool, &ctx.meta, tmo, &desc); return desc; } diff --git a/include/net/sch_generic.h b/include/net/sch_generic.h index 45a1e8c78222..cbc248776511 100644 --- a/include/net/sch_generic.h +++ b/include/net/sch_generic.h @@ -99,6 +99,7 @@ struct Qdisc { struct hlist_node hash; u32 handle; u32 parent; + int depth; struct netdev_queue *dev_queue; diff --git a/include/net/xdp_sock.h b/include/net/xdp_sock.h index 8b51876efbed..6e70b320b399 100644 --- a/include/net/xdp_sock.h +++ b/include/net/xdp_sock.h @@ -141,45 +141,16 @@ INDIRECT_CALLABLE_DECLARE(void xsk_destruct_skb(struct sk_buff *)); static inline void xsk_tx_metadata_to_compl(struct xsk_tx_metadata *meta, struct xsk_tx_metadata_compl *compl) { + compl->tx_timestamp = NULL; + if (!meta) return; - if (meta->flags & XDP_TXMD_FLAGS_TIMESTAMP) - compl->tx_timestamp = &meta->completion.tx_timestamp; - else - compl->tx_timestamp = NULL; -} + /* we can only arrive here if the completion timestamp has been + * requested via XDP_TXMD_FLAGS_TIMESTAMP, see xsk_tx_metadata_request + */ -/** - * xsk_tx_metadata_request - Evaluate AF_XDP TX metadata at submission - * and call appropriate xsk_tx_metadata_ops operation. - * @meta: pointer to AF_XDP metadata area - * @ops: pointer to struct xsk_tx_metadata_ops - * @priv: pointer to driver-private aread - * - * This function should be called by the networking device when - * it prepares AF_XDP egress packet. - */ -static inline void xsk_tx_metadata_request(const struct xsk_tx_metadata *meta, - const struct xsk_tx_metadata_ops *ops, - void *priv) -{ - if (!meta) - return; - - if (ops->tmo_request_launch_time) - if (meta->flags & XDP_TXMD_FLAGS_LAUNCH_TIME) - ops->tmo_request_launch_time(meta->request.launch_time, - priv); - - if (ops->tmo_request_timestamp) - if (meta->flags & XDP_TXMD_FLAGS_TIMESTAMP) - ops->tmo_request_timestamp(priv); - - if (ops->tmo_request_checksum) - if (meta->flags & XDP_TXMD_FLAGS_CHECKSUM) - ops->tmo_request_checksum(meta->request.csum_start, - meta->request.csum_offset, priv); + compl->tx_timestamp = &meta->completion.tx_timestamp; } /** @@ -231,12 +202,6 @@ static inline void xsk_tx_metadata_to_compl(struct xsk_tx_metadata *meta, { } -static inline void xsk_tx_metadata_request(struct xsk_tx_metadata *meta, - const struct xsk_tx_metadata_ops *ops, - void *priv) -{ -} - static inline void xsk_tx_metadata_complete(struct xsk_tx_metadata_compl *compl, const struct xsk_tx_metadata_ops *ops, void *priv) diff --git a/include/net/xdp_sock_drv.h b/include/net/xdp_sock_drv.h index 46797645a0c2..b344789f5df8 100644 --- a/include/net/xdp_sock_drv.h +++ b/include/net/xdp_sock_drv.h @@ -245,7 +245,7 @@ static inline void *xsk_buff_raw_get_data(struct xsk_buff_pool *pool, u64 addr) * details. * * Return: new &xdp_desc_ctx struct containing desc's DMA address and metadata - * pointer, if it is present and valid (initialized to %NULL otherwise). + * pointer, if it is present (initialized to %NULL otherwise). */ static inline struct xdp_desc_ctx xsk_buff_raw_get_ctx(const struct xsk_buff_pool *pool, u64 addr) @@ -260,24 +260,70 @@ xsk_buff_raw_get_ctx(const struct xsk_buff_pool *pool, u64 addr) 0) static inline bool -xsk_buff_valid_tx_metadata(const struct xsk_tx_metadata *meta) +xsk_buff_valid_tx_metadata(const struct xsk_buff_pool *pool, + const struct xsk_tx_metadata *meta, u64 *flags) { - return !(meta->flags & ~XDP_TXMD_FLAGS_VALID); + *flags = READ_ONCE(meta->flags); + if (*flags & XDP_TXMD_FLAGS_LAUNCH_TIME) + if (pool->tx_metadata_len < + offsetofend(struct xsk_tx_metadata, request.launch_time)) + return false; + return !(*flags & ~XDP_TXMD_FLAGS_VALID); +} + +/** + * xsk_tx_metadata_request - Evaluate AF_XDP TX metadata at submission + * and call appropriate xsk_tx_metadata_ops operation. + * @pool: pointer to AF_XDP buffer pool, used to validate the metadata + * @pmeta: pointer to pointer to AF_XDP metadata area + * @ops: pointer to struct xsk_tx_metadata_ops + * @priv: pointer to driver-private area + * + * This function should be called by the networking device when + * it prepares AF_XDP egress packet. + */ +static inline void +xsk_tx_metadata_request(const struct xsk_buff_pool *pool, + struct xsk_tx_metadata **pmeta, + const struct xsk_tx_metadata_ops *ops, void *priv) +{ + const struct xsk_tx_metadata *meta = *pmeta; + u64 flags; + + if (!meta) + return; + + if (unlikely(!xsk_buff_valid_tx_metadata(pool, meta, &flags))) { + *pmeta = NULL; + return; /* no way to signal the error to the user */ + } + + if (ops->tmo_request_launch_time) + if (flags & XDP_TXMD_FLAGS_LAUNCH_TIME) + ops->tmo_request_launch_time( + READ_ONCE(meta->request.launch_time), priv); + + if (ops->tmo_request_timestamp) + if (flags & XDP_TXMD_FLAGS_TIMESTAMP) + ops->tmo_request_timestamp(priv); + + if (ops->tmo_request_checksum) + if (flags & XDP_TXMD_FLAGS_CHECKSUM) + ops->tmo_request_checksum( + READ_ONCE(meta->request.csum_start), + READ_ONCE(meta->request.csum_offset), priv); + + if (!(flags & XDP_TXMD_FLAGS_TIMESTAMP)) + *pmeta = NULL; } static inline struct xsk_tx_metadata * __xsk_buff_get_metadata(const struct xsk_buff_pool *pool, void *data) { - struct xsk_tx_metadata *meta; - if (!pool->tx_metadata_len) return NULL; - meta = data - pool->tx_metadata_len; - if (unlikely(!xsk_buff_valid_tx_metadata(meta))) - return NULL; /* no way to signal the error to the user */ - - return meta; + return data - pool->tx_metadata_len; } static inline struct xsk_tx_metadata * @@ -469,11 +515,20 @@ xsk_buff_raw_get_ctx(const struct xsk_buff_pool *pool, u64 addr) return (struct xdp_desc_ctx){ }; } -static inline bool xsk_buff_valid_tx_metadata(struct xsk_tx_metadata *meta) +static inline bool +xsk_buff_valid_tx_metadata(const struct xsk_buff_pool *pool, + const struct xsk_tx_metadata *meta, u64 *flags) { return false; } +static inline void +xsk_tx_metadata_request(const struct xsk_buff_pool *pool, + struct xsk_tx_metadata **pmeta, + const struct xsk_tx_metadata_ops *ops, void *priv) +{ +} + static inline struct xsk_tx_metadata * __xsk_buff_get_metadata(const struct xsk_buff_pool *pool, void *data) { diff --git a/include/trace/events/btrfs.h b/include/trace/events/btrfs.h index 4c5c47c5edb7..6c1438f6a4d3 100644 --- a/include/trace/events/btrfs.h +++ b/include/trace/events/btrfs.h @@ -689,6 +689,41 @@ DEFINE_EVENT(btrfs__ordered_extent, btrfs_ordered_extent_lookup_first, TP_ARGS(inode, ordered) ); +/* + * The writepage fixup worker deferred a block because this still-running + * ordered extent covers it. + */ +DEFINE_EVENT(btrfs__ordered_extent, btrfs_writepage_fixup_defer, + + TP_PROTO(const struct btrfs_inode *inode, + const struct btrfs_ordered_extent *ordered), + + TP_ARGS(inode, ordered) +); + +/* The writepage fixup worker reserved space for a block and set delalloc. */ +TRACE_EVENT(btrfs_writepage_fixup_reserve, + + TP_PROTO(const struct btrfs_inode *inode, u64 start, u32 len), + + TP_ARGS(inode, start, len), + + TP_STRUCT__entry_btrfs( + __field( u64, ino ) + __field( u64, start ) + __field( u32, len ) + ), + + TP_fast_assign_btrfs(inode->root->fs_info, + __entry->ino = btrfs_ino(inode); + __entry->start = start; + __entry->len = len; + ), + + TP_printk_btrfs("ino=%llu start=%llu len=%u", + __entry->ino, __entry->start, __entry->len) +); + DEFINE_EVENT(btrfs__ordered_extent, btrfs_ordered_extent_split, TP_PROTO(const struct btrfs_inode *inode, diff --git a/kernel/bpf/rqspinlock.c b/kernel/bpf/rqspinlock.c index e4e338cdb437..2129defc4a9a 100644 --- a/kernel/bpf/rqspinlock.c +++ b/kernel/bpf/rqspinlock.c @@ -572,9 +572,10 @@ int __lockfunc resilient_queued_spin_lock_slowpath(rqspinlock_t *lock, u32 val) /* Disable queue destruction when we detect deadlocks. */ if (ret == -EDEADLK) { - if (!next) + if (!try_cmpxchg_tail(lock, tail, 0)) { next = smp_cond_load_relaxed(&node->next, (VAL)); - arch_mcs_spin_unlock_contended(&next->locked); + arch_mcs_spin_unlock_contended(&next->locked); + } goto err_release_node; } diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c index 7aa47342dc65..fdc5fbb1f78c 100644 --- a/kernel/bpf/verifier.c +++ b/kernel/bpf/verifier.c @@ -13557,23 +13557,21 @@ static void sanitize_mark_insn_seen(struct bpf_verifier_env *env) env->insn_aux_data[env->insn_idx].seen = env->pass_cnt; } -static int sanitize_err(struct bpf_verifier_env *env, - const struct bpf_insn *insn, int reason, - const struct bpf_reg_state *off_reg, - const struct bpf_reg_state *dst_reg) +static int sanitize_err(struct bpf_verifier_env *env, const struct bpf_insn *insn, int reason) { static const char *err = "pointer arithmetic with it prohibited for !root"; const char *op = BPF_OP(insn->code) == BPF_ADD ? "add" : "sub"; u32 dst = insn->dst_reg, src = insn->src_reg; + struct bpf_reg_state *regs = cur_regs(env); switch (reason) { case REASON_BOUNDS: verbose(env, "R%d has unknown scalar with mixed signed bounds, %s\n", - off_reg == dst_reg ? dst : src, err); + regs[src].type == SCALAR_VALUE ? src : dst, err); break; case REASON_TYPE: verbose(env, "R%d has pointer with unsupported alu operation, %s\n", - off_reg == dst_reg ? src : dst, err); + regs[src].type == SCALAR_VALUE ? dst : src, err); break; case REASON_PATHS: verbose(env, "R%d tried to %s from different maps, paths or scalars, %s\n", @@ -13709,13 +13707,6 @@ static int adjust_ptr_min_max_vals(struct bpf_verifier_env *env, return -EACCES; } - /* - * Accesses to untrusted PTR_TO_MEM are done through probe - * instructions, hence no need to track offsets. - */ - if (base_type(ptr_reg->type) == PTR_TO_MEM && (ptr_reg->type & PTR_UNTRUSTED)) - return 0; - switch (base_type(ptr_reg->type)) { case PTR_TO_CTX: case PTR_TO_MAP_VALUE: @@ -13745,11 +13736,19 @@ static int adjust_ptr_min_max_vals(struct bpf_verifier_env *env, return -EACCES; } - /* In case of 'scalar += pointer', dst_reg inherits pointer type and id. - * The id may be overwritten later if we create a new variable offset. + /* For 'scalar += pointer', dst_reg inherits the complete pointer + * register state. Individual fields may be adjusted later by pointer + * arithmetic. Callers guarantee that below does not overwrite off_reg. */ - dst_reg->type = ptr_reg->type; - dst_reg->id = ptr_reg->id; + if (dst_reg != ptr_reg) + *dst_reg = *ptr_reg; + + /* + * Accesses to untrusted PTR_TO_MEM are done through probe + * instructions, hence no need to track offsets. + */ + if (base_type(ptr_reg->type) == PTR_TO_MEM && (ptr_reg->type & PTR_UNTRUSTED)) + return 0; if (!check_reg_sane_offset_scalar(env, off_reg, ptr_reg->type) || !check_reg_sane_offset_ptr(env, ptr_reg, ptr_reg->type)) @@ -13762,7 +13761,7 @@ static int adjust_ptr_min_max_vals(struct bpf_verifier_env *env, ret = sanitize_ptr_alu(env, insn, ptr_reg, off_reg, dst_reg, &info, false); if (ret < 0) - return sanitize_err(env, insn, ret, off_reg, dst_reg); + return sanitize_err(env, insn, ret); } switch (opcode) { @@ -13792,7 +13791,7 @@ static int adjust_ptr_min_max_vals(struct bpf_verifier_env *env, } break; case BPF_SUB: - if (dst_reg == off_reg) { + if (dst_reg != ptr_reg) { /* scalar -= pointer. Creates an unknown scalar */ verbose(env, "R%d tried to subtract pointer from scalar\n", dst); @@ -13855,7 +13854,7 @@ static int adjust_ptr_min_max_vals(struct bpf_verifier_env *env, return -EFAULT; } if (ret < 0) - return sanitize_err(env, insn, ret, off_reg, dst_reg); + return sanitize_err(env, insn, ret); } return 0; @@ -14607,7 +14606,7 @@ static int adjust_scalar_min_max_vals(struct bpf_verifier_env *env, if (sanitize_needed(opcode)) { ret = sanitize_val_alu(env, insn); if (ret < 0) - return sanitize_err(env, insn, ret, NULL, NULL); + return sanitize_err(env, insn, ret); } /* Calculate sign/unsigned bounds and tnum for alu32 and alu64 bit ops. @@ -14810,8 +14809,8 @@ static int adjust_reg_min_max_vals(struct bpf_verifier_env *env, err = mark_chain_precision(env, insn->dst_reg); if (err) return err; - return adjust_ptr_min_max_vals(env, insn, - src_reg, dst_reg); + off_reg = *dst_reg; + return adjust_ptr_min_max_vals(env, insn, src_reg, &off_reg); } } else if (ptr_reg) { /* pointer += scalar */ diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c index 38f8d9df8fbc..b5b461d4418b 100644 --- a/kernel/cgroup/cgroup.c +++ b/kernel/cgroup/cgroup.c @@ -3996,6 +3996,7 @@ static ssize_t pressure_write(struct kernfs_open_file *of, char *buf, struct psi_trigger *new; struct cgroup *cgrp; struct psi_group *psi; + bool need_rtpoll_worker; ssize_t ret = 0; cgrp = cgroup_kn_lock_live(of->kn, false); @@ -4015,12 +4016,32 @@ static ssize_t pressure_write(struct kernfs_open_file *of, char *buf, } psi = cgroup_psi(cgrp); - new = psi_trigger_create(psi, buf, res, of->file, of); + new = psi_trigger_create(psi, buf, res, of->file, of, + &need_rtpoll_worker); if (IS_ERR(new)) { ret = PTR_ERR(new); goto out_unlock; } + /* + * The worker fork must run with neither cgroup_mutex nor the file's + * kernfs active reference held. The latter is broken since + * cgroup_kn_lock_live(). @of->priv may be released while unlocked, so + * recheck before publishing @new. + */ + if (need_rtpoll_worker) { + cgroup_unlock(); + ret = psi_trigger_create_rtpoll_worker(psi); + cgroup_lock(); + + if (!ret && !of->priv) + ret = -ENODEV; + if (ret) { + psi_trigger_destroy(new); + goto out_unlock; + } + } + smp_store_release(&ctx->psi.trigger, new); out_unlock: diff --git a/kernel/futex/core.c b/kernel/futex/core.c index 179b26e9c934..128c5752f225 100644 --- a/kernel/futex/core.c +++ b/kernel/futex/core.c @@ -982,8 +982,11 @@ static int handle_futex_death(u32 __user *uaddr, struct task_struct *curr, return -1; /* - * Special case for regular (non PI) futexes. The unlock path in - * user space has two race scenarios: + * Special case for regular (non PI) futexes. Ordinarily, we do + * not perform any processing here unless the current thread was + * the owner of the futex (by the TID check below). + * + * However, the unlock path has three race scenarios: * * 1. The unlock path releases the user space futex value and * before it can execute the futex() syscall to wake up @@ -992,42 +995,70 @@ static int handle_futex_death(u32 __user *uaddr, struct task_struct *curr, * 2. A woken up waiter is killed before it can acquire the * futex in user space. * - * In the second case, the wake up notification could be generated - * by the unlock path in user space after setting the futex value - * to zero or by the kernel after setting the OWNER_DIED bit below. + * 3. A woken up waiter is killed in user space after another + * thread has acquired the futex, but before it can set + * FUTEX_WAITERS. * - * In both cases the TID validation below prevents a wakeup of - * potential waiters which can cause these waiters to block - * forever. + * Note that, if userspace uses the FUTEX_ROBUST_UNLOCK flag, we + * will not see case 1 here. * - * In both cases the following conditions are met: + * In the second and third case, the wake up notification could + * be generated from any of: * - * 1) task->futex.robust_list->list_op_pending != NULL - * @pending_op == true - * 2) The owner part of user space futex value == 0 + * i. An ordinary futex wakeup after unlock (with or + * without FUTEX_ROBUST_UNLOCK) + * ii. A robust wakeup from another thread's death + * iii. A previous round through this special case + * + * As a result, the futex world will be in one of four states: + * + * A. The futex word is 0 (unlocked) + * B. The futex word is owned by another thread + * (FUTEX_WAITERS is not set) + * C. The futex word is owned by another thread + * (FUTEX_WAITERS set) + * D. The futex's owner died and OWNER_DIED is set + * (the owner part of the word is 0) + * + * The key issue is that the kernel usually (at least from + * sources ii. and iii. or when so requested by userspace from + * source i.) only ever wakes *one* waiter at a time. If this + * waiter dies before acquiring the futex (or setting the + * FUTEX_WAITERS bit), the kernel *must* still wake the next + * waiter down the line to uphold the futex invariants and + * avoid lost wakeups. Note we do not need to handle state C, + * as it does not matter to us whether *we* successfully set + * the bit or a third thread did so in the meantime. + * + * Therefore, in these cases we must issue an additional + * futex_wake(). Note however that we *must not* set OWNER_DIED + * here. Our thread is *not* the owner of the futex. + * + * Thus to summarize, the conditions for needing the additional + * futex_wake() are: + * + * 1) @pending_op == true (the thread has not finished the + * mutex operation) + * 2) The futex word is in one of the states A, B or D * 3) Regular futex: @pi == false * - * If these conditions are met, it is safe to attempt waking up a - * potential waiter without touching the user space futex value and - * trying to set the OWNER_DIED bit. If the futex value is zero, - * the rest of the user space mutex state is consistent, so a woken - * waiter will just take over the uncontended futex. Setting the - * OWNER_DIED bit would create inconsistent state and malfunction - * of the user space owner died handling. Otherwise, the OWNER_DIED - * bit is already set, and the woken waiter is expected to deal with - * this. + * Note in particular that in all of the states A-D the owner + * portion of the futex word differs from our thread's TID + * (unless the actual owner has the same TID in another PID + * namespace, but we cannot currently distinguish that + * scenario), so this can be a special-case wakeup in the bail + * path of the ordinary TID check. */ owner = uval & FUTEX_TID_MASK; - if (pending_op && !pi && !owner) { - futex_wake(uaddr, FLAGS_SIZE_32 | FLAGS_SHARED, NULL, 1, - FUTEX_BITSET_MATCH_ANY); + if (owner != task_pid_vnr(curr)) { + if (pending_op && !pi && (!owner || !(uval & FUTEX_WAITERS))) { + futex_wake(uaddr, FLAGS_SIZE_32 | FLAGS_SHARED, NULL, 1, + FUTEX_BITSET_MATCH_ANY); + } return 0; } - if (owner != task_pid_vnr(curr)) - return 0; - /* * Ok, this dying thread is truly holding a futex * of interest. Set the OWNER_DIED bit atomically @@ -1752,14 +1783,15 @@ void futex_hash_free(struct mm_struct *mm) static bool futex_pivot_pending(struct mm_struct *mm) { + struct futex_mm_phash *mmph = &mm->futex.phash; struct futex_private_hash *fph; - guard(rcu)(); + guard(mutex)(&mmph->lock); - if (!mm->futex.phash.hash_new) + if (!mmph->hash_new) return true; - fph = rcu_dereference(mm->futex.phash.hash); + fph = rcu_dereference_raw(mmph->hash); return futex_ref_is_dead(fph); } diff --git a/kernel/liveupdate/kexec_handover.c b/kernel/liveupdate/kexec_handover.c index 4834a809985a..175c08a6e41e 100644 --- a/kernel/liveupdate/kexec_handover.c +++ b/kernel/liveupdate/kexec_handover.c @@ -38,6 +38,16 @@ #include "../kexec_internal.h" #include "kexec_handover_internal.h" +/* + * This is the minimal alignment required by deferred struct page init. + * deferred_init_memmap_chunk frees memory to the buddy allocator, which looks + * at the neighboring pages (up to MAX_PAGE_ORDER) to merge them. + * If KHO scratch is not aligned to that value, buddy can access uninitialized + * struct pages, which can cause a crash. + */ +#define SCRATCH_ALIGNMENT_BYTES (PAGE_SIZE * MAX_ORDER_NR_PAGES) +static_assert(SCRATCH_ALIGNMENT_BYTES >= CMA_MIN_ALIGNMENT_BYTES); + /* The magic token for preserved pages */ #define KHO_PAGE_MAGIC 0x4b484f50U /* ASCII for 'KHOP' */ @@ -640,8 +650,8 @@ static void __init scratch_size_update(void) * Scratch areas are released as MIGRATE_CMA. Round them up to the right * size. */ - scratch_size_lowmem = round_up(scratch_size_lowmem, CMA_MIN_ALIGNMENT_BYTES); - scratch_size_global = round_up(scratch_size_global, CMA_MIN_ALIGNMENT_BYTES); + scratch_size_lowmem = round_up(scratch_size_lowmem, SCRATCH_ALIGNMENT_BYTES); + scratch_size_global = round_up(scratch_size_global, SCRATCH_ALIGNMENT_BYTES); } static phys_addr_t __init scratch_size_node(int nid) @@ -656,7 +666,7 @@ static phys_addr_t __init scratch_size_node(int nid) size = scratch_size_pernode; } - return round_up(size, CMA_MIN_ALIGNMENT_BYTES); + return round_up(size, SCRATCH_ALIGNMENT_BYTES); } /** @@ -692,7 +702,7 @@ static void __init kho_reserve_scratch(void) * next kernel */ size = scratch_size_lowmem; - addr = memblock_phys_alloc_range(size, CMA_MIN_ALIGNMENT_BYTES, 0, + addr = memblock_phys_alloc_range(size, SCRATCH_ALIGNMENT_BYTES, 0, ARCH_LOW_ADDRESS_LIMIT); if (!addr) { pr_err("Failed to reserve lowmem scratch buffer\n"); @@ -705,7 +715,7 @@ static void __init kho_reserve_scratch(void) /* reserve large contiguous area for allocations without nid */ size = scratch_size_global; - addr = memblock_phys_alloc(size, CMA_MIN_ALIGNMENT_BYTES); + addr = memblock_phys_alloc(size, SCRATCH_ALIGNMENT_BYTES); if (!addr) { pr_err("Failed to reserve global scratch buffer\n"); goto err_free_scratch_areas; @@ -721,7 +731,7 @@ static void __init kho_reserve_scratch(void) */ for_each_node_state(nid, N_MEMORY) { size = scratch_size_node(nid); - addr = memblock_alloc_range_nid(size, CMA_MIN_ALIGNMENT_BYTES, + addr = memblock_alloc_range_nid(size, SCRATCH_ALIGNMENT_BYTES, 0, MEMBLOCK_ALLOC_ACCESSIBLE, nid, true); if (!addr) { diff --git a/kernel/sched/ext/ext.c b/kernel/sched/ext/ext.c index e3fa7b2fac9d..18183062f751 100644 --- a/kernel/sched/ext/ext.c +++ b/kernel/sched/ext/ext.c @@ -3606,6 +3606,9 @@ static int __scx_init_task(struct scx_sched *sch, struct task_struct *p, bool fo } else if (unlikely(fork)) { scx_error(sch, "ops.init_task() set task->scx.disallow for %s[%d] during fork", p->comm, p->pid); + } else if (unlikely(scx_enable_state() != SCX_ENABLING)) { + scx_error(sch, "ops.init_task() set task->scx.disallow for %s[%d] outside the enable path", + p->comm, p->pid); } else { struct rq *rq; struct rq_flags rf; @@ -4577,20 +4580,25 @@ static struct cgroup *root_cgroup(void) return &cgrp_dfl_root.cgrp; } +/* + * cgroup_lock() must nest outside the rwsem write side: a writer waiting + * for cgroup_mutex deadlocks with cgroup teardown, which holds it while + * draining a set_* file write blocked on the rwsem behind the writer. + */ static void scx_cgroup_lock(void) { + cgroup_lock(); #ifdef CONFIG_EXT_GROUP_SCHED percpu_down_write(&scx_cgroup_ops_rwsem); #endif - cgroup_lock(); } static void scx_cgroup_unlock(void) { - cgroup_unlock(); #ifdef CONFIG_EXT_GROUP_SCHED percpu_up_write(&scx_cgroup_ops_rwsem); #endif + cgroup_unlock(); } #else /* CONFIG_EXT_GROUP_SCHED || CONFIG_EXT_SUB_SCHED */ static inline struct cgroup *root_cgroup(void) { return NULL; } @@ -5929,6 +5937,15 @@ static void scx_sub_disable(struct scx_sched *sch) percpu_down_write(&scx_fork_rwsem); scx_cgroup_lock(); + /* + * An enable that failed before scx_link_sched() never owned a cgroup or + * task and won't be waited on by an ancestor's drain_descendants(). + * Nothing to reparent and walking the tasks can misbehave as the task + * ownership invariant (either owned by self or parent) does not hold. + */ + if (list_empty(&sch->sibling)) + goto dump; + set_cgroup_sched(sch_cgroup(sch), parent); scx_task_iter_start(&sti, sch->cgrp); @@ -5941,8 +5958,8 @@ static void scx_sub_disable(struct scx_sched *sch) continue; /* - * By the time control reaches here, all descendant schedulers - * should already have been disabled. + * By the time control reaches here, all linked descendant + * schedulers should have been disabled. */ WARN_ON_ONCE(!scx_task_on_sched(sch, p)); @@ -5993,15 +6010,22 @@ static void scx_sub_disable(struct scx_sched *sch) /* * $p is initialized for $parent and still attached to * @sch. Disable and exit for @sch, switch over to - * $parent, override the state to READY to account for - * $p having already been initialized, and then enable. + * $parent and override the state to READY to account + * for $p having already been initialized. */ scx_disable_and_exit_task(sch, p); scx_set_task_state(p, SCX_TASK_INIT_BEGIN); scx_set_task_state(p, SCX_TASK_INIT); scx_set_task_sched(p, parent); scx_set_task_state(p, SCX_TASK_READY); - scx_enable_task(parent, p); + + /* + * A task on a non-ext class, possible under an + * %SCX_OPS_SWITCH_PARTIAL root, stays READY and is + * enabled by switching_to_scx() if it switches over. + */ + if (p->sched_class == &ext_sched_class) + scx_enable_task(parent, p); } task_rq_unlock(rq, p, &rf); @@ -6009,6 +6033,7 @@ static void scx_sub_disable(struct scx_sched *sch) } scx_task_iter_stop(&sti); +dump: scx_disable_dump(sch); scx_cgroup_unlock(); @@ -7708,10 +7733,14 @@ static void scx_sub_enable_workfn(struct kthread_work *work) /* * $p is now only initialized for @sch and READY, which - * is what we want. Assign it to @sch and enable. + * is what we want. Assign it to @sch and, if it's on + * the ext class, enable. A non-ext task, possible under + * an %SCX_OPS_SWITCH_PARTIAL root, stays READY and is + * enabled by switching_to_scx() if it switches over. */ scx_set_task_sched(p, sch); - scx_enable_task(sch, p); + if (p->sched_class == &ext_sched_class) + scx_enable_task(sch, p); p->scx.flags &= ~SCX_TASK_SUB_INIT; } diff --git a/kernel/sched/ext/idle.c b/kernel/sched/ext/idle.c index 8e8c6201b7df..6f93cc32b650 100644 --- a/kernel/sched/ext/idle.c +++ b/kernel/sched/ext/idle.c @@ -554,8 +554,10 @@ s32 scx_select_cpu_dfl(struct task_struct *p, s32 prev_cpu, u64 wake_flags, cpu_rq(cpu)->scx.local_dsq.nr == 0 && (!(flags & SCX_PICK_IDLE_IN_NODE) || (waker_node == node)) && !cpumask_empty(idle_cpumask(waker_node)->cpu)) { - if (cpumask_test_cpu(cpu, allowed)) + if (cpumask_test_cpu(cpu, allowed)) { + scx_idle_test_and_clear_cpu(cpu); goto out_unlock; + } } } diff --git a/kernel/sched/psi.c b/kernel/sched/psi.c index d9c9d9480a45..e2e825dcd088 100644 --- a/kernel/sched/psi.c +++ b/kernel/sched/psi.c @@ -1134,6 +1134,12 @@ void psi_cgroup_free(struct cgroup *cgroup) return; cancel_delayed_work_sync(&cgroup->psi->avgs_work); + /* + * A psi_schedule_rtpoll_work() call racing the last trigger's + * destruction may have re-armed the timer after psi_trigger_destroy() + * deleted it. Spurious firing while the group is alive is harmless. + */ + timer_shutdown_sync(&cgroup->psi->rtpoll_timer); free_percpu(cgroup->psi->pcpu); /* All triggers must be removed by now */ WARN_ONCE(cgroup->psi->rtpoll_states, "psi: trigger leak\n"); @@ -1292,9 +1298,44 @@ int psi_show(struct seq_file *m, struct psi_group *group, enum psi_res res) return 0; } +/* + * Create @group's rtpoll worker after psi_trigger_create() reported the need + * for one. kthread creation depends on the whole fork path and we don't want + * all of that nested inside cgroup_mutex, so the caller must drop it and any + * other lock that forks can wait behind. If two callers race, the loser stops + * its never-woken kthread. + */ +int psi_trigger_create_rtpoll_worker(struct psi_group *group) +{ + struct task_struct *task; + + task = kthread_create(psi_rtpoll_worker, group, "psimon"); + if (IS_ERR(task)) + return PTR_ERR(task); + + scoped_guard(mutex, &group->rtpoll_trigger_lock) { + if (!rcu_access_pointer(group->rtpoll_task)) { + atomic_set(&group->rtpoll_wakeup, 0); + wake_up_process(task); + rcu_assign_pointer(group->rtpoll_task, task); + + /* + * Poll once to catch up on scheduling attempts dropped + * while there was no rtpoll worker. + */ + psi_schedule_rtpoll_work(group, 1, true); + return 0; + } + } + + kthread_stop(task); + return 0; +} + struct psi_trigger *psi_trigger_create(struct psi_group *group, char *buf, enum psi_res res, struct file *file, - struct kernfs_open_file *of) + struct kernfs_open_file *of, + bool *need_rtpoll_worker) { struct psi_trigger *t; enum psi_states state; @@ -1302,6 +1343,8 @@ struct psi_trigger *psi_trigger_create(struct psi_group *group, char *buf, bool privileged; u32 window_us; + *need_rtpoll_worker = false; + if (static_branch_likely(&psi_disabled)) return ERR_PTR(-EOPNOTSUPP); @@ -1362,26 +1405,14 @@ struct psi_trigger *psi_trigger_create(struct psi_group *group, char *buf, if (privileged) { mutex_lock(&group->rtpoll_trigger_lock); - if (!rcu_access_pointer(group->rtpoll_task)) { - struct task_struct *task; - - task = kthread_create(psi_rtpoll_worker, group, "psimon"); - if (IS_ERR(task)) { - kfree(t); - mutex_unlock(&group->rtpoll_trigger_lock); - return ERR_CAST(task); - } - atomic_set(&group->rtpoll_wakeup, 0); - wake_up_process(task); - rcu_assign_pointer(group->rtpoll_task, task); - } - list_add(&t->node, &group->rtpoll_triggers); group->rtpoll_min_period = min(group->rtpoll_min_period, div_u64(t->win.size, UPDATES_PER_WINDOW)); group->rtpoll_nr_triggers[t->state]++; group->rtpoll_states |= (1 << t->state); + *need_rtpoll_worker = !rcu_access_pointer(group->rtpoll_task); + mutex_unlock(&group->rtpoll_trigger_lock); } else { mutex_lock(&group->avgs_lock); @@ -1541,6 +1572,8 @@ static ssize_t psi_write(struct file *file, const char __user *user_buf, size_t buf_size; struct seq_file *seq; struct psi_trigger *new; + bool need_rtpoll_worker; + int ret; if (static_branch_likely(&psi_disabled)) return -EOPNOTSUPP; @@ -1565,12 +1598,22 @@ static ssize_t psi_write(struct file *file, const char __user *user_buf, return -EBUSY; } - new = psi_trigger_create(&psi_system, buf, res, file, NULL); + new = psi_trigger_create(&psi_system, buf, res, file, NULL, + &need_rtpoll_worker); if (IS_ERR(new)) { mutex_unlock(&seq->lock); return PTR_ERR(new); } + if (need_rtpoll_worker) { + ret = psi_trigger_create_rtpoll_worker(&psi_system); + if (ret) { + psi_trigger_destroy(new); + mutex_unlock(&seq->lock); + return ret; + } + } + smp_store_release(&seq->private, new); mutex_unlock(&seq->lock); diff --git a/kernel/trace/ftrace.c b/kernel/trace/ftrace.c index 6c47a94f5924..7c50f8ae5a0c 100644 --- a/kernel/trace/ftrace.c +++ b/kernel/trace/ftrace.c @@ -2645,7 +2645,8 @@ unsigned long ftrace_find_rec_direct(unsigned long ip) { struct ftrace_func_entry *entry; - entry = __ftrace_lookup_ip(direct_functions, ip); + guard(preempt_notrace)(); + entry = __ftrace_lookup_ip(rcu_dereference_sched(direct_functions), ip); if (!entry) return 0; @@ -6511,6 +6512,7 @@ int update_ftrace_direct_del(struct ftrace_ops *ops, struct ftrace_hash *hash) struct ftrace_hash *new_direct_functions; struct ftrace_hash *new_filter_hash = NULL; struct ftrace_hash *old_filter_hash; + struct ftrace_hash *direct_hash; struct ftrace_func_entry *entry; struct ftrace_func_entry *del; unsigned long size; @@ -6522,11 +6524,13 @@ int update_ftrace_direct_del(struct ftrace_ops *ops, struct ftrace_hash *hash) return -EINVAL; if (!(ops->flags & FTRACE_OPS_FL_ENABLED)) return -EINVAL; - if (direct_functions == EMPTY_HASH) - return -EINVAL; mutex_lock(&direct_mutex); + direct_hash = rcu_dereference_protected(direct_functions, lockdep_is_held(&direct_mutex)); + if (direct_hash == EMPTY_HASH) + goto out_unlock; + old_filter_hash = ops->func_hash ? ops->func_hash->filter_hash : NULL; if (!hash_count(old_filter_hash)) @@ -6536,7 +6540,7 @@ int update_ftrace_direct_del(struct ftrace_ops *ops, struct ftrace_hash *hash) size = 1 << hash->size_bits; for (int i = 0; i < size; i++) { hlist_for_each_entry(entry, &hash->buckets[i], hlist) { - del = __ftrace_lookup_ip(direct_functions, entry->ip); + del = __ftrace_lookup_ip(direct_hash, entry->ip); if (!del || del->direct != entry->direct) goto out_unlock; } @@ -6547,7 +6551,7 @@ int update_ftrace_direct_del(struct ftrace_ops *ops, struct ftrace_hash *hash) if (!new_filter_hash) goto out_unlock; - new_direct_functions = hash_sub(direct_functions, hash); + new_direct_functions = hash_sub(direct_hash, hash); if (!new_direct_functions) goto out_unlock; @@ -6574,7 +6578,7 @@ int update_ftrace_direct_del(struct ftrace_ops *ops, struct ftrace_hash *hash) /* free the new_direct_functions */ old_direct_functions = new_direct_functions; } else { - old_direct_functions = direct_functions; + old_direct_functions = direct_hash; rcu_assign_pointer(direct_functions, new_direct_functions); } @@ -6613,6 +6617,7 @@ int update_ftrace_direct_mod(struct ftrace_ops *ops, struct ftrace_hash *hash, b .func = ftrace_stub, .flags = FTRACE_OPS_FL_STUB, }; + struct ftrace_hash *direct_hash; struct ftrace_hash *orig_hash; unsigned long size, i; int err = -EINVAL; @@ -6623,8 +6628,6 @@ int update_ftrace_direct_mod(struct ftrace_ops *ops, struct ftrace_hash *hash, b return -EINVAL; if (!(ops->flags & FTRACE_OPS_FL_ENABLED)) return -EINVAL; - if (direct_functions == EMPTY_HASH) - return -EINVAL; /* * We can be called from within ops_func callback with direct_mutex @@ -6632,6 +6635,12 @@ int update_ftrace_direct_mod(struct ftrace_ops *ops, struct ftrace_hash *hash, b */ if (do_direct_lock) mutex_lock(&direct_mutex); + else + lockdep_assert_held_once(&direct_mutex); + + direct_hash = rcu_dereference_protected(direct_functions, lockdep_is_held(&direct_mutex)); + if (direct_hash == EMPTY_HASH) + goto unlock; orig_hash = ops->func_hash ? ops->func_hash->filter_hash : NULL; if (!orig_hash) @@ -6663,7 +6672,7 @@ int update_ftrace_direct_mod(struct ftrace_ops *ops, struct ftrace_hash *hash, b size = 1 << hash->size_bits; for (i = 0; i < size; i++) { hlist_for_each_entry(entry, &hash->buckets[i], hlist) { - tmp = __ftrace_lookup_ip(direct_functions, entry->ip); + tmp = __ftrace_lookup_ip(direct_hash, entry->ip); if (!tmp) continue; tmp->direct = entry->direct; @@ -8296,7 +8305,8 @@ static void add_to_clear_hash_list(struct list_head *clear_list, void ftrace_free_mem(struct module *mod, void *start_ptr, void *end_ptr) { unsigned long start = (unsigned long)(start_ptr); - unsigned long end = (unsigned long)(end_ptr); + /* end is inclusive and end_ptr is exclusive */ + unsigned long end = (unsigned long)(end_ptr) - 1; struct ftrace_page **last_pg = &ftrace_pages_start; struct ftrace_page *tmp_page = NULL; struct ftrace_page *pg; @@ -8306,6 +8316,9 @@ void ftrace_free_mem(struct module *mod, void *start_ptr, void *end_ptr) struct ftrace_init_func *func, *func_next; LIST_HEAD(clear_hash); + if (start_ptr >= end_ptr) + return; + key.ip = start; key.flags = end; /* overload flags, as it is unsigned long */ diff --git a/kernel/trace/ring_buffer.c b/kernel/trace/ring_buffer.c index 8e2485bb3aa8..2667992f0aa2 100644 --- a/kernel/trace/ring_buffer.c +++ b/kernel/trace/ring_buffer.c @@ -2510,6 +2510,7 @@ rb_allocate_cpu_buffer(struct trace_buffer *buffer, long nr_pages, int cpu) bpage = alloc_cpu_page(cpu); if (!bpage) return NULL; + bpage->order = cpu_buffer->buffer->subbuf_order; rb_check_bpage(cpu_buffer, bpage); @@ -2528,6 +2529,8 @@ rb_allocate_cpu_buffer(struct trace_buffer *buffer, long nr_pages, int cpu) if (cpu_buffer->ring_meta->head_buffer) rb_meta_buffer_update(cpu_buffer, bpage); bpage->range = 1; + + atomic_inc(&cpu_buffer->resize_disabled); } else if (buffer->remote) { struct ring_buffer_desc *desc = ring_buffer_desc(buffer->remote->desc, cpu); @@ -6852,7 +6855,7 @@ int ring_buffer_swap_cpu(struct trace_buffer *buffer_a, { struct ring_buffer_per_cpu *cpu_buffer_a; struct ring_buffer_per_cpu *cpu_buffer_b; - int ret = -EINVAL; + int ret = -EBUSY; if (!cpumask_test_cpu(cpu, buffer_a->cpumask) || !cpumask_test_cpu(cpu, buffer_b->cpumask)) @@ -6893,10 +6896,10 @@ int ring_buffer_swap_cpu(struct trace_buffer *buffer_a, atomic_inc(&cpu_buffer_a->record_disabled); atomic_inc(&cpu_buffer_b->record_disabled); - ret = -EBUSY; - if (local_read(&cpu_buffer_a->committing)) + /* Do not swap if either buffer is in the process of writing */ + if (cpu_buffer_a->current_context) goto out_dec; - if (local_read(&cpu_buffer_b->committing)) + if (cpu_buffer_b->current_context) goto out_dec; /* @@ -7358,7 +7361,7 @@ int ring_buffer_subbuf_order_set(struct trace_buffer *buffer, int order) cpu_buffer = buffer->buffers[cpu]; - if (cpu_buffer->mapped) { + if (atomic_read(&cpu_buffer->resize_disabled)) { err = -EBUSY; goto error; } @@ -8214,7 +8217,7 @@ static __init int test_ringbuffer(void) out_free: for_each_online_cpu(cpu) { - if (!rb_threads[cpu]) + if (IS_ERR_OR_NULL(rb_threads[cpu])) break; kthread_stop(rb_threads[cpu]); } diff --git a/kernel/trace/trace.c b/kernel/trace/trace.c index 01a5e87af299..395238b2b715 100644 --- a/kernel/trace/trace.c +++ b/kernel/trace/trace.c @@ -1788,7 +1788,7 @@ void trace_buffered_event_enable(void) per_cpu(trace_buffered_event, cpu) = event; - scoped_guard(preempt,) { + scoped_guard(preempt) { if (cpu == smp_processor_id() && __this_cpu_read(trace_buffered_event) != per_cpu(trace_buffered_event, cpu)) diff --git a/mm/damon/lru_sort.c b/mm/damon/lru_sort.c index 32f41491b726..729d94b8d77e 100644 --- a/mm/damon/lru_sort.c +++ b/mm/damon/lru_sort.c @@ -233,6 +233,8 @@ static int damon_lru_sort_add_quota_goals(struct damos *hot_scheme, if (!active_mem_bp) return 0; + if (10000 < active_mem_bp) + return -EINVAL; goal = damos_new_quota_goal(DAMOS_QUOTA_ACTIVE_MEM_BP, active_mem_bp); if (!goal) return -ENOMEM; @@ -349,6 +351,8 @@ static int damon_lru_sort_commit_inputs_fn(void *arg) return damon_lru_sort_apply_parameters(); } +static bool damon_lru_sort_damon_has_started; + static int damon_lru_sort_commit_inputs_store(const char *val, const struct kernel_param *kp) { @@ -369,11 +373,8 @@ static int damon_lru_sort_commit_inputs_store(const char *val, if (!commit_inputs_request) return 0; - /* - * Skip damon_call() if ctx is not initialized to avoid - * NULL pointer dereference. - */ - if (!ctx) + /* Skip damon_call() if ctx has not successfully started. */ + if (!damon_lru_sort_damon_has_started) return -EINVAL; err = damon_call(ctx, &control); @@ -424,6 +425,8 @@ static int damon_lru_sort_turn(bool on) err = damon_start(&ctx, 1, true); if (err) return err; + if (!damon_lru_sort_damon_has_started) + damon_lru_sort_damon_has_started = true; return damon_call(ctx, &call_control); } diff --git a/mm/damon/ops-common.c b/mm/damon/ops-common.c index 6bdd1cfd3863..0bcad6b1e5b9 100644 --- a/mm/damon/ops-common.c +++ b/mm/damon/ops-common.c @@ -375,6 +375,8 @@ static unsigned int damon_migrate_folio_list(struct list_head *folio_list, while (!list_empty(folio_list)) { folio = lru_to_folio(folio_list); list_del(&folio->lru); + node_stat_sub_folio(folio, NR_ISOLATED_ANON + + folio_is_file_lru(folio)); folio_putback_lru(folio); } @@ -392,8 +394,17 @@ unsigned long damon_migrate_pages(struct list_head *folio_list, int target_nid) return nr_migrated; if (target_nid < 0 || target_nid >= MAX_NUMNODES || - !node_state(target_nid, N_MEMORY)) + !node_state(target_nid, N_MEMORY)) { + while (!list_empty(folio_list)) { + struct folio *folio = lru_to_folio(folio_list); + + list_del(&folio->lru); + node_stat_sub_folio(folio, NR_ISOLATED_ANON + + folio_is_file_lru(folio)); + folio_putback_lru(folio); + } return nr_migrated; + } noreclaim_flag = memalloc_noreclaim_save(); diff --git a/mm/damon/paddr.c b/mm/damon/paddr.c index 5c2da45f988c..f7613ce279a2 100644 --- a/mm/damon/paddr.c +++ b/mm/damon/paddr.c @@ -350,6 +350,8 @@ static unsigned long damon_pa_migrate(struct damon_region *r, if (!folio_isolate_lru(folio)) goto put_folio; + node_stat_add_folio(folio, NR_ISOLATED_ANON + + folio_is_file_lru(folio)); list_add(&folio->lru, &folio_list); put_folio: addr += folio_size(folio); diff --git a/mm/damon/reclaim.c b/mm/damon/reclaim.c index 11b70d0a9a6f..a1a65b1270c8 100644 --- a/mm/damon/reclaim.c +++ b/mm/damon/reclaim.c @@ -276,6 +276,8 @@ static int damon_reclaim_commit_inputs_fn(void *arg) return damon_reclaim_apply_parameters(); } +static bool damon_reclaim_damon_has_started; + static int damon_reclaim_commit_inputs_store(const char *val, const struct kernel_param *kp) { @@ -296,11 +298,8 @@ static int damon_reclaim_commit_inputs_store(const char *val, if (!commit_inputs_request) return 0; - /* - * Skip damon_call() if ctx is not initialized to avoid - * NULL pointer dereference. - */ - if (!ctx) + /* Skip damon_call() if ctx has not successfully started. */ + if (!damon_reclaim_damon_has_started) return -EINVAL; err = damon_call(ctx, &control); @@ -347,6 +346,8 @@ static int damon_reclaim_turn(bool on) err = damon_start(&ctx, 1, true); if (err) return err; + if (!damon_reclaim_damon_has_started) + damon_reclaim_damon_has_started = true; return damon_call(ctx, &call_control); } diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c index e73ec1ce016e..2c1c1952c008 100644 --- a/mm/damon/vaddr.c +++ b/mm/damon/vaddr.c @@ -649,7 +649,8 @@ static void damos_va_migrate_dests_add(struct folio *folio, isolate: if (!folio_isolate_lru(folio)) return; - + node_stat_add_folio(folio, NR_ISOLATED_ANON + + folio_is_file_lru(folio)); list_add(&folio->lru, &migration_lists[i]); } diff --git a/mm/filemap.c b/mm/filemap.c index 58eb9d240643..d721986d5f46 100644 --- a/mm/filemap.c +++ b/mm/filemap.c @@ -931,6 +931,12 @@ noinline int __filemap_add_folio(struct address_space *mapping, if (!xas_nomem(&xas, gfp)) break; + + /* + * Lock has been dropped: start again with the original index + * and order (but now with the memory reserved by xas_nomem()). + */ + xas_set_order(&xas, index, forder); } if (xas_error(&xas)) diff --git a/mm/huge_memory.c b/mm/huge_memory.c index 58cabe6af33d..5d94fa4c74fd 100644 --- a/mm/huge_memory.c +++ b/mm/huge_memory.c @@ -41,6 +41,7 @@ #include #include #include +#include #include #include "internal.h" @@ -77,9 +78,15 @@ static unsigned long deferred_split_scan(struct shrinker *shrink, struct shrink_control *sc); static bool split_underused_thp = true; -static atomic_t huge_zero_refcount; +#define HUGE_ZERO_UNSET_PFN (~0UL) struct folio *huge_zero_folio __read_mostly; -unsigned long huge_zero_pfn __read_mostly = ~0UL; +unsigned long huge_zero_pfn __read_mostly = HUGE_ZERO_UNSET_PFN; +#ifndef CONFIG_PERSISTENT_HUGE_ZERO_FOLIO +static atomic_t huge_zero_refcount; +static DEFINE_SPINLOCK(huge_zero_lock); +static struct shrinker *huge_zero_folio_shrinker; +#endif + unsigned long huge_anon_orders_always __read_mostly; unsigned long huge_anon_orders_madvise __read_mostly; unsigned long huge_anon_orders_inherit __read_mostly; @@ -221,33 +228,74 @@ unsigned long __thp_vma_allowable_orders(struct vm_area_struct *vma, return orders; } -static bool get_huge_zero_folio(void) +static struct folio *alloc_huge_zero_folio(void) { struct folio *zero_folio; -retry: - if (likely(atomic_inc_not_zero(&huge_zero_refcount))) - return true; zero_folio = folio_alloc((GFP_TRANSHUGE | __GFP_ZERO | __GFP_ZEROTAGS) & ~__GFP_MOVABLE, HPAGE_PMD_ORDER); if (!zero_folio) { count_vm_event(THP_ZERO_PAGE_ALLOC_FAILED); - return false; + return NULL; } - /* Ensure zero folio won't have large_rmappable flag set. */ - folio_clear_large_rmappable(zero_folio); - preempt_disable(); - if (cmpxchg(&huge_zero_folio, NULL, zero_folio)) { - preempt_enable(); - folio_put(zero_folio); - goto retry; - } - WRITE_ONCE(huge_zero_pfn, folio_pfn(zero_folio)); + folio_clear_large_rmappable(zero_folio); /* Explicitly not rmappable. */ + return zero_folio; +} + +#ifdef CONFIG_PERSISTENT_HUGE_ZERO_FOLIO +static int __init huge_zero_init(void) +{ + huge_zero_folio = alloc_huge_zero_folio(); + if (!huge_zero_folio) { + pr_warn("Allocating persistent huge zero folio failed\n"); + } else { + huge_zero_pfn = folio_pfn(huge_zero_folio); + count_vm_event(THP_ZERO_PAGE_ALLOC); + } + return 0; +} + +static void __init huge_zero_shrinker_exit(void) +{ +} + +struct folio *mm_get_huge_zero_folio(struct mm_struct *mm) +{ + return huge_zero_folio; +} + +void mm_put_huge_zero_folio(struct mm_struct *mm) +{ +} +#else +static bool get_huge_zero_folio(void) +{ + struct folio *zero_folio; + + /* Paired with atomic_set_release(). */ + if (likely(atomic_inc_not_zero(&huge_zero_refcount))) + return true; + + zero_folio = alloc_huge_zero_folio(); + if (unlikely(!zero_folio)) + return false; + + /* Paired with critical section in shrink_huge_zero_folio_scan(). */ + spin_lock(&huge_zero_lock); + if (huge_zero_folio) { + /* Somebody else already installed it. */ + atomic_inc(&huge_zero_refcount); + spin_unlock(&huge_zero_lock); + folio_put(zero_folio); + return true; + } + WRITE_ONCE(huge_zero_folio, zero_folio); + WRITE_ONCE(huge_zero_pfn, folio_pfn(zero_folio)); + /* Paired with atomic_inc_not_zero(). +1 for shrinker pin. */ + atomic_set_release(&huge_zero_refcount, 2); + spin_unlock(&huge_zero_lock); - /* We take additional reference here. It will be put back by shrinker */ - atomic_set(&huge_zero_refcount, 2); - preempt_enable(); count_vm_event(THP_ZERO_PAGE_ALLOC); return true; } @@ -258,14 +306,59 @@ static void put_huge_zero_folio(void) * Counter should never go to zero here. Only shrinker can put * last reference. */ - BUG_ON(atomic_dec_and_test(&huge_zero_refcount)); + WARN_ON_ONCE(atomic_dec_and_test(&huge_zero_refcount)); +} + +static unsigned long shrink_huge_zero_folio_count(struct shrinker *shrink, + struct shrink_control *sc) +{ + /* we can free zero page only if last reference remains */ + return atomic_read(&huge_zero_refcount) == 1 ? HPAGE_PMD_NR : 0; +} + +static unsigned long shrink_huge_zero_folio_scan(struct shrinker *shrink, + struct shrink_control *sc) +{ + struct folio *zero_folio; + + /* Paired with critical section in get_huge_zero_folio(). */ + scoped_guard(spinlock, &huge_zero_lock) { + /* Paired with atomic_inc_not_zero() in get_huge_zero_folio(). */ + if (atomic_cmpxchg(&huge_zero_refcount, 1, 0) != 1) + return 0; + + zero_folio = huge_zero_folio; + VM_WARN_ON_ONCE(!zero_folio); + WRITE_ONCE(huge_zero_folio, NULL); + WRITE_ONCE(huge_zero_pfn, HUGE_ZERO_UNSET_PFN); + } + + folio_put(zero_folio); + return HPAGE_PMD_NR; +} + +static int __init huge_zero_init(void) +{ + huge_zero_folio_shrinker = shrinker_alloc(0, "thp-zero"); + if (!huge_zero_folio_shrinker) { + shrinker_free(deferred_split_shrinker); + list_lru_destroy(&deferred_split_lru); + return -ENOMEM; + } + + huge_zero_folio_shrinker->count_objects = shrink_huge_zero_folio_count; + huge_zero_folio_shrinker->scan_objects = shrink_huge_zero_folio_scan; + shrinker_register(huge_zero_folio_shrinker); + return 0; +} + +static void __init huge_zero_shrinker_exit(void) +{ + shrinker_free(huge_zero_folio_shrinker); } struct folio *mm_get_huge_zero_folio(struct mm_struct *mm) { - if (IS_ENABLED(CONFIG_PERSISTENT_HUGE_ZERO_FOLIO)) - return huge_zero_folio; - if (mm_flags_test(MMF_HUGE_ZERO_FOLIO, mm)) return READ_ONCE(huge_zero_folio); @@ -280,35 +373,10 @@ struct folio *mm_get_huge_zero_folio(struct mm_struct *mm) void mm_put_huge_zero_folio(struct mm_struct *mm) { - if (IS_ENABLED(CONFIG_PERSISTENT_HUGE_ZERO_FOLIO)) - return; - if (mm_flags_test(MMF_HUGE_ZERO_FOLIO, mm)) put_huge_zero_folio(); } - -static unsigned long shrink_huge_zero_folio_count(struct shrinker *shrink, - struct shrink_control *sc) -{ - /* we can free zero page only if last reference remains */ - return atomic_read(&huge_zero_refcount) == 1 ? HPAGE_PMD_NR : 0; -} - -static unsigned long shrink_huge_zero_folio_scan(struct shrinker *shrink, - struct shrink_control *sc) -{ - if (atomic_cmpxchg(&huge_zero_refcount, 1, 0) == 1) { - struct folio *zero_folio = xchg(&huge_zero_folio, NULL); - BUG_ON(zero_folio == NULL); - WRITE_ONCE(huge_zero_pfn, ~0UL); - folio_put(zero_folio); - return HPAGE_PMD_NR; - } - - return 0; -} - -static struct shrinker *huge_zero_folio_shrinker; +#endif /* CONFIG_PERSISTENT_HUGE_ZERO_FOLIO */ #ifdef CONFIG_SYSFS static ssize_t enabled_show(struct kobject *kobj, @@ -972,39 +1040,14 @@ static int __init thp_shrinker_init(void) deferred_split_shrinker->scan_objects = deferred_split_scan; shrinker_register(deferred_split_shrinker); - if (IS_ENABLED(CONFIG_PERSISTENT_HUGE_ZERO_FOLIO)) { - /* - * Bump the reference of the huge_zero_folio and do not - * initialize the shrinker. - * - * huge_zero_folio will always be NULL on failure. We assume - * that get_huge_zero_folio() will most likely not fail as - * thp_shrinker_init() is invoked early on during boot. - */ - if (!get_huge_zero_folio()) - pr_warn("Allocating persistent huge zero folio failed\n"); - return 0; - } - - huge_zero_folio_shrinker = shrinker_alloc(0, "thp-zero"); - if (!huge_zero_folio_shrinker) { - shrinker_free(deferred_split_shrinker); - list_lru_destroy(&deferred_split_lru); - return -ENOMEM; - } - - huge_zero_folio_shrinker->count_objects = shrink_huge_zero_folio_count; - huge_zero_folio_shrinker->scan_objects = shrink_huge_zero_folio_scan; - shrinker_register(huge_zero_folio_shrinker); - - return 0; + return huge_zero_init(); } static void __init thp_shrinker_exit(void) { - shrinker_free(huge_zero_folio_shrinker); shrinker_free(deferred_split_shrinker); list_lru_destroy(&deferred_split_lru); + huge_zero_shrinker_exit(); } static int __init hugepage_init(void) @@ -4033,7 +4076,7 @@ static int __folio_split(struct folio *folio, unsigned int new_order, gfp_t gfp; mapping = folio->mapping; - min_order = mapping_min_folio_order(folio->mapping); + min_order = mapping_min_folio_order(mapping); if (new_order < min_order) { ret = -EINVAL; goto out; @@ -4047,6 +4090,8 @@ static int __folio_split(struct folio *folio, unsigned int new_order, goto out; } + mapping_set_update(&xas, mapping); + if (split_type == SPLIT_TYPE_UNIFORM) { xas_set_order(&xas, folio->index, new_order); xas_split_alloc(&xas, folio, old_order, gfp); diff --git a/mm/memory.c b/mm/memory.c index ff338c2abe92..6b8280cfc1db 100644 --- a/mm/memory.c +++ b/mm/memory.c @@ -1981,7 +1981,7 @@ static unsigned long zap_pte_range(struct mmu_gather *tlb, if (can_reclaim_pt) { if (direct_reclaim || zap_pte_table_if_empty(mm, pmd, start, &pmdval)) { - pte_free_tlb(tlb, pmd_pgtable(pmdval), addr); + pte_free_tlb(tlb, pmd_pgtable(pmdval), start); mm_dec_nr_ptes(mm); } } diff --git a/mm/page_table_check.c b/mm/page_table_check.c index 53a8997ec043..2403f5a11410 100644 --- a/mm/page_table_check.c +++ b/mm/page_table_check.c @@ -151,18 +151,29 @@ void __page_table_check_pte_clear(struct mm_struct *mm, unsigned long addr, if (&init_mm == mm) return; - if (pte_user_accessible_page(mm, addr, pte)) + if (pte_user_accessible_page(mm, addr, pte) && !pte_special(pte)) page_table_check_clear(pte_pfn(pte), PAGE_SIZE >> PAGE_SHIFT); } EXPORT_SYMBOL(__page_table_check_pte_clear); +static inline bool page_table_check_huge_zero_pmd(pmd_t pmd) +{ + unsigned long pfn = pmd_pfn(pmd); + + if (!pfn_valid(pfn)) + return false; + + return is_huge_zero_folio(page_folio(pfn_to_page(pfn))); +} + void __page_table_check_pmd_clear(struct mm_struct *mm, unsigned long addr, pmd_t pmd) { if (&init_mm == mm) return; - if (pmd_user_accessible_page(mm, addr, pmd)) + if (pmd_user_accessible_page(mm, addr, pmd) && + !page_table_check_huge_zero_pmd(pmd)) page_table_check_clear(pmd_pfn(pmd), PMD_SIZE >> PAGE_SHIFT); } EXPORT_SYMBOL(__page_table_check_pmd_clear); @@ -208,7 +219,7 @@ void __page_table_check_ptes_set(struct mm_struct *mm, unsigned long addr, for (i = 0; i < nr; i++) __page_table_check_pte_clear(mm, addr + PAGE_SIZE * i, ptep_get(ptep + i)); - if (pte_user_accessible_page(mm, addr, pte)) + if (pte_user_accessible_page(mm, addr, pte) && !pte_special(pte)) page_table_check_set(pte_pfn(pte), nr, pte_write(pte)); } EXPORT_SYMBOL(__page_table_check_ptes_set); @@ -238,7 +249,8 @@ void __page_table_check_pmds_set(struct mm_struct *mm, unsigned long addr, for (i = 0; i < nr; i++) __page_table_check_pmd_clear(mm, addr + PMD_SIZE * i, *(pmdp + i)); - if (pmd_user_accessible_page(mm, addr, pmd)) + if (pmd_user_accessible_page(mm, addr, pmd) && + !page_table_check_huge_zero_pmd(pmd)) page_table_check_set(pmd_pfn(pmd), stride * nr, pmd_write(pmd)); } EXPORT_SYMBOL(__page_table_check_pmds_set); diff --git a/mm/pagewalk.c b/mm/pagewalk.c index 3ae2586ff45b..5d87c632a255 100644 --- a/mm/pagewalk.c +++ b/mm/pagewalk.c @@ -678,6 +678,8 @@ int walk_kernel_page_table_range_lockless(unsigned long start, unsigned long end * will also not lock the PTEs for the pte_entry() callback. * * This is for debugging purposes ONLY. + * + * The mmap write lock must be held. */ int walk_page_range_debug(struct mm_struct *mm, unsigned long start, unsigned long end, const struct mm_walk_ops *ops, @@ -691,25 +693,29 @@ int walk_page_range_debug(struct mm_struct *mm, unsigned long start, .no_vma = true }; - /* For convenience, we allow traversal of kernel mappings. */ - if (mm == &init_mm) - return walk_kernel_page_table_range(start, end, ops, - pgd, private); - if (start >= end || !walk.mm) + /* + * When walking userland page tables, an mmap write lock must be held to + * account for munmap() downgrading to an mmap read lock when tearing + * down page tables. + * + * When walking kernel page tables, an mmap write lock must also be held + * to account for page table freeing on vmap huge page mapping. + */ + mmap_assert_write_locked(mm); + /* + * x86, arm64 ptdump allow walks of efi mm's and x86 ptdump allows walks + * of arbitrary mm's. + * + * However, they both must also hold the init_mm lock to account for + * concurrent kernel page table freeing. + */ + mmap_assert_write_locked(&init_mm); + + if (start >= end) return -EINVAL; if (!check_ops_safe(ops)) return -EINVAL; - /* - * The mmap lock protects the page walker from changes to the page - * tables during the walk. However a read lock is insufficient to - * protect those areas which don't have a VMA as munmap() detaches - * the VMAs before downgrading to a read lock and actually tearing - * down PTEs/page tables. In which case, the mmap write lock should - * be held. - */ - mmap_assert_write_locked(mm); - return walk_pgd_range(start, end, &walk); } diff --git a/mm/ptdump.c b/mm/ptdump.c index 973020000096..5851096e6f65 100644 --- a/mm/ptdump.c +++ b/mm/ptdump.c @@ -178,11 +178,18 @@ void ptdump_walk_pgd(struct ptdump_state *st, struct mm_struct *mm, pgd_t *pgd) get_online_mems(); mmap_write_lock(mm); + /* To stabilise kernel page tables we must hold the init_mm lock too. */ + if (mm != &init_mm) + mmap_write_lock_nested(&init_mm, SINGLE_DEPTH_NESTING); + while (range->start != range->end) { walk_page_range_debug(mm, range->start, range->end, &ptdump_ops, pgd, st); range++; } + + if (mm != &init_mm) + mmap_write_unlock(&init_mm); mmap_write_unlock(mm); put_online_mems(); diff --git a/mm/vmalloc.c b/mm/vmalloc.c index 1afca3568b9b..f4fa227a8d7f 100644 --- a/mm/vmalloc.c +++ b/mm/vmalloc.c @@ -43,6 +43,7 @@ #include #include #include +#include #define CREATE_TRACE_POINTS #include @@ -158,10 +159,21 @@ static int vmap_try_huge_pmd(pmd_t *pmd, unsigned long addr, unsigned long end, if (!IS_ALIGNED(phys_addr, PMD_SIZE)) return 0; - if (pmd_present(*pmd) && !pmd_free_pte_page(pmd, addr)) - return 0; + if (!pmd_present(*pmd)) + return pmd_set_huge(pmd, phys_addr, prot); - return pmd_set_huge(pmd, phys_addr, prot); + /* + * Acquire the mmap read lock to exclude ptdump, which walks + * kernel page tables it does not own under the mmap write lock. + * + * Concurrent read lock holders are safe: each exclusively owns + * the range it operates on and cannot reach this page table. + */ + scoped_cond_guard(mmap_read_lock_try, return 0, &init_mm) { + if (!pmd_free_pte_page(pmd, addr)) + return 0; + return pmd_set_huge(pmd, phys_addr, prot); + } } static int vmap_pmd_range(pud_t *pud, unsigned long addr, unsigned long end, @@ -210,10 +222,15 @@ static int vmap_try_huge_pud(pud_t *pud, unsigned long addr, unsigned long end, if (!IS_ALIGNED(phys_addr, PUD_SIZE)) return 0; - if (pud_present(*pud) && !pud_free_pmd_page(pud, addr)) - return 0; + if (!pud_present(*pud)) + return pud_set_huge(pud, phys_addr, prot); - return pud_set_huge(pud, phys_addr, prot); + /* See comment in vmap_try_huge_pmd(). */ + scoped_cond_guard(mmap_read_lock_try, return 0, &init_mm) { + if (!pud_free_pmd_page(pud, addr)) + return 0; + return pud_set_huge(pud, phys_addr, prot); + } } static int vmap_pud_range(p4d_t *p4d, unsigned long addr, unsigned long end, @@ -262,10 +279,15 @@ static int vmap_try_huge_p4d(p4d_t *p4d, unsigned long addr, unsigned long end, if (!IS_ALIGNED(phys_addr, P4D_SIZE)) return 0; - if (p4d_present(*p4d) && !p4d_free_pud_page(p4d, addr)) - return 0; + if (!p4d_present(*p4d)) + return p4d_set_huge(p4d, phys_addr, prot); - return p4d_set_huge(p4d, phys_addr, prot); + /* See comment in vmap_try_huge_pmd(). */ + scoped_cond_guard(mmap_read_lock_try, return 0, &init_mm) { + if (!p4d_free_pud_page(p4d, addr)) + return 0; + return p4d_set_huge(p4d, phys_addr, prot); + } } static int vmap_p4d_range(pgd_t *pgd, unsigned long addr, unsigned long end, diff --git a/net/atm/common.c b/net/atm/common.c index c7f92405daf0..81195727fa18 100644 --- a/net/atm/common.c +++ b/net/atm/common.c @@ -710,7 +710,7 @@ int vcc_setsockopt(struct socket *sock, int level, int optname, sockptr_t optval, unsigned int optlen) { struct atm_vcc *vcc; - unsigned long value; + int value; int error; if (__SO_LEVEL_MATCH(optname, level) && optlen != __SO_SIZE(optname)) @@ -722,8 +722,10 @@ int vcc_setsockopt(struct socket *sock, int level, int optname, { struct atm_qos qos; - if (copy_from_sockptr(&qos, optval, sizeof(qos))) - return -EFAULT; + error = copy_safe_from_sockptr(&qos, sizeof(qos), optval, + optlen); + if (error) + return error; error = check_qos(&qos); if (error) return error; @@ -737,8 +739,10 @@ int vcc_setsockopt(struct socket *sock, int level, int optname, return 0; } case SO_SETCLP: - if (copy_from_sockptr(&value, optval, sizeof(value))) - return -EFAULT; + error = copy_safe_from_sockptr(&value, sizeof(value), optval, + optlen); + if (error) + return error; if (value) vcc->atm_options |= ATM_ATMOPT_CLP; else diff --git a/net/bridge/br_mrp.c b/net/bridge/br_mrp.c index 179d2470b724..ef16d0703924 100644 --- a/net/bridge/br_mrp.c +++ b/net/bridge/br_mrp.c @@ -224,11 +224,9 @@ static struct sk_buff *br_mrp_alloc_test_skb(struct br_mrp *mrp, sub_opt = skb_put(skb, sizeof(*sub_opt)); memset(sub_opt, 0x0, sizeof(*sub_opt)); - sub_tlv = skb_put(skb, sizeof(*sub_tlv)); - sub_tlv->type = BR_MRP_SUB_TLV_HEADER_TEST_AUTO_MGR; - /* 32 bit alligment shall be ensured therefore add 2 bytes */ - skb_put(skb, MRP_OPT_PADDING); + sub_tlv = skb_put_zero(skb, sizeof(*sub_tlv) + MRP_OPT_PADDING); + sub_tlv->type = BR_MRP_SUB_TLV_HEADER_TEST_AUTO_MGR; } br_mrp_skb_tlv(skb, BR_MRP_TLV_HEADER_END, 0x0); diff --git a/net/bridge/netfilter/ebt_nflog.c b/net/bridge/netfilter/ebt_nflog.c index 61bf8f4465ab..426f8adc912c 100644 --- a/net/bridge/netfilter/ebt_nflog.c +++ b/net/bridge/netfilter/ebt_nflog.c @@ -41,11 +41,25 @@ ebt_nflog_tg(struct sk_buff *skb, const struct xt_action_param *par) static int ebt_nflog_tg_check(const struct xt_tgchk_param *par) { struct ebt_nflog_info *info = par->targinfo; + int ret; if (info->flags & ~EBT_NFLOG_MASK) return -EINVAL; info->prefix[EBT_NFLOG_PREFIX_SIZE - 1] = '\0'; - return 0; + + ret = nf_logger_find_get(par->family, NF_LOG_TYPE_ULOG); + if (ret != 0 && !par->nft_compat) { + request_module("%s", "nfnetlink_log"); + + ret = nf_logger_find_get(par->family, NF_LOG_TYPE_ULOG); + } + + return ret; +} + +static void ebt_nflog_tg_destroy(const struct xt_tgdtor_param *par) +{ + nf_logger_put(par->family, NF_LOG_TYPE_ULOG); } static struct xt_target ebt_nflog_tg_reg __read_mostly = { @@ -54,6 +68,7 @@ static struct xt_target ebt_nflog_tg_reg __read_mostly = { .family = NFPROTO_BRIDGE, .target = ebt_nflog_tg, .checkentry = ebt_nflog_tg_check, + .destroy = ebt_nflog_tg_destroy, .targetsize = sizeof(struct ebt_nflog_info), .me = THIS_MODULE, }; diff --git a/net/core/datagram.c b/net/core/datagram.c index c285c6465923..173b5d97bd40 100644 --- a/net/core/datagram.c +++ b/net/core/datagram.c @@ -712,6 +712,9 @@ zerocopy_fill_skb_from_devmem(struct sk_buff *skb, struct iov_iter *from, size_t virt_addr, size, off; struct net_iov *niov; + if (i && skb_frags_readable(skb)) + return -EFAULT; + /* Devmem filling works by taking an IOVEC from the user where the * iov_addrs are interpreted as an offset in bytes into the dma-buf to * send from. We do not support other iter types. diff --git a/net/core/dev.c b/net/core/dev.c index 5933c5dab09e..af260ff5462a 100644 --- a/net/core/dev.c +++ b/net/core/dev.c @@ -11494,6 +11494,7 @@ int register_netdevice(struct net_device *dev) * Prevent userspace races by waiting until the network * device is fully setup before sending notifications. */ + netdev_uevent_add(dev); if (!(dev->rtnl_link_ops && dev->rtnl_link_initializing)) rtmsg_ifinfo(RTM_NEWLINK, dev, ~0U, GFP_KERNEL, 0, NULL); @@ -12435,6 +12436,7 @@ void unregister_netdevice_many_notify(struct list_head *head, dev_tcx_uninstall(dev); dev_xdp_uninstall(dev); dev_memory_provider_uninstall(dev); + netdev_work_cancel_all(dev); netdev_unlock_ops(dev); bpf_dev_bound_netdev_unregister(dev); diff --git a/net/core/dev.h b/net/core/dev.h index 5d0b0305d3ba..b757faead4d1 100644 --- a/net/core/dev.h +++ b/net/core/dev.h @@ -179,6 +179,7 @@ enum netdev_work_core { void __netdev_work_core_sched(struct net_device *dev, unsigned long event); unsigned long __netdev_work_core_cancel(struct net_device *dev, unsigned long mask); +void netdev_work_cancel_all(struct net_device *dev); void __dev_notify_flags(struct net_device *dev, unsigned int old_flags, unsigned int gchanges, u32 portid, diff --git a/net/core/filter.c b/net/core/filter.c index 11bb0d236822..16845987b244 100644 --- a/net/core/filter.c +++ b/net/core/filter.c @@ -7684,7 +7684,7 @@ BPF_CALL_5(bpf_tcp_check_syncookie, struct sock *, sk, void *, iph, u32, iph_len return -EINVAL; /* sk_listener() allows TCP_NEW_SYN_RECV, which makes no sense here. */ - if (sk->sk_protocol != IPPROTO_TCP || sk->sk_state != TCP_LISTEN) + if (sk->sk_state != TCP_LISTEN || sk->sk_protocol != IPPROTO_TCP) return -EINVAL; if (!READ_ONCE(sock_net(sk)->ipv4.sysctl_tcp_syncookies)) @@ -7757,7 +7757,7 @@ BPF_CALL_5(bpf_tcp_gen_syncookie, struct sock *, sk, void *, iph, u32, iph_len, if (unlikely(!sk || th_len < sizeof(*th) || th_len != th->doff * 4)) return -EINVAL; - if (sk->sk_protocol != IPPROTO_TCP || sk->sk_state != TCP_LISTEN) + if (sk->sk_state != TCP_LISTEN || sk->sk_protocol != IPPROTO_TCP) return -EINVAL; if (!READ_ONCE(sock_net(sk)->ipv4.sysctl_tcp_syncookies)) diff --git a/net/core/net-sysfs.c b/net/core/net-sysfs.c index 0e71c9ed41e8..25546deacec8 100644 --- a/net/core/net-sysfs.c +++ b/net/core/net-sysfs.c @@ -2334,6 +2334,9 @@ int netdev_register_kobject(struct net_device *ndev) *groups++ = &wireless_group; #endif /* CONFIG_SYSFS */ + /* Hold back the KOBJ_ADD uevent until the device is listed. */ + dev_set_uevent_suppress(dev, 1); + error = device_add(dev); if (error) return error; @@ -2349,6 +2352,17 @@ int netdev_register_kobject(struct net_device *ndev) return error; } +/* Announce a fully registered device to userspace. This pairs with the uevent + * suppression from netdev_register_kobject(). + */ +void netdev_uevent_add(struct net_device *ndev) +{ + struct device *dev = &ndev->dev; + + dev_set_uevent_suppress(dev, 0); + kobject_uevent(&dev->kobj, KOBJ_ADD); +} + /* Change owner for sysfs entries when moving network devices across network * namespaces owned by different user namespaces. */ diff --git a/net/core/net-sysfs.h b/net/core/net-sysfs.h index 38e2e3ffd0bd..2f41a4dee866 100644 --- a/net/core/net-sysfs.h +++ b/net/core/net-sysfs.h @@ -4,6 +4,7 @@ int __init netdev_kobject_init(void); int netdev_register_kobject(struct net_device *); +void netdev_uevent_add(struct net_device *dev); void netdev_unregister_kobject(struct net_device *); int net_rx_queue_update_kobjects(struct net_device *, int old_num, int new_num); int netdev_queue_update_kobjects(struct net_device *net, diff --git a/net/core/netdev_work.c b/net/core/netdev_work.c index 3109fae132ad..e721a06d58df 100644 --- a/net/core/netdev_work.c +++ b/net/core/netdev_work.c @@ -31,6 +31,10 @@ static void netdev_work_enqueue(struct net_device *dev, unsigned long events, return; spin_lock_bh(&netdev_work_lock); + if (!dev_isalive(dev)) { + spin_unlock_bh(&netdev_work_lock); + return; + } if (list_empty(&dev->work_node)) { list_add_tail(&dev->work_node, &netdev_work_list); netdev_hold(dev, &dev->work_tracker, GFP_ATOMIC); @@ -61,6 +65,18 @@ netdev_work_dequeue(struct net_device *dev, unsigned long *pending, return events; } +void netdev_work_cancel_all(struct net_device *dev) +{ + spin_lock_bh(&netdev_work_lock); + dev->work_pending = 0; + dev->work_core_pending = 0; + if (!list_empty(&dev->work_node)) { + list_del_init(&dev->work_node); + netdev_put(dev, &dev->work_tracker); + } + spin_unlock_bh(&netdev_work_lock); +} + void netdev_work_sched(struct net_device *dev, unsigned long events) { netdev_work_enqueue(dev, events, 0); diff --git a/net/core/sock.c b/net/core/sock.c index ffa73594c13c..1ad41904db25 100644 --- a/net/core/sock.c +++ b/net/core/sock.c @@ -779,7 +779,6 @@ bool sk_mc_loop(const struct sock *sk) return inet6_test_bit(MC6_LOOP, sk); #endif } - WARN_ON_ONCE(1); return true; } EXPORT_SYMBOL(sk_mc_loop); diff --git a/net/core/xdp.c b/net/core/xdp.c index 9890a30584ba..0194e69da339 100644 --- a/net/core/xdp.c +++ b/net/core/xdp.c @@ -871,7 +871,7 @@ struct xdp_frame *xdpf_clone(struct xdp_frame *xdpf) headroom = xdpf->headroom + sizeof(*xdpf); totalsize = headroom + xdpf->len; - if (unlikely(totalsize > PAGE_SIZE)) + if (unlikely(totalsize > SKB_WITH_OVERHEAD(PAGE_SIZE))) return NULL; page = dev_alloc_page(); if (!page) diff --git a/net/devlink/dev.c b/net/devlink/dev.c index 57b2b8f03543..fd5633fa88ec 100644 --- a/net/devlink/dev.c +++ b/net/devlink/dev.c @@ -578,6 +578,7 @@ int devlink_nl_reload_doit(struct sk_buff *skb, struct genl_info *info) action != DEVLINK_RELOAD_ACTION_DRIVER_REINIT) { NL_SET_ERR_MSG_MOD(info->extack, "Changing namespace is only supported for reinit action"); + put_net(dest_net); return -EOPNOTSUPP; } } diff --git a/net/ipv4/fib_semantics.c b/net/ipv4/fib_semantics.c index 4f3c0740dde9..78f84ae3ee12 100644 --- a/net/ipv4/fib_semantics.c +++ b/net/ipv4/fib_semantics.c @@ -490,6 +490,34 @@ int ip_fib_check_default(__be32 gw, struct net_device *dev) return -1; } +static size_t fib_nexthop_nlmsg_size(const struct fib_nh_common *nhc, + bool skip_oif) +{ + size_t nhsize = 0; + + switch (nhc->nhc_gw_family) { + case AF_INET: + nhsize += nla_total_size(4); /* RTA_GATEWAY */ + break; + case AF_INET6: + nhsize += nla_total_size(sizeof(struct rtvia) + + sizeof(struct in6_addr)); + break; + } + + if (!skip_oif && nhc->nhc_dev) + nhsize += nla_total_size(4); /* RTA_OIF */ + + if (nhc->nhc_lwtstate) { + /* RTA_ENCAP */ + nhsize += lwtunnel_get_encap_size(nhc->nhc_lwtstate); + /* RTA_ENCAP_TYPE */ + nhsize += nla_total_size(2); + } + + return nhsize; +} + size_t fib_nlmsg_size(struct fib_info *fi) { size_t payload = NLMSG_ALIGN(sizeof(struct rtmsg)) @@ -507,32 +535,35 @@ size_t fib_nlmsg_size(struct fib_info *fi) payload += nla_total_size(4); /* RTA_NH_ID */ if (nhs) { - size_t nh_encapsize = 0; - /* Also handles the special case nhs == 1 */ - - /* each nexthop is packed in an attribute */ - size_t nhsize = nla_total_size(sizeof(struct rtnexthop)); + size_t mpsize = 0; unsigned int i; - /* may contain flow and gateway attribute */ - nhsize += 2 * nla_total_size(4); - - /* grab encap info */ for (i = 0; i < fib_info_num_path(fi); i++) { struct fib_nh_common *nhc = fib_info_nhc(fi, i); + size_t nhsize; - if (nhc->nhc_lwtstate) { - /* RTA_ENCAP_TYPE */ - nh_encapsize += lwtunnel_get_encap_size( - nhc->nhc_lwtstate); - /* RTA_ENCAP */ - nh_encapsize += nla_total_size(2); + nhsize = fib_nexthop_nlmsg_size(nhc, nhs != 1); + + if (nhs != 1) + nhsize += NLA_ALIGN(sizeof(struct rtnexthop)); + +#ifdef CONFIG_IP_ROUTE_CLASSID + if (nhc->nhc_family == AF_INET) { + struct fib_nh *nh; + + nh = container_of(nhc, struct fib_nh, nh_common); + if (nh->nh_tclassid) + nhsize += nla_total_size(4); } +#endif + if (nhs == 1) + payload += nhsize; + else + mpsize += nhsize; } - /* all nexthops are packed in a nested attribute */ - payload += nla_total_size((nhs * nhsize) + nh_encapsize); - + if (nhs != 1) + payload += nla_total_size(mpsize); } return payload; diff --git a/net/ipv4/inet_connection_sock.c b/net/ipv4/inet_connection_sock.c index 56902bba5483..6257459bcee2 100644 --- a/net/ipv4/inet_connection_sock.c +++ b/net/ipv4/inet_connection_sock.c @@ -943,11 +943,23 @@ static struct request_sock *inet_reqsk_clone(struct request_sock *req, nreq->rsk_listener = sk; - /* We need not acquire fastopenq->lock - * because the child socket is locked in inet_csk_listen_stop(). - */ - if (sk->sk_protocol == IPPROTO_TCP && tcp_rsk(nreq)->tfo_listener) + if (sk->sk_protocol == IPPROTO_TCP && tcp_rsk(nreq)->tfo_listener) { + struct fastopen_queue *fastopenq; + + /* reqsk_fastopen_remove() will uncharge nreq->rsk_listener, + * that is @sk, so charge it here. Unlike the listener + * being closed, @sk is live and needs its lock. + */ + fastopenq = &inet_csk(sk)->icsk_accept_queue.fastopenq; + spin_lock_bh(&fastopenq->lock); + fastopenq->qlen++; + spin_unlock_bh(&fastopenq->lock); + + /* We need not acquire fastopenq->lock + * because the child socket is locked in inet_csk_listen_stop(). + */ rcu_assign_pointer(tcp_sk(nreq->sk)->fastopen_rsk, nreq); + } return nreq; } diff --git a/net/ipv4/inet_fragment.c b/net/ipv4/inet_fragment.c index 848c0f0c2ed9..fc0cb993959f 100644 --- a/net/ipv4/inet_fragment.c +++ b/net/ipv4/inet_fragment.c @@ -393,8 +393,8 @@ static struct inet_frag_queue *inet_frag_create(struct fqdir *fqdir, *prev = ERR_PTR(-ENOMEM); return NULL; } - mod_timer(&q->timer, jiffies + fqdir->timeout); + spin_lock_bh(&q->lock); *prev = rhashtable_lookup_get_insert_key(&fqdir->rhashtable, &q->key, &q->node, f->rhash_params); if (*prev) { @@ -402,13 +402,13 @@ static struct inet_frag_queue *inet_frag_create(struct fqdir *fqdir, * we need to cancel what inet_frag_alloc() * anticipated. */ - int refs = 1; - q->flags |= INET_FRAG_COMPLETE; - inet_frag_kill(q, &refs); - inet_frag_putn(q, refs); + spin_unlock_bh(&q->lock); + inet_frag_putn(q, 2); return NULL; } + mod_timer(&q->timer, jiffies + fqdir->timeout); + spin_unlock_bh(&q->lock); return q; } diff --git a/net/ipv4/tcp_bpf.c b/net/ipv4/tcp_bpf.c index a30475afb6f8..2e234d155b5e 100644 --- a/net/ipv4/tcp_bpf.c +++ b/net/ipv4/tcp_bpf.c @@ -469,6 +469,7 @@ static int tcp_bpf_send_verdict(struct sock *sk, struct sk_psock *psock, case __SK_REDIRECT: redir_ingress = psock->redir_ingress; sk_redir = psock->sk_redir; + sock_hold(sk_redir); sk_msg_apply_bytes(psock, tosend); if (!psock->apply_bytes) { /* Clean up before releasing the sock lock. */ @@ -489,6 +490,7 @@ static int tcp_bpf_send_verdict(struct sock *sk, struct sk_psock *psock, if (eval == __SK_REDIRECT) sock_put(sk_redir); + sock_put(sk_redir); lock_sock(sk); sk_mem_uncharge(sk, sent); diff --git a/net/ipv4/tcp_input.c b/net/ipv4/tcp_input.c index daff93d51342..5b6378b94701 100644 --- a/net/ipv4/tcp_input.c +++ b/net/ipv4/tcp_input.c @@ -252,7 +252,7 @@ static void tcp_measure_rcv_mss(struct sock *sk, const struct sk_buff *skb) struct tcp_sock *tp = tcp_sk(sk); val = tcp_win_from_space(sk, sk->sk_rcvbuf); - tcp_set_window_clamp(sk, val); + WRITE_ONCE(tp->window_clamp, val); if (tp->window_clamp < tp->rcvq_space.space) tp->rcvq_space.space = tp->window_clamp; diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c index b8887cdd66c5..7f413f509d7d 100644 --- a/net/ipv4/tcp_ipv4.c +++ b/net/ipv4/tcp_ipv4.c @@ -3078,24 +3078,24 @@ static unsigned int bpf_iter_tcp_established_batch(struct seq_file *seq, { struct bpf_tcp_iter_state *iter = seq->private; struct hlist_nulls_node *node; - unsigned int expected = 1; - struct sock *sk; + struct sock *sk = *start_sk; + unsigned int expected = 0; - sock_hold(*start_sk); - iter->batch[iter->end_sk++].sk = *start_sk; - - sk = sk_nulls_next(*start_sk); *start_sk = NULL; sk_nulls_for_each_from(sk, node) { - if (seq_sk_match(seq, sk)) { - if (iter->end_sk < iter->max_sk) { - sock_hold(sk); - iter->batch[iter->end_sk++].sk = sk; - } else if (!*start_sk) { - /* Remember where we left off. */ - *start_sk = sk; - } - expected++; + if (!seq_sk_match(seq, sk)) + continue; + expected++; + if (iter->end_sk < iter->max_sk) { + /* reqsk_queue_hash_req() inserts with sk_refcnt == 0 + * and refcount_set()s it after the bucket lock drops. + */ + if (unlikely(!refcount_inc_not_zero(&sk->sk_refcnt))) + continue; + iter->batch[iter->end_sk++].sk = sk; + } else if (!*start_sk) { + /* Remember where we left off. */ + *start_sk = sk; } } @@ -3133,12 +3133,13 @@ static struct sock *bpf_iter_tcp_batch(struct seq_file *seq) struct sock *sk; int err; +again: sk = bpf_iter_tcp_resume(seq); if (!sk) return NULL; /* Done */ expected = bpf_iter_fill_batch(seq, &sk); - if (likely(iter->end_sk == expected)) + if (likely(!sk)) goto done; /* Batch size was too small. */ @@ -3157,7 +3158,7 @@ static struct sock *bpf_iter_tcp_batch(struct seq_file *seq) return NULL; /* Done */ expected = bpf_iter_fill_batch(seq, &sk); - if (likely(iter->end_sk == expected)) + if (likely(!sk)) goto done; /* Batch size was still too small. Hold onto the lock while we try @@ -3170,10 +3171,14 @@ static struct sock *bpf_iter_tcp_batch(struct seq_file *seq) return ERR_PTR(err); } - expected = bpf_iter_fill_batch(seq, &sk); - WARN_ON_ONCE(iter->end_sk != expected); + bpf_iter_fill_batch(seq, &sk); + WARN_ON_ONCE(sk); done: bpf_iter_tcp_unlock_bucket(seq); + if (unlikely(!iter->end_sk)) { + ++iter->state.bucket; + goto again; + } return iter->batch[0].sk; } diff --git a/net/ipv4/udp_offload.c b/net/ipv4/udp_offload.c index 29651b1a0bc7..abcd3cc2d9a3 100644 --- a/net/ipv4/udp_offload.c +++ b/net/ipv4/udp_offload.c @@ -178,17 +178,19 @@ static struct sk_buff *__skb_udp_tunnel_segment(struct sk_buff *skb, int tnl_hlen = skb_inner_mac_header(skb) - skb_transport_header(skb); bool remcsum, need_csum, offload_csum, gso_partial; struct sk_buff *segs = ERR_PTR(-EINVAL); - struct udphdr *uh = udp_hdr(skb); u16 mac_offset = skb->mac_header; __be16 protocol = skb->protocol; u16 mac_len = skb->mac_len; int udp_offset, outer_hlen; + struct udphdr *uh; __wsum partial; bool need_ipsec; if (unlikely(!pskb_may_pull(skb, tnl_hlen))) goto out; + uh = udp_hdr(skb); + /* Adjust partial header checksum to negate old length. * We cannot rely on the value contained in uh->len as it is * possible that the actual value exceeds the boundaries of the diff --git a/net/ipv6/ip6_tunnel.c b/net/ipv6/ip6_tunnel.c index bf8e40af60b0..ebf83f090376 100644 --- a/net/ipv6/ip6_tunnel.c +++ b/net/ipv6/ip6_tunnel.c @@ -684,6 +684,9 @@ ip6ip6_err(struct sk_buff *skb, struct inet6_skb_parm *opt, if (!skb2) return 0; + /* Remove debris left by outer IPv6 stack. */ + memset(IP6CB(skb2), 0, sizeof(*IP6CB(skb2))); + skb_dst_drop(skb2); skb_pull(skb2, offset); skb_reset_network_header(skb2); diff --git a/net/ipv6/route.c b/net/ipv6/route.c index fc42d67e5822..5968ce5ad150 100644 --- a/net/ipv6/route.c +++ b/net/ipv6/route.c @@ -988,13 +988,13 @@ int rt6_route_rcv(struct net_device *dev, u8 *opt, int len, } else if (rinfo->prefix_len > 128) { return -EINVAL; } else if (rinfo->prefix_len > 64) { - if (rinfo->length < 2) { + /* RFC 4191: Length MUST be 3 when Prefix Length > 64 */ + if (rinfo->length < 3) return -EINVAL; - } } else if (rinfo->prefix_len > 0) { - if (rinfo->length < 1) { + /* RFC 4191: Length MUST be 2 or 3 when Prefix Length > 0 */ + if (rinfo->length < 2) return -EINVAL; - } } pref = rinfo->route_pref; diff --git a/net/mac802154/scan.c b/net/mac802154/scan.c index 65089826ff59..005338f89b75 100644 --- a/net/mac802154/scan.c +++ b/net/mac802154/scan.c @@ -415,6 +415,7 @@ void mac802154_beacon_worker(struct work_struct *work) container_of(work, struct ieee802154_local, beacon_work.work); struct cfg802154_beacon_request *beacon_req; struct ieee802154_sub_if_data *sdata; + netdevice_tracker dev_tracker; struct wpan_dev *wpan_dev; u8 interval; int ret; @@ -427,12 +428,14 @@ void mac802154_beacon_worker(struct work_struct *work) } sdata = IEEE802154_WPAN_DEV_TO_SUB_IF(beacon_req->wpan_dev); + netdev_hold(sdata->dev, &dev_tracker, GFP_ATOMIC); /* Wait an arbitrary amount of time in case we cannot use the device */ if (local->suspended || !ieee802154_sdata_running(sdata)) { rcu_read_unlock(); queue_delayed_work(local->mac_wq, &local->beacon_work, msecs_to_jiffies(1000)); + netdev_put(sdata->dev, &dev_tracker); return; } @@ -450,6 +453,7 @@ void mac802154_beacon_worker(struct work_struct *work) if (interval < IEEE802154_ACTIVE_SCAN_DURATION) queue_delayed_work(local->mac_wq, &local->beacon_work, local->beacon_interval); + netdev_put(sdata->dev, &dev_tracker); } int mac802154_stop_beacons_locked(struct ieee802154_local *local, diff --git a/net/mptcp/fastopen.c b/net/mptcp/fastopen.c index 082c46c0f50e..f717750906ff 100644 --- a/net/mptcp/fastopen.c +++ b/net/mptcp/fastopen.c @@ -24,12 +24,13 @@ void mptcp_fastopen_subflow_synack_set_params(struct mptcp_subflow_context *subf sk = subflow->conn; tp = tcp_sk(ssk); - subflow->is_mptfo = 1; - + /* A valid TFO cookie does not guarantee SYN data. */ skb = skb_peek(&ssk->sk_receive_queue); - if (WARN_ON_ONCE(!skb)) + if (!skb) return; + subflow->is_mptfo = 1; + /* dequeue the skb from sk receive queue */ __skb_unlink(skb, &ssk->sk_receive_queue); skb_ext_reset(skb); diff --git a/net/mptcp/options.c b/net/mptcp/options.c index c664023d37ba..1057d500577b 100644 --- a/net/mptcp/options.c +++ b/net/mptcp/options.c @@ -50,6 +50,14 @@ static void mptcp_parse_option(const struct sk_buff *skb, } } + /* Only the MPC + ACK can be used with a RM_ADDR */ + if (subopt == OPTION_MPTCP_MPC_ACK) { + if ((mp_opt->suboptions & ~OPTION_MPTCP_RM_ADDR) != 0) + break; + } else if (mp_opt->suboptions != 0) { + break; + } + /* Cfr RFC 8684 Section 3.3.0: * If a checksum is present but its use had * not been negotiated in the MP_CAPABLE handshake, the receiver MUST @@ -122,6 +130,11 @@ static void mptcp_parse_option(const struct sk_buff *skb, break; case MPTCPOPT_MP_JOIN: + /* Can be used with a restricted number of other options */ + if ((mp_opt->suboptions & ~(OPTION_MPTCP_RM_ADDR | + OPTION_MPTCP_PRIO)) != 0) + break; + if (opsize == TCPOLEN_MPTCP_MPJ_SYN) { mp_opt->suboptions |= OPTION_MPTCP_MPJ_SYN; mp_opt->backup = *ptr++ & MPTCPOPT_BACKUP; @@ -153,6 +166,14 @@ static void mptcp_parse_option(const struct sk_buff *skb, break; case MPTCPOPT_DSS: + /* Can be used with a restricted number of other options */ + if ((mp_opt->suboptions & ~(OPTION_MPTCP_ADD_ADDR | + OPTION_MPTCP_RM_ADDR | + OPTION_MPTCP_PRIO | + OPTION_MPTCP_FASTCLOSE | + OPTION_MPTCP_FAIL)) != 0) + break; + pr_debug("DSS\n"); ptr++; @@ -188,8 +209,14 @@ static void mptcp_parse_option(const struct sk_buff *skb, * RFC 8684 Section 3.3.0 checks later in subflow_data_ready */ if (opsize != expected_opsize && - opsize != expected_opsize + TCPOLEN_MPTCP_DSS_CHECKSUM) + opsize != expected_opsize + TCPOLEN_MPTCP_DSS_CHECKSUM) { + mp_opt->dsn64 = 0; + mp_opt->use_map = 0; + mp_opt->ack64 = 0; + mp_opt->use_ack = 0; + mp_opt->data_fin = 0; break; + } mp_opt->suboptions |= OPTION_MPTCP_DSS; if (mp_opt->use_ack) { @@ -234,6 +261,12 @@ static void mptcp_parse_option(const struct sk_buff *skb, break; case MPTCPOPT_ADD_ADDR: + /* Can be used with a restricted number of other options */ + if ((mp_opt->suboptions & ~(OPTIONS_MPTCP_DSS | + OPTION_MPTCP_RM_ADDR | + OPTION_MPTCP_PRIO)) != 0) + break; + mp_opt->echo = (*ptr++) & MPTCP_ADDR_ECHO; if (!mp_opt->echo) { if (opsize == TCPOLEN_MPTCP_ADD_ADDR || @@ -293,6 +326,14 @@ static void mptcp_parse_option(const struct sk_buff *skb, break; case MPTCPOPT_RM_ADDR: + /* Can be used with a restricted number of other options */ + if ((mp_opt->suboptions & ~(OPTION_MPTCP_MPC_ACK | + OPTIONS_MPTCP_MPJ | + OPTIONS_MPTCP_DSS | + OPTION_MPTCP_ADD_ADDR | + OPTION_MPTCP_PRIO)) != 0) + break; + if (opsize < TCPOLEN_MPTCP_RM_ADDR_BASE + 1 || opsize > TCPOLEN_MPTCP_RM_ADDR_BASE + MPTCP_RM_IDS_MAX) break; @@ -307,6 +348,13 @@ static void mptcp_parse_option(const struct sk_buff *skb, break; case MPTCPOPT_MP_PRIO: + /* Can be used with a restricted number of other options */ + if ((mp_opt->suboptions & ~(OPTIONS_MPTCP_MPJ | + OPTIONS_MPTCP_DSS | + OPTION_MPTCP_ADD_ADDR | + OPTION_MPTCP_RM_ADDR)) != 0) + break; + if (opsize != TCPOLEN_MPTCP_PRIO) break; @@ -316,6 +364,11 @@ static void mptcp_parse_option(const struct sk_buff *skb, break; case MPTCPOPT_MP_FASTCLOSE: + /* Can be used with a restricted number of other options */ + if ((mp_opt->suboptions & ~(OPTIONS_MPTCP_DSS | + OPTION_MPTCP_RST)) != 0) + break; + if (opsize != TCPOLEN_MPTCP_FASTCLOSE) break; @@ -327,6 +380,11 @@ static void mptcp_parse_option(const struct sk_buff *skb, break; case MPTCPOPT_RST: + /* Can be used with a restricted number of other options */ + if ((mp_opt->suboptions & ~(OPTION_MPTCP_FAIL | + OPTION_MPTCP_FASTCLOSE)) != 0) + break; + if (opsize != TCPOLEN_MPTCP_RST) break; @@ -342,6 +400,11 @@ static void mptcp_parse_option(const struct sk_buff *skb, break; case MPTCPOPT_MP_FAIL: + /* Can be used with a restricted number of other options */ + if ((mp_opt->suboptions & ~(OPTIONS_MPTCP_DSS | + OPTION_MPTCP_RST)) != 0) + break; + if (opsize != TCPOLEN_MPTCP_FAIL) break; @@ -1400,7 +1463,7 @@ void mptcp_write_options(struct tcphdr *th, __be32 *ptr, struct tcp_sock *tp, * RM | C | C | C | P |------|------|------|------| * PRIO | X | C | C | C | C |------|------|------| * FAIL | X | X | C | X | X | X |------|------| - * FC | X | X | X | X | X | X | X |------| + * FC | X | X | P | X | X | X | X |------| * RST | X | X | X | X | X | X | O | O | * ------|------|------|------|------|------|------|------|------| * diff --git a/net/mptcp/pm.c b/net/mptcp/pm.c index 6afd39aea110..64a1236aabee 100644 --- a/net/mptcp/pm.c +++ b/net/mptcp/pm.c @@ -380,6 +380,7 @@ static void mptcp_pm_add_addr_timer(struct timer_list *timer) struct mptcp_sock *msk = entry->sock; struct sock *sk = (struct sock *)msk; unsigned int timeout = 0; + bool retransmit; pr_debug("msk=%p\n", msk); @@ -412,14 +413,15 @@ static void mptcp_pm_add_addr_timer(struct timer_list *timer) entry->retrans_times++; } - if (entry->retrans_times < ADD_ADDR_RETRANS_MAX) + retransmit = entry->retrans_times < ADD_ADDR_RETRANS_MAX; + if (retransmit) timeout <<= entry->retrans_times; else timeout = 0; spin_unlock_bh(&msk->pm.lock); - if (entry->retrans_times == ADD_ADDR_RETRANS_MAX) + if (!retransmit) mptcp_pm_subflow_established(msk); out: @@ -441,6 +443,9 @@ bool mptcp_pm_announced_alloc(struct mptcp_sock *msk, lockdep_assert_held(&msk->pm.lock); + if (msk->pm.status & BIT(MPTCP_PM_DESTROYING)) + return false; + add_entry = mptcp_pm_announced_lookup(msk, addr); if (add_entry) { if (WARN_ON_ONCE(mptcp_pm_is_kernel(msk))) @@ -1143,10 +1148,16 @@ void mptcp_pm_worker(struct mptcp_sock *msk) void mptcp_pm_destroy(struct mptcp_sock *msk) { + spin_lock_bh(&msk->pm.lock); + msk->pm.status |= BIT(MPTCP_PM_DESTROYING); + spin_unlock_bh(&msk->pm.lock); + mptcp_pm_free_announced_list(msk); - if (mptcp_pm_is_userspace(msk)) - mptcp_userspace_pm_free_local_addr_list(msk); + /* Free the userspace local address list unconditionally: the socket + * can be reused (mptcp_disconnect()) and re-selected to a different PM + */ + mptcp_userspace_pm_free_local_addr_list(msk); } void mptcp_pm_data_reset(struct mptcp_sock *msk) diff --git a/net/mptcp/pm_userspace.c b/net/mptcp/pm_userspace.c index 945aa5afc2dd..2203cc2d2748 100644 --- a/net/mptcp/pm_userspace.c +++ b/net/mptcp/pm_userspace.c @@ -54,6 +54,10 @@ static int mptcp_userspace_pm_append_new_local_addr(struct mptcp_sock *msk, bitmap_zero(id_bitmap, MPTCP_PM_MAX_ADDR_ID + 1); spin_lock_bh(&msk->pm.lock); + if (msk->pm.status & BIT(MPTCP_PM_DESTROYING)) { + ret = -EINVAL; + goto append_err; + } mptcp_for_each_userspace_pm_addr(msk, e) { addr_match = mptcp_addresses_equal(&e->addr, &entry->addr, true); if (addr_match && entry->addr.id == 0 && needs_id) diff --git a/net/mptcp/protocol.c b/net/mptcp/protocol.c index ca644ec53eed..7c8180d8d5ef 100644 --- a/net/mptcp/protocol.c +++ b/net/mptcp/protocol.c @@ -149,6 +149,12 @@ struct sock *__mptcp_nmpc_sk(struct mptcp_sock *msk) static void mptcp_drop(struct sock *sk, struct sk_buff *skb) { + /* The skb forward memory was already transferred to sk by + * mptcp_borrow_fwdmem(), even before setting the destructor. + */ + if (!skb->destructor) + sk_mem_reclaim(sk); + sk_drops_skbadd(sk, skb); __kfree_skb(skb); } diff --git a/net/mptcp/protocol.h b/net/mptcp/protocol.h index 4a2d40cd7b13..1b80f2d6ec5a 100644 --- a/net/mptcp/protocol.h +++ b/net/mptcp/protocol.h @@ -37,6 +37,7 @@ OPTION_MPTCP_MPC_ACK) #define OPTIONS_MPTCP_MPJ (OPTION_MPTCP_MPJ_SYN | OPTION_MPTCP_MPJ_SYNACK | \ OPTION_MPTCP_MPJ_ACK) +#define OPTIONS_MPTCP_DSS (OPTION_MPTCP_DSS | OPTION_MPTCP_CSUMREQD) /* MPTCP option subtypes */ #define MPTCPOPT_MP_CAPABLE 0 @@ -189,9 +190,10 @@ enum mptcp_pm_status { MPTCP_PM_ESTABLISHED, MPTCP_PM_SUBFLOW_ESTABLISHED, MPTCP_PM_ALREADY_ESTABLISHED, /* persistent status, set after ESTABLISHED event */ - MPTCP_PM_MPC_ENDPOINT_ACCOUNTED /* persistent status, set after MPC local address is - * accounted int id_avail_bitmap - */ + MPTCP_PM_MPC_ENDPOINT_ACCOUNTED, /* persistent status, set after MPC local address is + * accounted int id_avail_bitmap + */ + MPTCP_PM_DESTROYING, /* To fence out PM list allocs */ }; enum mptcp_pm_type { diff --git a/net/mptcp/subflow.c b/net/mptcp/subflow.c index 8e386899ceb9..e1f20ff8fdb4 100644 --- a/net/mptcp/subflow.c +++ b/net/mptcp/subflow.c @@ -174,8 +174,6 @@ static int subflow_check_req(struct request_sock *req, if (unlikely(listener->pm_listener)) return subflow_reset_req_endp(req, skb); - if (opt_mp_join) - return 0; } else if (opt_mp_join) { SUBFLOW_REQ_INC_STATS(req, MPTCP_MIB_JOINSYNRX); @@ -277,9 +275,6 @@ int mptcp_subflow_init_cookie_req(struct request_sock *req, opt_mp_capable = !!(mp_opt.suboptions & OPTION_MPTCP_MPC_ACK); opt_mp_join = !!(mp_opt.suboptions & OPTION_MPTCP_MPJ_ACK); - if (opt_mp_capable && opt_mp_join) - return -EINVAL; - if (opt_mp_capable && listener->request_mptcp) { if (mp_opt.sndr_key == 0) return -EINVAL; diff --git a/net/ncsi/ncsi-netlink.c b/net/ncsi/ncsi-netlink.c index 2f872d064396..8cc538358f6a 100644 --- a/net/ncsi/ncsi-netlink.c +++ b/net/ncsi/ncsi-netlink.c @@ -461,6 +461,10 @@ static int ncsi_send_cmd_nl(struct sk_buff *msg, struct genl_info *info) nca.req_flags = NCSI_REQ_FLAG_NETLINK_DRIVEN; nca.info = info; nca.payload = ntohs(hdr->length); + if (nca.payload > len - sizeof(*hdr)) { + ret = -EINVAL; + goto out_netlink; + } nca.data = data + sizeof(*hdr); ret = ncsi_xmit_cmd(&nca); diff --git a/net/netfilter/ipset/ip_set_bitmap_gen.h b/net/netfilter/ipset/ip_set_bitmap_gen.h index bb9b5bed10e1..226fdf17b683 100644 --- a/net/netfilter/ipset/ip_set_bitmap_gen.h +++ b/net/netfilter/ipset/ip_set_bitmap_gen.h @@ -77,7 +77,7 @@ mtype_flush(struct ip_set *set) mtype_ext_cleanup(set); bitmap_zero(map->members, map->elements); set->elements = 0; - set->ext_size = 0; + atomic64_set(&set->ext_size, 0); } /* Calculate the actual memory size of the set data */ @@ -93,7 +93,7 @@ mtype_head(struct ip_set *set, struct sk_buff *skb) { const struct mtype *map = set->data; struct nlattr *nested; - size_t memsize = mtype_memsize(map, set->dsize) + set->ext_size; + size_t memsize = mtype_memsize(map, set->dsize) + atomic64_read(&set->ext_size); nested = nla_nest_start(skb, IPSET_ATTR_DATA); if (!nested) diff --git a/net/netfilter/ipset/ip_set_core.c b/net/netfilter/ipset/ip_set_core.c index 6cfad152d7d1..543851a923d0 100644 --- a/net/netfilter/ipset/ip_set_core.c +++ b/net/netfilter/ipset/ip_set_core.c @@ -25,6 +25,7 @@ static LIST_HEAD(ip_set_type_list); /* all registered set types */ static DEFINE_MUTEX(ip_set_type_mutex); /* protects ip_set_type_list */ static DEFINE_RWLOCK(ip_set_ref_lock); /* protects the set refs */ +static struct workqueue_struct *ipset_destroy_wq; struct ip_set_net { struct ip_set * __rcu *ip_set_list; /* all individual sets */ @@ -350,7 +351,7 @@ ip_set_init_comment(struct ip_set *set, struct ip_set_comment *comment, size_t len = ext->comment ? strlen(ext->comment) : 0; if (unlikely(c)) { - set->ext_size -= sizeof(*c) + strlen(c->str) + 1; + atomic64_sub(sizeof(*c) + strlen(c->str) + 1, &set->ext_size); rcu_assign_pointer(comment->c, NULL); kfree_rcu(c, rcu); } @@ -362,7 +363,7 @@ ip_set_init_comment(struct ip_set *set, struct ip_set_comment *comment, if (unlikely(!c)) return; strscpy(c->str, ext->comment, len + 1); - set->ext_size += sizeof(*c) + strlen(c->str) + 1; + atomic64_add(sizeof(*c) + strlen(c->str) + 1, &set->ext_size); rcu_assign_pointer(comment->c, c); } EXPORT_SYMBOL_GPL(ip_set_init_comment); @@ -392,7 +393,7 @@ ip_set_comment_free(struct ip_set *set, void *ptr) c = rcu_dereference_protected(comment->c, 1); if (unlikely(!c)) return; - set->ext_size -= sizeof(*c) + strlen(c->str) + 1; + atomic64_sub(sizeof(*c) + strlen(c->str) + 1, &set->ext_size); rcu_assign_pointer(comment->c, NULL); kfree_rcu(c, rcu); } @@ -1178,22 +1179,26 @@ ip_set_setname_policy[IPSET_ATTR_CMD_MAX + 1] = { .len = IPSET_MAXNAMELEN - 1 }, }; -/* In order to return quickly when destroying a single set, it is split - * into two stages: - * - Cancel garbage collector - * - Destroy the set itself via call_rcu() - */ - static void -ip_set_destroy_set_rcu(struct rcu_head *head) +destroy_and_free_set(struct ip_set *set) { - struct ip_set *set = container_of(head, struct ip_set, rcu); - set->variant->destroy(set); module_put(set->type->me); kfree(set); } +/* In order to return quickly when destroying a single set, + * destruction is done asynchronously via work queues. + */ +static void +ip_set_destroy_set_work(struct work_struct *work) +{ + struct ip_set *set = container_of(to_rcu_work(work), + struct ip_set, rwork); + + destroy_and_free_set(set); +} + static void _destroy_all_sets(struct ip_set_net *inst) { @@ -1283,7 +1288,8 @@ static int ip_set_destroy(struct sk_buff *skb, const struct nfnl_info *info, /* Must wait for flush to be really finished */ rcu_barrier(); } - call_rcu(&s->rcu, ip_set_destroy_set_rcu); + INIT_RCU_WORK(&s->rwork, ip_set_destroy_set_work); + queue_rcu_work(ipset_destroy_wq, &s->rwork); } return 0; out: @@ -2421,18 +2427,23 @@ static struct pernet_operations ip_set_net_ops = { static int __init ip_set_init(void) { - int ret = register_pernet_subsys(&ip_set_net_ops); + int ret; + ipset_destroy_wq = alloc_ordered_workqueue("ipset_destroy_wq", 0); + if (!ipset_destroy_wq) + return -ENOMEM; + + ret = register_pernet_subsys(&ip_set_net_ops); if (ret) { pr_err("ip_set: cannot register pernet_subsys.\n"); - return ret; + goto out_wq; } ret = nfnetlink_subsys_register(&ip_set_netlink_subsys); if (ret != 0) { pr_err("ip_set: cannot register with nfnetlink.\n"); unregister_pernet_subsys(&ip_set_net_ops); - return ret; + goto out_wq; } ret = nf_register_sockopt(&so_set); @@ -2440,10 +2451,13 @@ ip_set_init(void) pr_err("SO_SET registry failed: %d\n", ret); nfnetlink_subsys_unregister(&ip_set_netlink_subsys); unregister_pernet_subsys(&ip_set_net_ops); - return ret; + goto out_wq; } return 0; +out_wq: + destroy_workqueue(ipset_destroy_wq); + return ret; } static void __exit @@ -2453,9 +2467,7 @@ ip_set_fini(void) nfnetlink_subsys_unregister(&ip_set_netlink_subsys); unregister_pernet_subsys(&ip_set_net_ops); - /* Wait for call_rcu() in destroy */ - rcu_barrier(); - + destroy_workqueue(ipset_destroy_wq); pr_debug("these are the famous last words\n"); } diff --git a/net/netfilter/ipset/ip_set_hash_gen.h b/net/netfilter/ipset/ip_set_hash_gen.h index b2d77973272d..f00c82acd7f0 100644 --- a/net/netfilter/ipset/ip_set_hash_gen.h +++ b/net/netfilter/ipset/ip_set_hash_gen.h @@ -99,9 +99,15 @@ struct htable { #endif /* Book-keeping of the prefixes added to the set */ +struct net_prefix { + u8 cidr; /* the cidr value */ + u32 count; /* number of elements of this cidr */ +}; + struct net_prefixes { - u32 nets[IPSET_NET_COUNT]; /* number of elements for this cidr */ - u8 cidr[IPSET_NET_COUNT]; /* the cidr value */ + struct rcu_head rcu; + u8 len; + struct net_prefix nets[] __counted_by(len); }; /* Compute the hash table size */ @@ -127,11 +133,6 @@ htable_size(u8 hbits) #else #define __CIDR(cidr, i) (cidr) #endif - -/* cidr + 1 is stored in net_prefixes to support /0 */ -#define NCIDR_PUT(cidr) ((cidr) + 1) -#define NCIDR_GET(cidr) ((cidr) - 1) - #ifdef IP_SET_HASH_WITH_NETS_PACKED /* When cidr is packed with nomatch, cidr - 1 is stored in the data entry */ #define DCIDR_PUT(cidr) ((cidr) - 1) @@ -141,21 +142,11 @@ htable_size(u8 hbits) #define DCIDR_GET(cidr, i) __CIDR(cidr, i) #endif -#define INIT_CIDR(cidr, host_mask) \ - DCIDR_PUT(((cidr) ? NCIDR_GET(cidr) : host_mask)) +#define INIT_CIDR(n, host_mask) ({ \ + const struct net_prefixes *__n = rcu_dereference(n); \ + DCIDR_PUT((__n)->len ? (__n)->nets[0].cidr : host_mask);\ +}) -#ifdef IP_SET_HASH_WITH_NET0 -/* cidr from 0 to HOST_MASK value and c = cidr + 1 */ -#define NLEN (HOST_MASK + 1) -#define CIDR_POS(c) ((c) - 1) -#else -/* cidr from 1 to HOST_MASK value and c = cidr + 1 */ -#define NLEN HOST_MASK -#define CIDR_POS(c) ((c) - 2) -#endif - -#else -#define NLEN 0 #endif /* IP_SET_HASH_WITH_NETS */ #define SET_ELEM_EXPIRED(set, d) \ @@ -204,12 +195,15 @@ static const union nf_inet_addr zeromask = {}; #undef mtype_ext_cleanup #undef mtype_add_cidr #undef mtype_del_cidr +#undef mtype_del_cidr_all #undef mtype_ahash_memsize #undef mtype_flush #undef mtype_destroy #undef mtype_same_set #undef mtype_kadt #undef mtype_uadt +#undef mtype_bucket_size +#undef mtype_hash_size #undef mtype_add #undef mtype_del @@ -249,12 +243,15 @@ static const union nf_inet_addr zeromask = {}; #define mtype_ext_cleanup IPSET_TOKEN(MTYPE, _ext_cleanup) #define mtype_add_cidr IPSET_TOKEN(MTYPE, _add_cidr) #define mtype_del_cidr IPSET_TOKEN(MTYPE, _del_cidr) +#define mtype_del_cidr_all IPSET_TOKEN(MTYPE, _del_cidr_all) #define mtype_ahash_memsize IPSET_TOKEN(MTYPE, _ahash_memsize) #define mtype_flush IPSET_TOKEN(MTYPE, _flush) #define mtype_destroy IPSET_TOKEN(MTYPE, _destroy) #define mtype_same_set IPSET_TOKEN(MTYPE, _same_set) #define mtype_kadt IPSET_TOKEN(MTYPE, _kadt) #define mtype_uadt IPSET_TOKEN(MTYPE, _uadt) +#define mtype_bucket_size IPSET_TOKEN(MTYPE, _bucket_size) +#define mtype_hash_size IPSET_TOKEN(MTYPE, _hash_size) #define mtype_add IPSET_TOKEN(MTYPE, _add) #define mtype_del IPSET_TOKEN(MTYPE, _del) @@ -292,6 +289,7 @@ static const union nf_inet_addr zeromask = {}; /* The generic hash structure */ struct htype { struct htable __rcu *table; /* the hash table */ + struct net_prefixes __rcu *rnets[IPSET_NET_COUNT]; /* cidr prefixes */ struct htable_gc gc; /* gc workqueue */ u32 maxelem; /* max elements in the hash */ u32 initval; /* random jhash init value */ @@ -302,9 +300,6 @@ struct htype { #if defined(IP_SET_HASH_WITH_NETMASK) || defined(IP_SET_HASH_WITH_BITMASK) u8 netmask; /* netmask value for subnets to store */ union nf_inet_addr bitmask; /* stores bitmask */ -#endif -#ifdef IP_SET_HASH_WITH_NETS - struct net_prefixes nets[NLEN]; /* book-keeping of prefixes */ #endif /* Because 'next' is IPv4/IPv6 dependent, no elements of this * structure and referred in create() may come after 'next'. @@ -326,55 +321,108 @@ struct mtype_resize_ad { /* Network cidr size book keeping when the hash stores different * sized networks. cidr == real cidr + 1 to support /0. */ -static void +static int mtype_add_cidr(struct ip_set *set, struct htype *h, u8 cidr, u8 n) { - int i, j; + struct net_prefixes *nets, *tmp; + int i, j, found, len = 0, ret = 0; spin_lock_bh(&set->lock); + nets = __ipset_dereference(h->rnets[n]); /* Add in increasing prefix order, so larger cidr first */ - for (i = 0, j = -1; i < NLEN && h->nets[i].cidr[n]; i++) { - if (j != -1) { + for (i = 0, found = -1; i < nets->len; i++) { + if (nets->nets[i].count) + len++; + if (found != -1) { continue; - } else if (h->nets[i].cidr[n] < cidr) { - j = i; - } else if (h->nets[i].cidr[n] == cidr) { - h->nets[CIDR_POS(cidr)].nets[n]++; + } else if (nets->nets[i].cidr < cidr) { + found = i; + } else if (nets->nets[i].cidr == cidr) { + nets->nets[i].count++; goto unlock; } } - if (j != -1) { - for (; i > j; i--) - h->nets[i].cidr[n] = h->nets[i - 1].cidr[n]; + len++; + tmp = kzalloc_flex(*tmp, nets, len, GFP_ATOMIC); + if (!tmp) { + ret = -ENOMEM; + goto unlock; } - h->nets[i].cidr[n] = cidr; - h->nets[CIDR_POS(cidr)].nets[n] = 1; + + tmp->len = len; + for (i = 0, j = 0; i < nets->len; i++) { + if (i == found) { + tmp->nets[j].cidr = cidr; + tmp->nets[j++].count = 1; + } + if (!nets->nets[i].count) + continue; + tmp->nets[j].cidr = nets->nets[i].cidr; + tmp->nets[j++].count = nets->nets[i].count; + } + if (found == -1) { + tmp->nets[j].cidr = cidr; + tmp->nets[j].count = 1; + } + rcu_assign_pointer(h->rnets[n], tmp); + kfree_rcu(nets, rcu); unlock: spin_unlock_bh(&set->lock); + return ret; } static void mtype_del_cidr(struct ip_set *set, struct htype *h, u8 cidr, u8 n) { - u8 i, j, net_end = NLEN - 1; + struct net_prefixes *nets, *tmp; + u8 i, j, len = 0; + int found; spin_lock_bh(&set->lock); - for (i = 0; i < NLEN; i++) { - if (h->nets[i].cidr[n] != cidr) - continue; - h->nets[CIDR_POS(cidr)].nets[n]--; - if (h->nets[CIDR_POS(cidr)].nets[n] > 0) - goto unlock; - for (j = i; j < net_end && h->nets[j].cidr[n]; j++) - h->nets[j].cidr[n] = h->nets[j + 1].cidr[n]; - h->nets[j].cidr[n] = 0; - goto unlock; + nets = __ipset_dereference(h->rnets[n]); + for (i = 0, found = -1; i < nets->len; i++) { + if (nets->nets[i].count) + len++; + if (nets->nets[i].cidr == cidr) + found = i; } + if (unlikely(found == -1)) + goto unlock; + + nets->nets[found].count--; + if (nets->nets[found].count) + goto unlock; + len--; + tmp = kzalloc_flex(*tmp, nets, len, GFP_ATOMIC); + if (!tmp) + /* Leave a hole */ + goto unlock; + + tmp->len = len; + for (i = 0, j = 0; i < nets->len; i++) { + if (!nets->nets[i].count || i == found) + continue; + tmp->nets[j].cidr = nets->nets[i].cidr; + tmp->nets[j++].count = nets->nets[i].count; + } + rcu_assign_pointer(h->rnets[n], tmp); + kfree_rcu(nets, rcu); unlock: spin_unlock_bh(&set->lock); } #endif +static void +mtype_del_cidr_all(struct ip_set *set, struct htype *h, const struct mtype_elem *data) +{ +#ifdef IP_SET_HASH_WITH_NETS + int k; + + for (k = 0; k < IPSET_NET_COUNT; k++) + mtype_del_cidr(set, h, DCIDR_GET(data->cidr, k), k); +#endif +} + /* Calculate the actual memory size of the set data */ static size_t mtype_ahash_memsize(const struct htype *h, const struct htable *t) @@ -402,6 +450,9 @@ static void mtype_flush(struct ip_set *set) { struct htype *h = set->data; +#ifdef IP_SET_HASH_WITH_NETS + struct net_prefixes *nets, *tmp; +#endif struct htable *t; struct hbucket *n; u32 r, i; @@ -425,7 +476,19 @@ mtype_flush(struct ip_set *set) spin_unlock_bh(&t->hregion[r].lock); } #ifdef IP_SET_HASH_WITH_NETS - memset(h->nets, 0, sizeof(h->nets)); + for (i = 0; i < IPSET_NET_COUNT; i++) { + nets = ipset_dereference_nfnl(h->rnets[i]); + tmp = kzalloc_obj(*tmp, GFP_ATOMIC); + if (!tmp) { + u8 j; + + for (j = 0; j < nets->len; j++) + nets->nets[j].count = 0; + } else { + rcu_assign_pointer(h->rnets[i], tmp); + kfree_rcu(nets, rcu); + } + } #endif } @@ -433,6 +496,9 @@ mtype_flush(struct ip_set *set) static void mtype_ahash_destroy(struct ip_set *set, struct htable *t, bool ext_destroy) { +#ifdef IP_SET_HASH_WITH_NETS + struct htype *h = set->data; +#endif struct hbucket *n; u32 i; @@ -446,6 +512,11 @@ mtype_ahash_destroy(struct ip_set *set, struct htable *t, bool ext_destroy) kfree(n); } +#ifdef IP_SET_HASH_WITH_NETS + if (ext_destroy) + for (i = 0; i < IPSET_NET_COUNT; i++) + kfree(rcu_dereference_raw(h->rnets[i])); +#endif ip_set_free(t->hregion); ip_set_free(t); } @@ -493,9 +564,6 @@ mtype_gc_do(struct ip_set *set, struct htype *h, struct htable *t, u32 r) struct mtype_elem *data; u32 i, j, d; size_t dsize = set->dsize; -#ifdef IP_SET_HASH_WITH_NETS - u8 k; -#endif u8 pos, htable_bits = t->htable_bits; spin_lock_bh(&t->hregion[r].lock); @@ -516,12 +584,7 @@ mtype_gc_do(struct ip_set *set, struct htype *h, struct htable *t, u32 r) pr_debug("expired %u/%u\n", i, j); clear_bit(j, n->used); smp_mb__after_atomic(); -#ifdef IP_SET_HASH_WITH_NETS - for (k = 0; k < IPSET_NET_COUNT; k++) - mtype_del_cidr(set, h, - NCIDR_PUT(DCIDR_GET(data->cidr, k)), - k); -#endif + mtype_del_cidr_all(set, h, data); t->hregion[r].elements--; ip_set_ext_destroy(set, data); d++; @@ -947,12 +1010,7 @@ mtype_add(struct ip_set *set, void *value, const struct ip_set_ext *ext, j = 0; data = ahash_data(n, j, set->dsize); if (!deleted) { -#ifdef IP_SET_HASH_WITH_NETS - for (i = 0; i < IPSET_NET_COUNT; i++) - mtype_del_cidr(set, h, - NCIDR_PUT(DCIDR_GET(data->cidr, i)), - i); -#endif + mtype_del_cidr_all(set, h, data); ip_set_ext_destroy(set, data); t->hregion[r].elements--; } @@ -996,7 +1054,7 @@ mtype_add(struct ip_set *set, void *value, const struct ip_set_ext *ext, t->hregion[r].elements++; #ifdef IP_SET_HASH_WITH_NETS for (i = 0; i < IPSET_NET_COUNT; i++) - mtype_add_cidr(set, h, NCIDR_PUT(DCIDR_GET(d->cidr, i)), i); + mtype_add_cidr(set, h, DCIDR_GET(d->cidr, i), i); #endif memcpy(data, d, sizeof(struct mtype_elem)); overwrite_extensions: @@ -1107,11 +1165,7 @@ mtype_del(struct ip_set *set, void *value, const struct ip_set_ext *ext, if (i + 1 == pos) smp_store_release(&n->pos, --pos); t->hregion[r].elements--; -#ifdef IP_SET_HASH_WITH_NETS - for (j = 0; j < IPSET_NET_COUNT; j++) - mtype_del_cidr(set, h, - NCIDR_PUT(DCIDR_GET(d->cidr, j)), j); -#endif + mtype_del_cidr_all(set, h, d); ip_set_ext_destroy(set, data); if (t->resizing && ext && ext->target) { @@ -1193,28 +1247,37 @@ mtype_test_cidrs(struct ip_set *set, struct mtype_elem *d, { struct htype *h = set->data; struct htable *t = rcu_dereference_bh(h->table); + struct net_prefixes *nets0; struct hbucket *n; struct mtype_elem *data; #if IPSET_NET_COUNT == 2 + struct net_prefixes *nets1; struct mtype_elem orig = *d; - int ret, i, j = 0, k; + int ret, i, j, k; #else - int ret, i, j = 0; + int ret, i, j; #endif u32 key, multi = 0; u8 pos; pr_debug("test by nets\n"); - for (; j < NLEN && h->nets[j].cidr[0] && !multi; j++) { + rcu_read_lock_bh(); + nets0 = rcu_dereference_bh(h->rnets[0]); +#if IPSET_NET_COUNT == 2 + nets1 = rcu_dereference_bh(h->rnets[1]); +#endif + for (j = 0; j < nets0->len && !multi; j++) { + if (!nets0->nets[j].count) + continue; #if IPSET_NET_COUNT == 2 mtype_data_reset_elem(d, &orig); - mtype_data_netmask(d, NCIDR_GET(h->nets[j].cidr[0]), false); - for (k = 0; k < NLEN && h->nets[k].cidr[1] && !multi; - k++) { - mtype_data_netmask(d, NCIDR_GET(h->nets[k].cidr[1]), - true); + mtype_data_netmask(d, nets0->nets[j].cidr, false); + for (k = 0; k < nets1->len && !multi; k++) { + if (!nets1->nets[k].count) + continue; + mtype_data_netmask(d, nets1->nets[k].cidr, true); #else - mtype_data_netmask(d, NCIDR_GET(h->nets[j].cidr[0])); + mtype_data_netmask(d, nets0->nets[j].cidr); #endif key = HKEY(d, h->initval, t->htable_bits); n = rcu_dereference_bh(hbucket(t, key)); @@ -1229,7 +1292,7 @@ mtype_test_cidrs(struct ip_set *set, struct mtype_elem *d, continue; ret = mtype_data_match(data, ext, mext, set, flags); if (ret != 0) - return ret; + goto unlock; #ifdef IP_SET_HASH_WITH_MULTI /* No match, reset multiple match flag */ multi = 0; @@ -1239,7 +1302,10 @@ mtype_test_cidrs(struct ip_set *set, struct mtype_elem *d, } #endif } - return 0; + ret = 0; +unlock: + rcu_read_unlock_bh(); + return ret; } #endif @@ -1294,6 +1360,24 @@ mtype_test(struct ip_set *set, void *value, const struct ip_set_ext *ext, return ret; } +static u32 mtype_hash_size(const struct htype *h) +{ + const struct htable *t; + u8 htable_bits; + + rcu_read_lock(); + t = rcu_dereference(h->table); + htable_bits = t->htable_bits; + rcu_read_unlock(); + + return jhash_size(htable_bits); +} + +static u32 mtype_bucket_size(const struct htype *h) +{ + return h->bucketsize; +} + /* Reply a HEADER request: fill out the header part of the set */ static int mtype_head(struct ip_set *set, struct sk_buff *skb) @@ -1304,21 +1388,20 @@ mtype_head(struct ip_set *set, struct sk_buff *skb) size_t memsize; u32 elements = 0; size_t ext_size = 0; - u8 htable_bits; rcu_read_lock_bh(); t = rcu_dereference_bh(h->table); mtype_ext_size(set, &elements, &ext_size); - memsize = mtype_ahash_memsize(h, t) + ext_size + set->ext_size; - htable_bits = t->htable_bits; + memsize = mtype_ahash_memsize(h, t) + ext_size + atomic64_read(&set->ext_size); rcu_read_unlock_bh(); nested = nla_nest_start(skb, IPSET_ATTR_DATA); if (!nested) goto nla_put_failure; - if (nla_put_net32(skb, IPSET_ATTR_HASHSIZE, - htonl(jhash_size(htable_bits))) || - nla_put_net32(skb, IPSET_ATTR_MAXELEM, htonl(h->maxelem))) + + if (nla_put_net32(skb, IPSET_ATTR_HASHSIZE, htonl(mtype_hash_size(h)))) + goto nla_put_failure; + if (nla_put_net32(skb, IPSET_ATTR_MAXELEM, htonl(h->maxelem))) goto nla_put_failure; #ifdef IP_SET_HASH_WITH_BITMASK /* if netmask is set to anything other than HOST_MASK we know that the user supplied netmask @@ -1342,8 +1425,9 @@ mtype_head(struct ip_set *set, struct sk_buff *skb) goto nla_put_failure; #endif if (set->flags & IPSET_CREATE_FLAG_BUCKETSIZE) { - if (nla_put_u8(skb, IPSET_ATTR_BUCKETSIZE, h->bucketsize) || - nla_put_net32(skb, IPSET_ATTR_INITVAL, htonl(h->initval))) + if (nla_put_u8(skb, IPSET_ATTR_BUCKETSIZE, mtype_bucket_size(h))) + goto nla_put_failure; + if (nla_put_net32(skb, IPSET_ATTR_INITVAL, htonl(h->initval))) goto nla_put_failure; } if (nla_put_net32(skb, IPSET_ATTR_REFERENCES, htonl(set->ref)) || @@ -1504,6 +1588,9 @@ IPSET_TOKEN(HTYPE, _create)(struct net *net, struct ip_set *set, int ret __attribute__((unused)) = 0; u8 netmask = set->family == NFPROTO_IPV4 ? 32 : 128; union nf_inet_addr bitmask = onesmask; +#endif +#ifdef IP_SET_HASH_WITH_NETS + struct net_prefixes *nets; #endif size_t hsize; struct htype *h; @@ -1604,21 +1691,25 @@ IPSET_TOKEN(HTYPE, _create)(struct net *net, struct ip_set *set, */ hbits = fls(hashsize - 1); hsize = htable_size(hbits); - if (hsize == 0) { - kfree(h); - return -ENOMEM; - } + if (hsize == 0) + goto free_h; t = ip_set_alloc(hsize); - if (!t) { - kfree(h); - return -ENOMEM; - } + if (!t) + goto free_h; t->hregion = ip_set_alloc(ahash_sizeof_regions(hbits)); - if (!t->hregion) { - ip_set_free(t); - kfree(h); - return -ENOMEM; + if (!t->hregion) + goto free_t; +#ifdef IP_SET_HASH_WITH_NETS + for (i = 0; i < IPSET_NET_COUNT; i++) { + nets = kzalloc_obj(*nets); + if (!nets) { + while (i > 0) + kfree(rcu_dereference_raw(h->rnets[--i])); + goto free_hregion; + } + RCU_INIT_POINTER(h->rnets[i], nets); } +#endif h->gc.set = set; spin_lock_init(&h->gc.lock); for (i = 0; i < ahash_numof_locks(hbits); i++) @@ -1650,6 +1741,7 @@ IPSET_TOKEN(HTYPE, _create)(struct net *net, struct ip_set *set, INIT_LIST_HEAD(&t->ad); RCU_INIT_POINTER(h->table, t); set->data = h; + #ifndef IP_SET_PROTO_UNDEF if (set->family == NFPROTO_IPV4) { #endif @@ -1678,10 +1770,20 @@ IPSET_TOKEN(HTYPE, _create)(struct net *net, struct ip_set *set, #endif } pr_debug("create %s hashsize %u (%u) maxelem %u: %p(%p)\n", - set->name, jhash_size(t->htable_bits), + set->name, mtype_hash_size(h), t->htable_bits, h->maxelem, set->data, t); return 0; + +#ifdef IP_SET_HASH_WITH_NETS +free_hregion: + ip_set_free(t->hregion); +#endif +free_t: + ip_set_free(t); +free_h: + kfree(h); + return -ENOMEM; } #endif /* IP_SET_EMIT_CREATE */ diff --git a/net/netfilter/ipset/ip_set_hash_ipportnet.c b/net/netfilter/ipset/ip_set_hash_ipportnet.c index 2d6652d43199..195853a25b06 100644 --- a/net/netfilter/ipset/ip_set_hash_ipportnet.c +++ b/net/netfilter/ipset/ip_set_hash_ipportnet.c @@ -138,7 +138,7 @@ hash_ipportnet4_kadt(struct ip_set *set, const struct sk_buff *skb, const struct hash_ipportnet4 *h = set->data; ipset_adtfn adtfn = set->variant->adt[adt]; struct hash_ipportnet4_elem e = { - .cidr = INIT_CIDR(h->nets[0].cidr[0], HOST_MASK), + .cidr = INIT_CIDR(h->rnets[0], HOST_MASK), }; struct ip_set_ext ext = IP_SET_INIT_KEXT(skb, opt, set); @@ -398,7 +398,7 @@ hash_ipportnet6_kadt(struct ip_set *set, const struct sk_buff *skb, const struct hash_ipportnet6 *h = set->data; ipset_adtfn adtfn = set->variant->adt[adt]; struct hash_ipportnet6_elem e = { - .cidr = INIT_CIDR(h->nets[0].cidr[0], HOST_MASK), + .cidr = INIT_CIDR(h->rnets[0], HOST_MASK), }; struct ip_set_ext ext = IP_SET_INIT_KEXT(skb, opt, set); diff --git a/net/netfilter/ipset/ip_set_hash_net.c b/net/netfilter/ipset/ip_set_hash_net.c index ce0a9ce5a91f..092f3c9281b8 100644 --- a/net/netfilter/ipset/ip_set_hash_net.c +++ b/net/netfilter/ipset/ip_set_hash_net.c @@ -117,7 +117,7 @@ hash_net4_kadt(struct ip_set *set, const struct sk_buff *skb, const struct hash_net4 *h = set->data; ipset_adtfn adtfn = set->variant->adt[adt]; struct hash_net4_elem e = { - .cidr = INIT_CIDR(h->nets[0].cidr[0], HOST_MASK), + .cidr = INIT_CIDR(h->rnets[0], HOST_MASK), }; struct ip_set_ext ext = IP_SET_INIT_KEXT(skb, opt, set); @@ -291,7 +291,7 @@ hash_net6_kadt(struct ip_set *set, const struct sk_buff *skb, const struct hash_net6 *h = set->data; ipset_adtfn adtfn = set->variant->adt[adt]; struct hash_net6_elem e = { - .cidr = INIT_CIDR(h->nets[0].cidr[0], HOST_MASK), + .cidr = INIT_CIDR(h->rnets[0], HOST_MASK), }; struct ip_set_ext ext = IP_SET_INIT_KEXT(skb, opt, set); diff --git a/net/netfilter/ipset/ip_set_hash_netiface.c b/net/netfilter/ipset/ip_set_hash_netiface.c index 30a655e5c4fd..b44b95f766b7 100644 --- a/net/netfilter/ipset/ip_set_hash_netiface.c +++ b/net/netfilter/ipset/ip_set_hash_netiface.c @@ -161,7 +161,7 @@ hash_netiface4_kadt(struct ip_set *set, const struct sk_buff *skb, struct hash_netiface4 *h = set->data; ipset_adtfn adtfn = set->variant->adt[adt]; struct hash_netiface4_elem e = { - .cidr = INIT_CIDR(h->nets[0].cidr[0], HOST_MASK), + .cidr = INIT_CIDR(h->rnets[0], HOST_MASK), .elem = 1, }; struct ip_set_ext ext = IP_SET_INIT_KEXT(skb, opt, set); @@ -382,7 +382,7 @@ hash_netiface6_kadt(struct ip_set *set, const struct sk_buff *skb, struct hash_netiface6 *h = set->data; ipset_adtfn adtfn = set->variant->adt[adt]; struct hash_netiface6_elem e = { - .cidr = INIT_CIDR(h->nets[0].cidr[0], HOST_MASK), + .cidr = INIT_CIDR(h->rnets[0], HOST_MASK), .elem = 1, }; struct ip_set_ext ext = IP_SET_INIT_KEXT(skb, opt, set); diff --git a/net/netfilter/ipset/ip_set_hash_netnet.c b/net/netfilter/ipset/ip_set_hash_netnet.c index 8fbe649c9dd3..f7c8a1cc30fc 100644 --- a/net/netfilter/ipset/ip_set_hash_netnet.c +++ b/net/netfilter/ipset/ip_set_hash_netnet.c @@ -149,8 +149,10 @@ hash_netnet4_kadt(struct ip_set *set, const struct sk_buff *skb, struct hash_netnet4_elem e = { }; struct ip_set_ext ext = IP_SET_INIT_KEXT(skb, opt, set); - e.cidr[0] = INIT_CIDR(h->nets[0].cidr[0], HOST_MASK); - e.cidr[1] = INIT_CIDR(h->nets[0].cidr[1], HOST_MASK); + rcu_read_lock_bh(); + e.cidr[0] = INIT_CIDR(h->rnets[0], HOST_MASK); + e.cidr[1] = INIT_CIDR(h->rnets[1], HOST_MASK); + rcu_read_unlock_bh(); if (adt == IPSET_TEST) e.ccmp = (HOST_MASK << (sizeof(e.cidr[0]) * 8)) | HOST_MASK; @@ -388,8 +390,10 @@ hash_netnet6_kadt(struct ip_set *set, const struct sk_buff *skb, struct hash_netnet6_elem e = { }; struct ip_set_ext ext = IP_SET_INIT_KEXT(skb, opt, set); - e.cidr[0] = INIT_CIDR(h->nets[0].cidr[0], HOST_MASK); - e.cidr[1] = INIT_CIDR(h->nets[0].cidr[1], HOST_MASK); + rcu_read_lock_bh(); + e.cidr[0] = INIT_CIDR(h->rnets[0], HOST_MASK); + e.cidr[1] = INIT_CIDR(h->rnets[1], HOST_MASK); + rcu_read_unlock_bh(); if (adt == IPSET_TEST) e.ccmp = (HOST_MASK << (sizeof(u8) * 8)) | HOST_MASK; diff --git a/net/netfilter/ipset/ip_set_hash_netport.c b/net/netfilter/ipset/ip_set_hash_netport.c index d1a0628df4ef..5de4b511de76 100644 --- a/net/netfilter/ipset/ip_set_hash_netport.c +++ b/net/netfilter/ipset/ip_set_hash_netport.c @@ -133,7 +133,7 @@ hash_netport4_kadt(struct ip_set *set, const struct sk_buff *skb, const struct hash_netport4 *h = set->data; ipset_adtfn adtfn = set->variant->adt[adt]; struct hash_netport4_elem e = { - .cidr = INIT_CIDR(h->nets[0].cidr[0], HOST_MASK), + .cidr = INIT_CIDR(h->rnets[0], HOST_MASK), }; struct ip_set_ext ext = IP_SET_INIT_KEXT(skb, opt, set); @@ -353,7 +353,7 @@ hash_netport6_kadt(struct ip_set *set, const struct sk_buff *skb, const struct hash_netport6 *h = set->data; ipset_adtfn adtfn = set->variant->adt[adt]; struct hash_netport6_elem e = { - .cidr = INIT_CIDR(h->nets[0].cidr[0], HOST_MASK), + .cidr = INIT_CIDR(h->rnets[0], HOST_MASK), }; struct ip_set_ext ext = IP_SET_INIT_KEXT(skb, opt, set); diff --git a/net/netfilter/ipset/ip_set_hash_netportnet.c b/net/netfilter/ipset/ip_set_hash_netportnet.c index bf4f91b78e1d..6291532be7a5 100644 --- a/net/netfilter/ipset/ip_set_hash_netportnet.c +++ b/net/netfilter/ipset/ip_set_hash_netportnet.c @@ -157,8 +157,10 @@ hash_netportnet4_kadt(struct ip_set *set, const struct sk_buff *skb, struct hash_netportnet4_elem e = { }; struct ip_set_ext ext = IP_SET_INIT_KEXT(skb, opt, set); - e.cidr[0] = INIT_CIDR(h->nets[0].cidr[0], HOST_MASK); - e.cidr[1] = INIT_CIDR(h->nets[0].cidr[1], HOST_MASK); + rcu_read_lock_bh(); + e.cidr[0] = INIT_CIDR(h->rnets[0], HOST_MASK); + e.cidr[1] = INIT_CIDR(h->rnets[1], HOST_MASK); + rcu_read_unlock_bh(); if (adt == IPSET_TEST) e.ccmp = (HOST_MASK << (sizeof(e.cidr[0]) * 8)) | HOST_MASK; @@ -452,8 +454,10 @@ hash_netportnet6_kadt(struct ip_set *set, const struct sk_buff *skb, struct hash_netportnet6_elem e = { }; struct ip_set_ext ext = IP_SET_INIT_KEXT(skb, opt, set); - e.cidr[0] = INIT_CIDR(h->nets[0].cidr[0], HOST_MASK); - e.cidr[1] = INIT_CIDR(h->nets[0].cidr[1], HOST_MASK); + rcu_read_lock_bh(); + e.cidr[0] = INIT_CIDR(h->rnets[0], HOST_MASK); + e.cidr[1] = INIT_CIDR(h->rnets[1], HOST_MASK); + rcu_read_unlock_bh(); if (adt == IPSET_TEST) e.ccmp = (HOST_MASK << (sizeof(u8) * 8)) | HOST_MASK; diff --git a/net/netfilter/ipset/ip_set_list_set.c b/net/netfilter/ipset/ip_set_list_set.c index 1cef84f15e8c..ca3ef9479e83 100644 --- a/net/netfilter/ipset/ip_set_list_set.c +++ b/net/netfilter/ipset/ip_set_list_set.c @@ -421,7 +421,7 @@ list_set_flush(struct ip_set *set) list_for_each_entry_safe(e, n, &map->members, list) list_set_del(set, e); set->elements = 0; - set->ext_size = 0; + atomic64_set(&set->ext_size, 0); } static void @@ -455,7 +455,7 @@ list_set_head(struct ip_set *set, struct sk_buff *skb) { const struct list_set *map = set->data; struct nlattr *nested; - size_t memsize = list_set_memsize(map, set->dsize) + set->ext_size; + size_t memsize = list_set_memsize(map, set->dsize) + atomic64_read(&set->ext_size); nested = nla_nest_start(skb, IPSET_ATTR_DATA); if (!nested) diff --git a/net/netfilter/ipvs/ip_vs_core.c b/net/netfilter/ipvs/ip_vs_core.c index 6b79e0c4d9e2..0bdaeb4ed61e 100644 --- a/net/netfilter/ipvs/ip_vs_core.c +++ b/net/netfilter/ipvs/ip_vs_core.c @@ -925,28 +925,27 @@ static int ip_vs_route_me_harder(struct netns_ipvs *ipvs, int af, */ void ip_vs_nat_icmp(struct sk_buff *skb, struct ip_vs_protocol *pp, struct ip_vs_conn *cp, int inout, unsigned int toff, - bool has_ports) + bool has_ports, struct ip_vs_iphdr *ciph) { struct iphdr *iph = ip_hdr(skb); struct icmphdr *icmph = (struct icmphdr *)(skb->data + toff); - struct iphdr *ciph = (struct iphdr *)(icmph + 1); - unsigned int coff __maybe_unused = toff + sizeof(struct icmphdr); + struct iphdr *cih = (struct iphdr *)(icmph + 1); if (inout) { iph->saddr = cp->vaddr.ip; ip_send_check(iph); - ciph->daddr = cp->vaddr.ip; - ip_send_check(ciph); + cih->daddr = cp->vaddr.ip; + ip_send_check(cih); } else { iph->daddr = cp->daddr.ip; ip_send_check(iph); - ciph->saddr = cp->daddr.ip; - ip_send_check(ciph); + cih->saddr = cp->daddr.ip; + ip_send_check(cih); } /* the TCP/UDP/SCTP port */ if (has_ports) { - __be16 *ports = (void *)ciph + ciph->ihl*4; + __be16 *ports = (void *)(skb->data + ciph->len); if (inout) ports[1] = cp->vport; @@ -960,10 +959,10 @@ void ip_vs_nat_icmp(struct sk_buff *skb, struct ip_vs_protocol *pp, skb->ip_summed = CHECKSUM_UNNECESSARY; if (inout) - IP_VS_DBG_PKT(11, AF_INET, pp, skb, coff, + IP_VS_DBG_PKT(11, AF_INET, pp, skb, ciph->off, "Forwarding altered outgoing ICMP"); else - IP_VS_DBG_PKT(11, AF_INET, pp, skb, coff, + IP_VS_DBG_PKT(11, AF_INET, pp, skb, ciph->off, "Forwarding altered incoming ICMP"); } @@ -1056,7 +1055,7 @@ static int handle_response_icmp(int af, struct sk_buff *skb, ip_vs_nat_icmp_v6(skb, pp, cp, 1, toff, has_ports, ciph); else #endif - ip_vs_nat_icmp(skb, pp, cp, 1, toff, has_ports); + ip_vs_nat_icmp(skb, pp, cp, 1, toff, has_ports, ciph); if (ip_vs_route_me_harder(cp->ipvs, af, skb, hooknum)) goto out; @@ -1092,7 +1091,7 @@ static int ip_vs_out_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, struct ip_vs_iphdr ciph; struct ip_vs_conn *cp; struct ip_vs_protocol *pp; - unsigned int offset, ihl; + unsigned int offset; union nf_inet_addr snet; *related = 1; @@ -1105,7 +1104,6 @@ static int ip_vs_out_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, return NF_ACCEPT; } - ihl = ipvsh->len; offset = ipvsh->len; ic = skb_header_pointer(skb, offset, sizeof(_icmph), &_icmph); if (ic == NULL) @@ -1131,11 +1129,15 @@ static int ip_vs_out_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, /* Now find the contained IP header */ offset += sizeof(_icmph); - cih = skb_header_pointer(skb, offset, sizeof(_ciph), &_ciph); - if (!(cih && cih->version == 4 && cih->ihl >= 5)) + if (!ip_vs_fill_iph_skb_icmp(AF_INET, skb, offset, true, &ciph)) return NF_ACCEPT; /* The packet looks wrong, ignore */ - pp = ip_vs_proto_get(cih->protocol); + cih = skb_header_pointer(skb, offset, sizeof(_ciph), &_ciph); + if (!(cih && cih->version == 4 && + ciph.len - ciph.off >= sizeof(struct iphdr))) + return NF_ACCEPT; /* The packet looks wrong, ignore */ + + pp = ip_vs_proto_get(ciph.protocol); if (!pp) return NF_ACCEPT; @@ -1146,8 +1148,6 @@ static int ip_vs_out_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, IP_VS_DBG_PKT(11, AF_INET, pp, skb, offset, "Checking outgoing ICMP for"); - ip_vs_fill_iph_skb_icmp(AF_INET, skb, offset, true, &ciph); - /* The embedded headers contain source and dest in reverse order */ cp = INDIRECT_CALL_1(pp->conn_out_get, ip_vs_conn_out_get_proto, ipvs, AF_INET, skb, &ciph); @@ -1155,8 +1155,8 @@ static int ip_vs_out_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, return NF_ACCEPT; snet.ip = ipvsh->saddr.ip; - return handle_response_icmp(AF_INET, skb, &snet, cp, pp, &ciph, ihl, - hooknum); + return handle_response_icmp(AF_INET, skb, &snet, cp, pp, &ciph, + ipvsh->len, hooknum); } #ifdef CONFIG_IP_VS_IPV6 @@ -1803,10 +1803,12 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related, /* Now find the contained IP header */ offset += sizeof(_icmph); cih = skb_header_pointer(skb, offset, sizeof(_ciph), &_ciph); - if (!(cih && cih->version == 4 && cih->ihl >= 5)) + if (!cih) + return NF_ACCEPT; /* The packet looks wrong, ignore */ + hlen_ipip = cih->ihl * 4; + if (!(cih->version == 4 && hlen_ipip >= sizeof(struct iphdr))) return NF_ACCEPT; /* The packet looks wrong, ignore */ raddr = (union nf_inet_addr *)&cih->daddr; - hlen_ipip = cih->ihl * 4; /* Special case for errors for IPIP/UDP/GRE tunnel packets */ tunnel = false; @@ -1823,9 +1825,6 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related, if (!dest || dest->tun_type != IP_VS_CONN_F_TUNNEL_TYPE_IPIP) return NF_ACCEPT; offset += hlen_ipip; - cih = skb_header_pointer(skb, offset, sizeof(_ciph), &_ciph); - if (!(cih && cih->version == 4 && cih->ihl >= 5)) - return NF_ACCEPT; /* The packet looks wrong, ignore */ tunnel = true; } else if ((cih->protocol == IPPROTO_UDP || /* Can be UDP encap */ cih->protocol == IPPROTO_GRE) && /* Can be GRE encap */ @@ -1850,21 +1849,25 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related, /* Skip IP and UDP/GRE tunnel headers */ offset = offset2 + ulen; /* Now we should be at the original IP header */ - cih = skb_header_pointer(skb, offset, sizeof(_ciph), - &_ciph); - if (cih && cih->version == 4 && cih->ihl >= 5 && - iproto == IPPROTO_IPIP) + if (iproto == IPPROTO_IPIP) tunnel = true; else return NF_ACCEPT; } } - pd = ip_vs_proto_data_get(ipvs, cih->protocol); + if (!ip_vs_fill_iph_skb_icmp(AF_INET, skb, offset, !tunnel, &ciph)) + return NF_ACCEPT; + pd = ip_vs_proto_data_get(ipvs, ciph.protocol); if (!pd) return NF_ACCEPT; pp = pd->pp; + cih = skb_header_pointer(skb, offset, sizeof(_ciph), &_ciph); + if (!(cih && cih->version == 4 && + ciph.len - ciph.off >= sizeof(struct iphdr))) + return NF_ACCEPT; /* The packet looks wrong, ignore */ + /* Is the embedded protocol header present? */ if (unlikely(cih->frag_off & htons(IP_OFFSET) && !pp->dont_defrag)) return NF_ACCEPT; @@ -1872,9 +1875,6 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related, IP_VS_DBG_PKT(11, AF_INET, pp, skb, offset, "Checking incoming ICMP for"); - offset2 = offset; - ip_vs_fill_iph_skb_icmp(AF_INET, skb, offset, !tunnel, &ciph); - /* The embedded headers contain source and dest in reverse order. * For IPIP/UDP/GRE tunnel this is error for request, not for reply. */ @@ -1904,11 +1904,12 @@ ip_vs_in_icmp(struct netns_ipvs *ipvs, struct sk_buff *skb, int *related, } if (tunnel) { - unsigned int hlen_orig = cih->ihl * 4; + unsigned int hlen_orig = ciph.len - ciph.off; __be32 info = ic->un.gateway; __u8 type = ic->type; __u8 code = ic->code; + offset2 = offset; /* Update the MTU */ if (ic->type == ICMP_DEST_UNREACH && ic->code == ICMP_FRAG_NEEDED) { diff --git a/net/netfilter/ipvs/ip_vs_est.c b/net/netfilter/ipvs/ip_vs_est.c index ab09f5182951..05a216a47b45 100644 --- a/net/netfilter/ipvs/ip_vs_est.c +++ b/net/netfilter/ipvs/ip_vs_est.c @@ -191,8 +191,11 @@ static int ip_vs_estimation_kthread(void *data) } /* kthread 0 will handle the calc phase */ - if (ipvs->est_calc_phase) + if (ipvs->est_calc_phase) { ip_vs_est_calc_phase(ipvs); + if (kthread_should_stop() || !READ_ONCE(ipvs->enable)) + return 0; + } } while (1) { @@ -270,6 +273,7 @@ int ip_vs_est_kthread_start(struct netns_ipvs *ipvs, kd->task = NULL; goto out; } + get_task_struct(kd->task); set_user_nice(kd->task, sysctl_est_nice(ipvs)); if (sysctl_est_preferred_cpulist(ipvs)) @@ -286,7 +290,7 @@ void ip_vs_est_kthread_stop(struct ip_vs_est_kt_data *kd) { if (kd->task) { pr_info("stopping estimator thread %d...\n", kd->id); - kthread_stop(kd->task); + kthread_stop_put(kd->task); kd->task = NULL; } } @@ -526,7 +530,7 @@ static void ip_vs_est_kthread_destroy(struct ip_vs_est_kt_data *kd) if (kd) { if (kd->task) { pr_info("stop unused estimator thread %d...\n", kd->id); - kthread_stop(kd->task); + kthread_stop_put(kd->task); } ip_vs_stats_free(kd->calc_stats); kfree(kd); diff --git a/net/netfilter/ipvs/ip_vs_proto_sctp.c b/net/netfilter/ipvs/ip_vs_proto_sctp.c index 3dbd3096e163..c80567c73469 100644 --- a/net/netfilter/ipvs/ip_vs_proto_sctp.c +++ b/net/netfilter/ipvs/ip_vs_proto_sctp.c @@ -193,7 +193,7 @@ sctp_csum_check(int af, struct sk_buff *skb, struct ip_vs_protocol *pp, struct sctphdr *sh; __le32 cmp, val; - if (!ip_vs_checksum_needed(skb, af)) + if (!ip_vs_checksum_needed(skb)) return 1; sh = (struct sctphdr *)(skb->data + sctphoff); cmp = sh->checksum; diff --git a/net/netfilter/ipvs/ip_vs_xmit.c b/net/netfilter/ipvs/ip_vs_xmit.c index 0b0c5304993a..c4508f3f43dd 100644 --- a/net/netfilter/ipvs/ip_vs_xmit.c +++ b/net/netfilter/ipvs/ip_vs_xmit.c @@ -1580,7 +1580,7 @@ ip_vs_icmp_xmit(struct sk_buff *skb, struct ip_vs_conn *cp, if (skb_cow(skb, rt->dst.dev->hard_header_len)) goto tx_error; - ip_vs_nat_icmp(skb, pp, cp, 0, toff, has_ports); + ip_vs_nat_icmp(skb, pp, cp, 0, toff, has_ports, ciph); /* Another hack: avoid icmp_send in ip_fragment */ skb->ignore_df = 1; diff --git a/net/netfilter/nf_conntrack_bpf.c b/net/netfilter/nf_conntrack_bpf.c index f98d1d4b42c3..c2df7c948281 100644 --- a/net/netfilter/nf_conntrack_bpf.c +++ b/net/netfilter/nf_conntrack_bpf.c @@ -122,42 +122,54 @@ __bpf_nf_ct_alloc_entry(struct net *net, struct bpf_sock_tuple *bpf_tuple, struct nf_conntrack_tuple otuple, rtuple; struct nf_conntrack_zone ct_zone; struct nf_conn *ct; + u8 ct_zone_dir = 0; + u16 ct_zone_id; + s32 netns_id; + u8 l4proto; int err; if (!(opts_len == NF_BPF_CT_OPTS_SZ || opts_len == 12)) return ERR_PTR(-EINVAL); + + netns_id = READ_ONCE(opts->netns_id); + l4proto = READ_ONCE(opts->l4proto); + ct_zone_id = READ_ONCE(opts->ct_zone_id); if (opts_len == NF_BPF_CT_OPTS_SZ) { - if (opts->reserved[0] || opts->reserved[1] || opts->reserved[2]) + ct_zone_dir = READ_ONCE(opts->ct_zone_dir); + if (READ_ONCE(opts->reserved[0]) || + READ_ONCE(opts->reserved[1]) || + READ_ONCE(opts->reserved[2])) return ERR_PTR(-EINVAL); } else { - if (opts->ct_zone_id) + if (ct_zone_id) return ERR_PTR(-EINVAL); } - if (unlikely(opts->netns_id < BPF_F_CURRENT_NETNS)) + if (unlikely(netns_id < BPF_F_CURRENT_NETNS)) return ERR_PTR(-EINVAL); - err = bpf_nf_ct_tuple_parse(bpf_tuple, tuple_len, opts->l4proto, + err = bpf_nf_ct_tuple_parse(bpf_tuple, tuple_len, l4proto, IP_CT_DIR_ORIGINAL, &otuple); if (err < 0) return ERR_PTR(err); - err = bpf_nf_ct_tuple_parse(bpf_tuple, tuple_len, opts->l4proto, + err = bpf_nf_ct_tuple_parse(bpf_tuple, tuple_len, l4proto, IP_CT_DIR_REPLY, &rtuple); if (err < 0) return ERR_PTR(err); - if (opts->netns_id >= 0) { - net = get_net_ns_by_id(net, opts->netns_id); + if (netns_id >= 0) { + net = get_net_ns_by_id(net, netns_id); if (unlikely(!net)) return ERR_PTR(-ENONET); } if (opts_len == NF_BPF_CT_OPTS_SZ) { - if (opts->ct_zone_dir == 0) - opts->ct_zone_dir = NF_CT_DEFAULT_ZONE_DIR; - nf_ct_zone_init(&ct_zone, - opts->ct_zone_id, opts->ct_zone_dir, 0); + if (ct_zone_dir == 0) { + ct_zone_dir = NF_CT_DEFAULT_ZONE_DIR; + opts->ct_zone_dir = ct_zone_dir; + } + nf_ct_zone_init(&ct_zone, ct_zone_id, ct_zone_dir, 0); } else { ct_zone = nf_ct_zone_dflt; } @@ -171,7 +183,7 @@ __bpf_nf_ct_alloc_entry(struct net *net, struct bpf_sock_tuple *bpf_tuple, __nf_ct_set_timeout(ct, timeout * HZ); out: - if (opts->netns_id >= 0) + if (netns_id >= 0) put_net(net); return ct; @@ -186,46 +198,58 @@ static struct nf_conn *__bpf_nf_ct_lookup(struct net *net, struct nf_conntrack_tuple tuple; struct nf_conntrack_zone ct_zone; struct nf_conn *ct; + u8 ct_zone_dir = 0; + u16 ct_zone_id; + s32 netns_id; + u8 l4proto; int err; if (!opts || !bpf_tuple) return ERR_PTR(-EINVAL); if (!(opts_len == NF_BPF_CT_OPTS_SZ || opts_len == 12)) return ERR_PTR(-EINVAL); + + netns_id = READ_ONCE(opts->netns_id); + l4proto = READ_ONCE(opts->l4proto); + ct_zone_id = READ_ONCE(opts->ct_zone_id); if (opts_len == NF_BPF_CT_OPTS_SZ) { - if (opts->reserved[0] || opts->reserved[1] || opts->reserved[2]) + ct_zone_dir = READ_ONCE(opts->ct_zone_dir); + if (READ_ONCE(opts->reserved[0]) || + READ_ONCE(opts->reserved[1]) || + READ_ONCE(opts->reserved[2])) return ERR_PTR(-EINVAL); } else { - if (opts->ct_zone_id) + if (ct_zone_id) return ERR_PTR(-EINVAL); } - if (unlikely(opts->l4proto != IPPROTO_TCP && opts->l4proto != IPPROTO_UDP)) + if (unlikely(l4proto != IPPROTO_TCP && l4proto != IPPROTO_UDP)) return ERR_PTR(-EPROTO); - if (unlikely(opts->netns_id < BPF_F_CURRENT_NETNS)) + if (unlikely(netns_id < BPF_F_CURRENT_NETNS)) return ERR_PTR(-EINVAL); - err = bpf_nf_ct_tuple_parse(bpf_tuple, tuple_len, opts->l4proto, + err = bpf_nf_ct_tuple_parse(bpf_tuple, tuple_len, l4proto, IP_CT_DIR_ORIGINAL, &tuple); if (err < 0) return ERR_PTR(err); - if (opts->netns_id >= 0) { - net = get_net_ns_by_id(net, opts->netns_id); + if (netns_id >= 0) { + net = get_net_ns_by_id(net, netns_id); if (unlikely(!net)) return ERR_PTR(-ENONET); } if (opts_len == NF_BPF_CT_OPTS_SZ) { - if (opts->ct_zone_dir == 0) - opts->ct_zone_dir = NF_CT_DEFAULT_ZONE_DIR; - nf_ct_zone_init(&ct_zone, - opts->ct_zone_id, opts->ct_zone_dir, 0); + if (ct_zone_dir == 0) { + ct_zone_dir = NF_CT_DEFAULT_ZONE_DIR; + opts->ct_zone_dir = ct_zone_dir; + } + nf_ct_zone_init(&ct_zone, ct_zone_id, ct_zone_dir, 0); } else { ct_zone = nf_ct_zone_dflt; } hash = nf_conntrack_find_get(net, &ct_zone, &tuple); - if (opts->netns_id >= 0) + if (netns_id >= 0) put_net(net); if (!hash) return ERR_PTR(-ENOENT); diff --git a/net/netfilter/nf_flow_table_ip.c b/net/netfilter/nf_flow_table_ip.c index 0b78decce8a9..c9e332fafcb5 100644 --- a/net/netfilter/nf_flow_table_ip.c +++ b/net/netfilter/nf_flow_table_ip.c @@ -310,6 +310,7 @@ static unsigned int nf_flow_xmit_xfrm(struct sk_buff *skb, struct dst_entry *dst) { skb_orphan(skb); + skb_dst_drop(skb); skb_dst_set_noref(skb, dst); dst_output(state->net, state->sk, skb); return NF_STOLEN; @@ -861,6 +862,7 @@ nf_flow_offload_ip_hook(void *priv, struct sk_buff *skb, return NF_DROP; } xmit.dest = neigh->ha; + skb_dst_drop(skb); skb_dst_set_noref(skb, &rt->dst); break; case FLOW_OFFLOAD_XMIT_DIRECT: @@ -1178,6 +1180,7 @@ nf_flow_offload_ipv6_hook(void *priv, struct sk_buff *skb, return NF_DROP; } xmit.dest = neigh->ha; + skb_dst_drop(skb); skb_dst_set_noref(skb, &rt->dst); break; case FLOW_OFFLOAD_XMIT_DIRECT: diff --git a/net/openvswitch/datapath.c b/net/openvswitch/datapath.c index eaf332b156d7..ae69b2cabab9 100644 --- a/net/openvswitch/datapath.c +++ b/net/openvswitch/datapath.c @@ -1113,9 +1113,8 @@ static int ovs_flow_cmd_new(struct sk_buff *skb, struct genl_info *info) error = -EEXIST; goto err_unlock_ovs; } - /* The flow identifier has to be the same for flow updates. - * Look for any overlapping flow. - */ + + /* Look for any overlapping flow. */ if (unlikely(!ovs_flow_cmp(flow, &match))) { if (ovs_identifier_is_key(&flow->id)) flow = ovs_flow_tbl_lookup_exact(&dp->table, @@ -1127,6 +1126,30 @@ static int ovs_flow_cmd_new(struct sk_buff *skb, struct genl_info *info) goto err_unlock_ovs; } } + + if (unlikely(reply)) { + size_t cur, req; + + cur = ovs_flow_cmd_msg_size(acts, &new_flow->id, + ufid_flags); + req = ovs_flow_cmd_msg_size(acts, &flow->id, + ufid_flags); + if (cur < req) { + struct sk_buff *resized; + + resized = ovs_flow_cmd_alloc_info(acts, + &flow->id, + info, false, + ufid_flags); + if (IS_ERR(resized)) { + error = PTR_ERR(resized); + goto err_unlock_ovs; + } + kfree_skb(reply); + reply = resized; + } + } + /* Update actions. */ old_acts = ovsl_dereference(flow->sf_acts); rcu_assign_pointer(flow->sf_acts, acts); diff --git a/net/openvswitch/flow.c b/net/openvswitch/flow.c index 66366982f604..46c1d66aad8c 100644 --- a/net/openvswitch/flow.c +++ b/net/openvswitch/flow.c @@ -889,8 +889,6 @@ static int key_extract_l3l4(struct sk_buff *skb, struct sw_flow_key *key) * Ethernet header * @key: output flow key * - * The caller must ensure that skb->len >= ETH_HLEN. - * * Initializes @skb header fields as follows: * * - skb->mac_header: the L2 header. @@ -910,8 +908,6 @@ static int key_extract_l3l4(struct sk_buff *skb, struct sw_flow_key *key) */ static int key_extract(struct sk_buff *skb, struct sw_flow_key *key) { - struct ethhdr *eth; - /* Flags are always used as part of stats */ key->tp.flags = 0; @@ -926,6 +922,13 @@ static int key_extract(struct sk_buff *skb, struct sw_flow_key *key) skb_reset_network_header(skb); key->eth.type = skb->protocol; } else { + struct ethhdr *eth; + int err; + + err = check_header(skb, ETH_HLEN); + if (unlikely(err)) + return err; + eth = eth_hdr(skb); ether_addr_copy(key->eth.src, eth->h_source); ether_addr_copy(key->eth.dst, eth->h_dest); diff --git a/net/packet/af_packet.c b/net/packet/af_packet.c index e75d2932475a..435756877aba 100644 --- a/net/packet/af_packet.c +++ b/net/packet/af_packet.c @@ -1315,13 +1315,25 @@ static int packet_rcv_has_room(struct packet_sock *po, struct sk_buff *skb) return ret; } -static void packet_rcv_try_clear_pressure(struct packet_sock *po) +static void __packet_rcv_try_clear_pressure(struct packet_sock *po) { if (packet_sock_flag(po, PACKET_SOCK_PRESSURE) && __packet_rcv_has_room(po, NULL) == ROOM_NORMAL) packet_sock_flag_set(po, PACKET_SOCK_PRESSURE, false); } +static void packet_rcv_try_clear_pressure(struct packet_sock *po) +{ + struct sock *sk = &po->sk; + + if (!packet_sock_flag(po, PACKET_SOCK_PRESSURE)) + return; + + spin_lock_bh(&sk->sk_receive_queue.lock); + __packet_rcv_try_clear_pressure(po); + spin_unlock_bh(&sk->sk_receive_queue.lock); +} + static void packet_sock_destruct(struct sock *sk) { skb_queue_purge(&sk->sk_error_queue); @@ -1924,11 +1936,12 @@ static void packet_parse_headers(struct sk_buff *skb, struct socket *sock) { int depth; + /* On TX skb->data is the L2 header; anchor it for all socket types. */ + skb_reset_mac_header(skb); + if ((!skb->protocol || skb->protocol == htons(ETH_P_ALL)) && - sock->type == SOCK_RAW) { - skb_reset_mac_header(skb); + sock->type == SOCK_RAW) skb->protocol = dev_parse_header_protocol(skb); - } /* Move network header to the right position for VLAN tagged packets */ if (likely(skb->dev->type == ARPHRD_ETHER) && @@ -1953,8 +1966,9 @@ static int packet_sendmsg_spkt(struct socket *sock, struct msghdr *msg, struct net_device *dev; struct sockcm_cookie sockc; __be16 proto = 0; - int err; + int hard_header_len; int extra_len = 0; + int err; /* * Get and verify the address. @@ -1997,14 +2011,18 @@ static int packet_sendmsg_spkt(struct socket *sock, struct msghdr *msg, extra_len = 4; /* We're doing our own CRC */ } + /* Keep the allocation-time header length across retry. */ + if (!skb) + hard_header_len = READ_ONCE(dev->hard_header_len); + err = -EMSGSIZE; - if (len > dev->mtu + dev->hard_header_len + VLAN_HLEN + extra_len) + if (len > dev->mtu + hard_header_len + VLAN_HLEN + extra_len) goto out_unlock; if (!skb) { - size_t reserved = LL_RESERVED_SPACE(dev); + size_t reserved = LL_RESERVED_SPACE_EX(dev, hard_header_len); int tlen = dev->needed_tailroom; - unsigned int hhlen = dev->header_ops ? dev->hard_header_len : 0; + unsigned int hhlen = dev->header_ops ? hard_header_len : 0; rcu_read_unlock(); skb = sock_wmalloc(sk, len + reserved + tlen, 0, GFP_KERNEL); @@ -2034,7 +2052,7 @@ static int packet_sendmsg_spkt(struct socket *sock, struct msghdr *msg, err = -EINVAL; goto out_unlock; } - if (len > (dev->mtu + dev->hard_header_len + extra_len) && + if (len > (dev->mtu + hard_header_len + extra_len) && !packet_extra_vlan_len_allowed(dev, skb)) { err = -EMSGSIZE; goto out_unlock; @@ -2569,6 +2587,7 @@ static int packet_snd_vnet_parse(struct msghdr *msg, size_t *len, static int tpacket_fill_skb(struct packet_sock *po, struct sk_buff *skb, void *frame, struct net_device *dev, void *data, int tp_len, __be16 proto, unsigned char *addr, int hlen, int copylen, + int hard_header_len, const struct sockcm_cookie *sockc) { union tpacket_uhdr ph; @@ -2600,8 +2619,8 @@ static int tpacket_fill_skb(struct packet_sock *po, struct sk_buff *skb, } else if (copylen) { int hdrlen = min_t(int, copylen, tp_len); - skb_push(skb, dev->hard_header_len); - skb_put(skb, copylen - dev->hard_header_len); + skb_push(skb, hard_header_len); + skb_put(skb, copylen - hard_header_len); err = skb_store_bits(skb, 0, data, hdrlen); if (unlikely(err)) return err; @@ -2732,7 +2751,7 @@ static int tpacket_snd(struct packet_sock *po, struct msghdr *msg) void *data; int len_sum = 0; int status = TP_STATUS_AVAILABLE; - int hlen, tlen, copylen = 0; + int hard_header_len, hlen, tlen, copylen = 0; long timeo; mutex_lock(&po->pg_vec_lock); @@ -2779,8 +2798,9 @@ static int tpacket_snd(struct packet_sock *po, struct msghdr *msg) goto out_put; } + hard_header_len = READ_ONCE(dev->hard_header_len); if (po->sk.sk_socket->type == SOCK_RAW) - reserve = dev->hard_header_len; + reserve = hard_header_len; size_max = po->tx_ring.frame_size - (po->tp_hdrlen - sizeof(struct sockaddr_ll)); @@ -2817,7 +2837,7 @@ static int tpacket_snd(struct packet_sock *po, struct msghdr *msg) goto tpacket_error; status = TP_STATUS_SEND_REQUEST; - hlen = LL_RESERVED_SPACE(dev); + hlen = LL_RESERVED_SPACE_EX(dev, hard_header_len); tlen = dev->needed_tailroom; if (vnet_hdr_sz) { data += vnet_hdr_sz; @@ -2835,10 +2855,10 @@ static int tpacket_snd(struct packet_sock *po, struct msghdr *msg) vnet_hdr.hdr_len); has_vnet_hdr = true; } - copylen = max_t(int, copylen, dev->hard_header_len); + copylen = max_t(int, copylen, hard_header_len); skb = sock_alloc_send_skb(&po->sk, hlen + tlen + sizeof(struct sockaddr_ll) + - (copylen - dev->hard_header_len), + (copylen - hard_header_len), !need_wait, &err); if (unlikely(skb == NULL)) { @@ -2848,7 +2868,8 @@ static int tpacket_snd(struct packet_sock *po, struct msghdr *msg) goto out_status; } tp_len = tpacket_fill_skb(po, skb, ph, dev, data, tp_len, proto, - addr, hlen, copylen, &sockc); + addr, hlen, copylen, hard_header_len, + &sockc); if (likely(tp_len >= 0) && tp_len > dev->mtu + reserve && !vnet_hdr_sz && @@ -2956,7 +2977,7 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len) int offset = 0; struct packet_sock *po = pkt_sk(sk); int vnet_hdr_sz = READ_ONCE(po->vnet_hdr_sz); - int hlen, tlen, linear; + int hard_header_len, hlen, tlen, linear; int extra_len = 0; /* @@ -2996,8 +3017,9 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len) goto out_unlock; } + hard_header_len = READ_ONCE(dev->hard_header_len); if (sock->type == SOCK_RAW) - reserve = dev->hard_header_len; + reserve = hard_header_len; if (vnet_hdr_sz) { err = packet_snd_vnet_parse(msg, &len, &vnet_hdr, vnet_hdr_sz); if (err) @@ -3018,10 +3040,10 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len) goto out_unlock; err = -ENOBUFS; - hlen = LL_RESERVED_SPACE(dev); + hlen = LL_RESERVED_SPACE_EX(dev, hard_header_len); tlen = dev->needed_tailroom; linear = __virtio16_to_cpu(vio_le(), vnet_hdr.hdr_len); - linear = max(linear, min_t(int, len, dev->hard_header_len)); + linear = max(linear, min_t(int, len, hard_header_len)); skb = packet_alloc_skb(sk, hlen + tlen, hlen, len, linear, msg->msg_flags & MSG_DONTWAIT, &err); if (skb == NULL) @@ -3037,7 +3059,7 @@ static int packet_snd(struct socket *sock, struct msghdr *msg, size_t len) } else if (reserve) { skb_reserve(skb, -reserve); if (len < reserve + sizeof(struct ipv6hdr) && - dev->min_header_len != dev->hard_header_len) + dev->min_header_len != hard_header_len) skb_reset_network_header(skb); } @@ -4304,7 +4326,7 @@ static __poll_t packet_poll(struct file *file, struct socket *sock, TP_STATUS_KERNEL)) mask |= EPOLLIN | EPOLLRDNORM; } - packet_rcv_try_clear_pressure(po); + __packet_rcv_try_clear_pressure(po); spin_unlock_bh(&sk->sk_receive_queue.lock); spin_lock_bh(&sk->sk_write_queue.lock); if (po->tx_ring.pg_vec) { @@ -4544,14 +4566,14 @@ static int packet_set_ring(struct sock *sk, union tpacket_req_u *req_u, rb->frame_max = (req->tp_frame_nr - 1); rb->head = 0; rb->frame_size = req->tp_frame_size; + po->prot_hook.func = (po->rx_ring.pg_vec) ? + tpacket_rcv : packet_rcv; spin_unlock_bh(&rb_queue->lock); swap(rb->pg_vec_order, order); swap(rb->pg_vec_len, req->tp_block_nr); rb->pg_vec_pages = req->tp_block_size/PAGE_SIZE; - po->prot_hook.func = (po->rx_ring.pg_vec) ? - tpacket_rcv : packet_rcv; skb_queue_purge(rb_queue); if (atomic_long_read(&po->mapped)) pr_err("packet_mmap: vma is busy: %ld\n", diff --git a/net/qrtr/ns.c b/net/qrtr/ns.c index e5b2adb161d9..c5e7e01db249 100644 --- a/net/qrtr/ns.c +++ b/net/qrtr/ns.c @@ -78,7 +78,7 @@ struct qrtr_node { */ #define QRTR_NS_MAX_NODES 512 #define QRTR_NS_MAX_SERVERS 256 -#define QRTR_NS_MAX_LOOKUPS 64 +#define QRTR_NS_MAX_LOOKUPS 128 static u16 node_count; diff --git a/net/sched/cls_api.c b/net/sched/cls_api.c index fee4524adc98..4e6a2812a4f3 100644 --- a/net/sched/cls_api.c +++ b/net/sched/cls_api.c @@ -443,7 +443,22 @@ static void tcf_chain_put(struct tcf_chain *chain); static void tcf_proto_destroy(struct tcf_proto *tp, bool rtnl_held, bool sig_destroy, struct netlink_ext_ack *extack) { - tp->ops->destroy(tp, rtnl_held, extack); + /* A locked classifier's destroy callback (e.g. u32_destroy) uses + * rtnl_dereference() and mutates shared structures (e.g. the + * tc_u_common hash list) that are only safe under rtnl_lock. When an + * unlocked classifier's request (e.g. flower on ingress) loses the + * tcf_chain_tp_insert_unique() race and ends up dropping the last + * reference on a locked classifier's proto, destroy() would run + * without rtnl held. Take it here in that case. + */ + bool not_lockless = !rtnl_held && + !(tp->ops->flags & TCF_PROTO_OPS_DOIT_UNLOCKED); + + if (not_lockless) + rtnl_lock(); + tp->ops->destroy(tp, rtnl_held || not_lockless, extack); + if (not_lockless) + rtnl_unlock(); tcf_proto_count_usesw(tp, false); if (sig_destroy) tcf_proto_signal_destroyed(tp->chain, tp); diff --git a/net/sched/cls_route.c b/net/sched/cls_route.c index bd6f945bd388..eded7aacd3f7 100644 --- a/net/sched/cls_route.c +++ b/net/sched/cls_route.c @@ -52,6 +52,7 @@ struct route4_filter { struct tcf_result res; struct tcf_exts exts; u32 handle; + bool dying; struct route4_bucket *bkt; struct tcf_proto *tp; struct rcu_work rwork; @@ -66,9 +67,11 @@ static inline int route4_fastmap_hash(u32 id, int iif) static DEFINE_SPINLOCK(fastmap_lock); static void -route4_reset_fastmap(struct route4_head *head) +route4_reset_fastmap(struct route4_head *head, struct route4_filter *f) { spin_lock_bh(&fastmap_lock); + if (f) + f->dying = true; memset(head->fastmap, 0, sizeof(head->fastmap)); spin_unlock_bh(&fastmap_lock); } @@ -81,9 +84,11 @@ route4_set_fastmap(struct route4_head *head, u32 id, int iif, /* fastmap updates must look atomic to aling id, iff, filter */ spin_lock_bh(&fastmap_lock); - head->fastmap[h].id = id; - head->fastmap[h].iif = iif; - head->fastmap[h].filter = f; + if (f == ROUTE4_FAILURE || !f->dying) { + head->fastmap[h].id = id; + head->fastmap[h].iif = iif; + head->fastmap[h].filter = f; + } spin_unlock_bh(&fastmap_lock); } @@ -297,6 +302,13 @@ static void route4_destroy(struct tcf_proto *tp, bool rtnl_held, next = rtnl_dereference(f->next); RCU_INIT_POINTER(b->ht[h2], next); tcf_unbind_filter(tp, &f->res); + /* Mark the filter dying under fastmap_lock so + * any in-flight reader that still holds it + * will skip the republish in route4_set_fastmap(). + */ + spin_lock_bh(&fastmap_lock); + f->dying = true; + spin_unlock_bh(&fastmap_lock); if (tcf_exts_get_net(&f->exts)) route4_queue_work(f); else @@ -307,6 +319,11 @@ static void route4_destroy(struct tcf_proto *tp, bool rtnl_held, kfree_rcu(b, rcu); } } + + /* All filters are unlinked and marked dying, so no in-flight + * reader can republish a stale entry after this reset. + */ + route4_reset_fastmap(head, NULL); kfree_rcu(head, rcu); } @@ -334,11 +351,11 @@ static int route4_delete(struct tcf_proto *tp, void *arg, bool *last, /* unlink it */ RCU_INIT_POINTER(*fp, rtnl_dereference(f->next)); - /* Remove any fastmap lookups that might ref filter - * notice we unlink'd the filter so we can't get it - * back in the fastmap. + /* Clear any fastmap entries that may ref this filter and + * mark it dying so in-flight readers can't republish it + * after the reset. */ - route4_reset_fastmap(head); + route4_reset_fastmap(head, f); /* Delete it */ tcf_unbind_filter(tp, &f->res); @@ -558,7 +575,7 @@ static int route4_change(struct net *net, struct sk_buff *in_skb, } } - route4_reset_fastmap(head); + route4_reset_fastmap(head, fold); *arg = f; if (fold) { tcf_unbind_filter(tp, &fold->res); diff --git a/net/sched/sch_api.c b/net/sched/sch_api.c index 668bcd60d183..65b35528d125 100644 --- a/net/sched/sch_api.c +++ b/net/sched/sch_api.c @@ -1114,6 +1114,9 @@ static int qdisc_graft(struct net_device *dev, struct Qdisc *parent, unsigned int i, num_q, ingress; struct netdev_queue *dev_queue; + if (new) + new->depth = 0; + ingress = 0; num_q = dev->num_tx_queues; if ((q && q->flags & TCQ_F_INGRESS) || @@ -1211,9 +1214,15 @@ static int qdisc_graft(struct net_device *dev, struct Qdisc *parent, NL_SET_ERR_MSG(extack, "STAB not supported on a non root"); return -EINVAL; } + if (new && parent->depth >= 7) { + NL_SET_ERR_MSG(extack, "Qdisc hierarchy is too deep"); + return -E2BIG; + } err = cops->graft(parent, cl, new, &old, extack); if (err) return err; + if (new) + new->depth = parent->depth + 1; notify_and_destroy(net, skb, n, classid, old, new, extack); } return 0; diff --git a/net/sched/sch_cake.c b/net/sched/sch_cake.c index f64be54ead49..f25f60978631 100644 --- a/net/sched/sch_cake.c +++ b/net/sched/sch_cake.c @@ -1287,7 +1287,6 @@ static struct sk_buff *cake_ack_filter(struct cake_sched_data *q, seglen = ipv6_payload_len(skb, ipv6h_check); } else { - WARN_ON(1); /* shouldn't happen */ continue; } diff --git a/net/sctp/associola.c b/net/sctp/associola.c index b6ac0966420a..5b0ae616e1ff 100644 --- a/net/sctp/associola.c +++ b/net/sctp/associola.c @@ -573,6 +573,10 @@ void sctp_assoc_rm_peer(struct sctp_association *asoc, if (ch->transport == peer) ch->transport = NULL; + list_for_each_entry(ch, &asoc->outqueue.control_chunk_list, list) + if (ch->transport == peer) + ch->transport = NULL; + asoc->peer.transport_count--; sctp_ulpevent_notify_peer_addr_change(peer, SCTP_ADDR_REMOVED, 0); diff --git a/net/sctp/outqueue.c b/net/sctp/outqueue.c index f6b8c13dafa4..e92fb9da4647 100644 --- a/net/sctp/outqueue.c +++ b/net/sctp/outqueue.c @@ -650,6 +650,7 @@ static int __sctp_outq_flush_rtx(struct sctp_outq *q, struct sctp_packet *pkt, if (chunk->tsn_gap_acked) { list_move_tail(&chunk->transmitted_list, &transport->transmitted); + chunk->transport = transport; continue; } diff --git a/net/sctp/sm_make_chunk.c b/net/sctp/sm_make_chunk.c index 0ae30c3c8913..e25612e9d082 100644 --- a/net/sctp/sm_make_chunk.c +++ b/net/sctp/sm_make_chunk.c @@ -3336,12 +3336,11 @@ struct sctp_chunk *sctp_process_asconf(struct sctp_association *asoc, goto done; } done: - asoc->peer.addip_serial++; - /* If we are sending a new ASCONF_ACK hold a reference to it in assoc * after freeing the reference to old asconf ack if any. */ if (asconf_ack) { + asoc->peer.addip_serial++; sctp_chunk_hold(asconf_ack); list_add_tail(&asconf_ack->transmitted_list, &asoc->asconf_ack_list); diff --git a/net/smc/af_smc.c b/net/smc/af_smc.c index b5db69073e20..00403175b740 100644 --- a/net/smc/af_smc.c +++ b/net/smc/af_smc.c @@ -1931,11 +1931,12 @@ static void smc_listen_out(struct smc_sock *new_smc) atomic_dec(&lsmc->queued_smc_hs); release_sock(newsmcsk); /* lock in smc_listen_work() */ + lock_sock_nested(&lsmc->sk, SINGLE_DEPTH_NESTING); if (lsmc->sk.sk_state == SMC_LISTEN) { - lock_sock_nested(&lsmc->sk, SINGLE_DEPTH_NESTING); smc_accept_enqueue(&lsmc->sk, newsmcsk); release_sock(&lsmc->sk); } else { /* no longer listening */ + release_sock(&lsmc->sk); smc_close_non_accepted(newsmcsk); } diff --git a/net/smc/smc_llc.c b/net/smc/smc_llc.c index 954b2ff1815c..aa6d83af55ed 100644 --- a/net/smc/smc_llc.c +++ b/net/smc/smc_llc.c @@ -1927,7 +1927,8 @@ static void smc_llc_event_handler(struct smc_llc_qentry *qentry) return; case SMC_LLC_CONFIRM_LINK: case SMC_LLC_ADD_LINK_CONT: - if (lgr->llc_flow_lcl.type != SMC_LLC_FLOW_NONE) { + if (lgr->llc_flow_lcl.type != SMC_LLC_FLOW_NONE && + !lgr->llc_flow_lcl.qentry) { /* a flow is waiting for this message */ smc_llc_flow_qentry_set(&lgr->llc_flow_lcl, qentry); wake_up(&lgr->llc_msg_waiter); diff --git a/net/smc/smc_rx.c b/net/smc/smc_rx.c index c1d9b923938d..5c9e4d8b57de 100644 --- a/net/smc/smc_rx.c +++ b/net/smc/smc_rx.c @@ -150,7 +150,12 @@ static const struct pipe_buf_operations smc_pipe_ops = { static void smc_rx_spd_release(struct splice_pipe_desc *spd, unsigned int i) { + struct smc_spd_priv *priv = (struct smc_spd_priv *)spd->partial[i].private; + struct sock *sk = &priv->smc->sk; + + kfree(priv); put_page(spd->pages[i]); + sock_put(sk); } static int smc_rx_splice(struct pipe_inode_info *pipe, char *src, size_t len, @@ -209,6 +214,10 @@ static int smc_rx_splice(struct pipe_inode_info *pipe, char *src, size_t len, offset = 0; } } + for (i = 0; i < nr_pages; i++) { + get_page(pages[i]); + sock_hold(&smc->sk); + } spd.nr_pages_max = nr_pages; spd.nr_pages = nr_pages; spd.pages = pages; @@ -217,16 +226,8 @@ static int smc_rx_splice(struct pipe_inode_info *pipe, char *src, size_t len, spd.spd_release = smc_rx_spd_release; bytes = splice_to_pipe(pipe, &spd); - if (bytes > 0) { - sock_hold(&smc->sk); - if (!lgr->is_smcd && smc->conn.rmb_desc->is_vm) { - for (i = 0; i < PAGE_ALIGN(bytes + offset) / PAGE_SIZE; i++) - get_page(pages[i]); - } else { - get_page(smc->conn.rmb_desc->pages); - } + if (bytes > 0) atomic_add(bytes, &smc->conn.splice_pending); - } kfree(priv); kfree(partial); kfree(pages); diff --git a/net/tls/tls_sw.c b/net/tls/tls_sw.c index d4afc90fd796..62d46736e24b 100644 --- a/net/tls/tls_sw.c +++ b/net/tls/tls_sw.c @@ -458,7 +458,7 @@ int tls_tx_records(struct sock *sk, int flags) } tx_err: - if (rc < 0 && rc != -EAGAIN) + if (rc < 0 && rc != -EAGAIN && rc != -EINTR && rc != -ERESTARTSYS) tls_err_abort(sk, rc); return rc; @@ -832,6 +832,14 @@ static int tls_sw_sendmsg_locked(struct sock *sk, struct msghdr *msg, if (!sk_stream_memory_free(sk)) goto wait_for_sndbuf; + /* open record may be full if we couldn't push it in the last sendmsg call */ + if (sk_msg_full(msg_pl)) { + full_record = true; + sk_msg_trim(sk, msg_en, + msg_pl->sg.size + prot->overhead_size); + goto copied; + } + alloc_encrypted: ret = tls_alloc_encrypted_msg(sk, required_size); if (ret) { @@ -921,6 +929,12 @@ static int tls_sw_sendmsg_locked(struct sock *sk, struct msghdr *msg, msg_pl, try_to_copy); if (ret < 0) goto trim_sgl; + + if (sk_msg_full(msg_pl)) { + full_record = true; + sk_msg_trim(sk, msg_en, + msg_pl->sg.size + prot->overhead_size); + } } /* Open records defined only if successfully copied, otherwise @@ -1442,6 +1456,8 @@ tls_decrypt_sw(struct sock *sk, struct tls_context *tls_ctx, /* If opportunistic TLS 1.3 ZC failed retry without ZC */ if (unlikely(darg->zc && prot->version == TLS_1_3_VERSION && darg->tail != TLS_RECORD_TYPE_DATA)) { + iov_iter_revert(&msg->msg_iter, strp_msg(darg->skb)->full_len - + prot->overhead_size); darg->zc = false; if (!darg->tail) TLS_INC_STATS(sock_net(sk), LINUX_MIB_TLSRXNOPADVIOL); diff --git a/net/vmw_vsock/virtio_transport.c b/net/vmw_vsock/virtio_transport.c index 57f2d6ec3ffc..96c9fe8d357c 100644 --- a/net/vmw_vsock/virtio_transport.c +++ b/net/vmw_vsock/virtio_transport.c @@ -346,12 +346,13 @@ static void virtio_transport_tx_work(struct work_struct *work) struct virtqueue *vq; bool added = false; - vq = vsock->vqs[VSOCK_VQ_TX]; mutex_lock(&vsock->tx_lock); if (!vsock->tx_run) goto out; + vq = vsock->vqs[VSOCK_VQ_TX]; + do { struct sk_buff *skb; unsigned int len; @@ -451,13 +452,13 @@ static void virtio_transport_event_work(struct work_struct *work) container_of(work, struct virtio_vsock, event_work); struct virtqueue *vq; - vq = vsock->vqs[VSOCK_VQ_EVENT]; - mutex_lock(&vsock->event_lock); if (!vsock->event_run) goto out; + vq = vsock->vqs[VSOCK_VQ_EVENT]; + do { struct virtio_vsock_event *event; unsigned int len; @@ -634,12 +635,12 @@ static void virtio_transport_rx_work(struct work_struct *work) container_of(work, struct virtio_vsock, rx_work); struct virtqueue *vq; - vq = vsock->vqs[VSOCK_VQ_RX]; - mutex_lock(&vsock->rx_lock); if (!vsock->rx_run) - goto out; + goto out_nofill; + + vq = vsock->vqs[VSOCK_VQ_RX]; do { virtqueue_disable_cb(vq); @@ -691,6 +692,7 @@ static void virtio_transport_rx_work(struct work_struct *work) out: if (vsock->rx_buf_nr < vsock->rx_buf_max_nr / 2) virtio_vsock_rx_fill(vsock); +out_nofill: mutex_unlock(&vsock->rx_lock); } diff --git a/net/x25/af_x25.c b/net/x25/af_x25.c index 8aae9273b7c1..033e7d059f58 100644 --- a/net/x25/af_x25.c +++ b/net/x25/af_x25.c @@ -363,6 +363,7 @@ static void x25_destroy_timer(struct timer_list *t) struct sock *sk = timer_container_of(sk, t, sk_timer); x25_destroy_socket_from_timer(sk); + sock_put(sk); } /* @@ -398,9 +399,8 @@ static void __x25_destroy_socket(struct sock *sk) if (sk_has_allocations(sk)) { /* Defer: outstanding buffers */ - sk->sk_timer.expires = jiffies + 10 * HZ; sk->sk_timer.function = x25_destroy_timer; - add_timer(&sk->sk_timer); + sk_reset_timer(sk, &sk->sk_timer, jiffies + 10 * HZ); } else { /* drop last reference so sock_put will free */ __sock_put(sk); diff --git a/net/x25/x25_timer.c b/net/x25/x25_timer.c index 2ec63a1f4c6d..7896cd43f1cc 100644 --- a/net/x25/x25_timer.c +++ b/net/x25/x25_timer.c @@ -36,45 +36,45 @@ void x25_init_timers(struct sock *sk) void x25_start_heartbeat(struct sock *sk) { - mod_timer(&sk->sk_timer, jiffies + 5 * HZ); + sk_reset_timer(sk, &sk->sk_timer, jiffies + 5 * HZ); } void x25_stop_heartbeat(struct sock *sk) { - timer_delete(&sk->sk_timer); + sk_stop_timer(sk, &sk->sk_timer); } void x25_start_t2timer(struct sock *sk) { struct x25_sock *x25 = x25_sk(sk); - mod_timer(&x25->timer, jiffies + x25->t2); + sk_reset_timer(sk, &x25->timer, jiffies + x25->t2); } void x25_start_t21timer(struct sock *sk) { struct x25_sock *x25 = x25_sk(sk); - mod_timer(&x25->timer, jiffies + x25->t21); + sk_reset_timer(sk, &x25->timer, jiffies + x25->t21); } void x25_start_t22timer(struct sock *sk) { struct x25_sock *x25 = x25_sk(sk); - mod_timer(&x25->timer, jiffies + x25->t22); + sk_reset_timer(sk, &x25->timer, jiffies + x25->t22); } void x25_start_t23timer(struct sock *sk) { struct x25_sock *x25 = x25_sk(sk); - mod_timer(&x25->timer, jiffies + x25->t23); + sk_reset_timer(sk, &x25->timer, jiffies + x25->t23); } void x25_stop_timer(struct sock *sk) { - timer_delete(&x25_sk(sk)->timer); + sk_stop_timer(sk, &x25_sk(sk)->timer); } unsigned long x25_display_timer(struct sock *sk) @@ -108,7 +108,7 @@ static void x25_heartbeat_expiry(struct timer_list *t) sock_flag(sk, SOCK_DEAD))) { bh_unlock_sock(sk); x25_destroy_socket_from_timer(sk); - return; + goto out; } break; @@ -120,8 +120,14 @@ static void x25_heartbeat_expiry(struct timer_list *t) break; } restart_heartbeat: - x25_start_heartbeat(sk); + /* Do not rearm once __x25_destroy_socket() has unlinked the socket: + * it is past its cancel point and owns the teardown from there on. + */ + if (sk_hashed(sk)) + x25_start_heartbeat(sk); bh_unlock_sock(sk); +out: + sock_put(sk); } /* @@ -166,4 +172,5 @@ static void x25_timer_expiry(struct timer_list *t) } else x25_do_timer_expiry(sk); bh_unlock_sock(sk); + sock_put(sk); } diff --git a/net/xdp/xdp_umem.c b/net/xdp/xdp_umem.c index 58da2f4f4397..cd8643360eb3 100644 --- a/net/xdp/xdp_umem.c +++ b/net/xdp/xdp_umem.c @@ -210,6 +210,8 @@ static int xdp_umem_reg(struct xdp_umem *umem, struct xdp_umem_reg *mr) if (mr->flags & XDP_UMEM_TX_METADATA_LEN) { if (mr->tx_metadata_len >= 256 || mr->tx_metadata_len % 8) return -EINVAL; + if (mr->tx_metadata_len < 16) + return -EINVAL; umem->tx_metadata_len = mr->tx_metadata_len; } diff --git a/net/xdp/xsk.c b/net/xdp/xsk.c index f906d51b6699..7855ee09c4b6 100644 --- a/net/xdp/xsk.c +++ b/net/xdp/xsk.c @@ -967,15 +967,16 @@ static int xsk_skb_metadata(struct sk_buff *skb, void *buffer, { struct xsk_tx_metadata *meta = NULL; u16 csum_start, csum_offset; + u64 flags; if (unlikely(pool->tx_metadata_len == 0)) return -EINVAL; meta = buffer - pool->tx_metadata_len; - if (unlikely(!xsk_buff_valid_tx_metadata(meta))) + if (unlikely(!xsk_buff_valid_tx_metadata(pool, meta, &flags))) return -EINVAL; - if (meta->flags & XDP_TXMD_FLAGS_CHECKSUM) { + if (flags & XDP_TXMD_FLAGS_CHECKSUM) { csum_start = READ_ONCE(meta->request.csum_start); csum_offset = READ_ONCE(meta->request.csum_offset); @@ -996,8 +997,10 @@ static int xsk_skb_metadata(struct sk_buff *skb, void *buffer, } } - if (meta->flags & XDP_TXMD_FLAGS_LAUNCH_TIME) - skb->skb_mstamp_ns = meta->request.launch_time; + if (flags & XDP_TXMD_FLAGS_LAUNCH_TIME) + skb->skb_mstamp_ns = READ_ONCE(meta->request.launch_time); + if (!(flags & XDP_TXMD_FLAGS_TIMESTAMP)) + meta = NULL; xsk_tx_metadata_to_compl(meta, &skb_shinfo(skb)->xsk_meta); return 0; diff --git a/net/xdp/xsk_buff_pool.c b/net/xdp/xsk_buff_pool.c index a4089480b22b..78c14f106395 100644 --- a/net/xdp/xsk_buff_pool.c +++ b/net/xdp/xsk_buff_pool.c @@ -765,11 +765,11 @@ EXPORT_SYMBOL(xp_raw_get_dma); * @addr: desc address (from userspace) * * Helper for getting desc's DMA address and metadata pointer, if present. - * Saves one call on hotpath, double calculation of the actual address, - * and inline checks for metadata presence and sanity. + * Saves one call on hotpath and double calculation of the actual address. + * Metadata is validated later by xsk_tx_metadata_request(). * * Return: new &xdp_desc_ctx struct containing desc's DMA address and metadata - * pointer, if it is present and valid (initialized to %NULL otherwise). + * pointer, if it is present (initialized to %NULL otherwise). */ struct xdp_desc_ctx xp_raw_get_ctx(const struct xsk_buff_pool *pool, u64 addr) { diff --git a/rust/helpers/io.c b/rust/helpers/io.c index 397810864a24..1edbc274951c 100644 --- a/rust/helpers/io.c +++ b/rust/helpers/io.c @@ -3,6 +3,7 @@ #include #include +#ifdef CONFIG_HAS_IOMEM __rust_helper void __iomem *rust_helper_ioremap(phys_addr_t offset, size_t size) { return ioremap(offset, size); @@ -18,6 +19,7 @@ __rust_helper void rust_helper_iounmap(void __iomem *addr) { iounmap(addr); } +#endif /* CONFIG_HAS_IOMEM */ __rust_helper u8 rust_helper_readb(const void __iomem *addr) { diff --git a/rust/kernel/devres.rs b/rust/kernel/devres.rs index 11ce500e9b76..8ff8aedf251a 100644 --- a/rust/kernel/devres.rs +++ b/rust/kernel/devres.rs @@ -59,6 +59,7 @@ struct Inner { /// # Examples /// /// ```no_run +/// # #![cfg(CONFIG_HAS_IOMEM)] /// use kernel::{ /// bindings, /// device::{ diff --git a/rust/kernel/io.rs b/rust/kernel/io.rs index fcc7678fd9e3..d4063ee41200 100644 --- a/rust/kernel/io.rs +++ b/rust/kernel/io.rs @@ -9,6 +9,7 @@ prelude::*, // }; +#[cfg(CONFIG_HAS_IOMEM)] pub mod mem; pub mod poll; pub mod register; @@ -80,6 +81,7 @@ pub fn maxsize(&self) -> usize { /// # Examples /// /// ```no_run +/// # #![cfg(CONFIG_HAS_IOMEM)] /// use kernel::{ /// bindings, /// ffi::c_void, diff --git a/rust/kernel/platform.rs b/rust/kernel/platform.rs index 9b362e0495d3..d41555a4b31d 100644 --- a/rust/kernel/platform.rs +++ b/rust/kernel/platform.rs @@ -17,10 +17,7 @@ from_result, to_result, // }, - io::{ - mem::IoRequest, - Resource, // - }, + io::Resource, irq::{ self, IrqRequest, // @@ -31,6 +28,9 @@ ThisModule, // }; +#[cfg(CONFIG_HAS_IOMEM)] +use crate::io::mem::IoRequest; + use core::{ marker::PhantomData, mem::offset_of, @@ -307,6 +307,7 @@ pub fn resource_by_name(&self, name: &CStr) -> Option<&Resource> { } } +#[cfg(CONFIG_HAS_IOMEM)] impl Device { /// Returns an `IoRequest` for the resource at `index`, if any. pub fn io_request_by_index(&self, index: u32) -> Option> { diff --git a/samples/damon/mtier.c b/samples/damon/mtier.c index 3785b0c7ffb1..dae6929f827c 100644 --- a/samples/damon/mtier.c +++ b/samples/damon/mtier.c @@ -156,6 +156,9 @@ static struct damon_ctx *damon_sample_mtier_build_ctx(bool promote) if (!scheme) goto free_out; damon_set_schemes(ctx, &scheme, 1); + /* zero target value causes division by zero in damos_quota_store() */ + if (!node0_mem_used_bp || !node0_mem_free_bp) + goto free_out; quota_goal = damos_new_quota_goal( promote ? DAMOS_QUOTA_NODE_MEM_USED_BP : DAMOS_QUOTA_NODE_MEM_FREE_BP, diff --git a/security/integrity/ima/ima_appraise.c b/security/integrity/ima/ima_appraise.c index 18d0d9154317..ced2e131b061 100644 --- a/security/integrity/ima/ima_appraise.c +++ b/security/integrity/ima/ima_appraise.c @@ -274,8 +274,13 @@ static int xattr_verify(enum ima_hooks func, struct ima_iint_cache *iint, } else { set_bit(IMA_DIGSIG, &iint->atomic_flags); } - if (xattr_len - sizeof(xattr_value->type) - hash_start >= - iint->ima_hash->length) + /* + * Use addition, not subtraction: sizeof() forces unsigned + * math and a short xattr_len would wrap around, bypassing + * this bounds check. + */ + if (xattr_len >= (int)sizeof(xattr_value->type) + hash_start + + (int)iint->ima_hash->length) /* * xattr length may be longer. md5 hash in previous * version occupied 20 bytes in xattr, instead of 16 diff --git a/security/integrity/ima/ima_main.c b/security/integrity/ima/ima_main.c index 5cea53fc36df..ff52becc3031 100644 --- a/security/integrity/ima/ima_main.c +++ b/security/integrity/ima/ima_main.c @@ -687,6 +687,43 @@ static int ima_file_check(struct file *file, int mask) MAY_APPEND), FILE_CHECK, 0, false); } +/* + * ima_reset_action_flags - invalidate action flags after a content change + * @inode: inode of the file whose content is about to be truncated + * + * Clear IMA_DONE_MASK so the file is re-collected, re-measured, + * re-audited, and re-appraised on next access. + */ +static void ima_reset_action_flags(struct inode *inode) +{ + struct ima_iint_cache *iint; + + if (!ima_policy_flag || !S_ISREG(inode->i_mode)) + return; + + iint = ima_iint_find(inode); + if (!iint) + return; + + mutex_lock(&iint->mutex); + iint->flags &= ~IMA_DONE_MASK; + iint->measured_pcrs = 0; + mutex_unlock(&iint->mutex); + return; +} + +static int ima_path_truncate(const struct path *path) +{ + ima_reset_action_flags(path->dentry->d_inode); + return 0; +} + +static int ima_file_truncate(struct file *file) +{ + ima_reset_action_flags(file_inode(file)); + return 0; +} + static int __ima_inode_hash(struct inode *inode, struct file *file, char *buf, size_t buf_size) { @@ -1300,11 +1337,13 @@ static struct security_hook_list ima_hooks[] __ro_after_init = { LSM_HOOK_INIT(file_release, ima_file_free), LSM_HOOK_INIT(mmap_file, ima_file_mmap), LSM_HOOK_INIT(file_mprotect, ima_file_mprotect), + LSM_HOOK_INIT(file_truncate, ima_file_truncate), LSM_HOOK_INIT(kernel_load_data, ima_load_data), LSM_HOOK_INIT(kernel_post_load_data, ima_post_load_data), LSM_HOOK_INIT(kernel_read_file, ima_read_file), LSM_HOOK_INIT(kernel_post_read_file, ima_post_read_file), LSM_HOOK_INIT(path_post_mknod, ima_post_path_mknod), + LSM_HOOK_INIT(path_truncate, ima_path_truncate), #ifdef CONFIG_IMA_MEASURE_ASYMMETRIC_KEYS LSM_HOOK_INIT(key_post_create_or_update, ima_post_key_create_or_update), #endif diff --git a/security/selinux/hooks.c b/security/selinux/hooks.c index 8d6945edae7a..18dd28b2bb13 100644 --- a/security/selinux/hooks.c +++ b/security/selinux/hooks.c @@ -2974,6 +2974,10 @@ static int selinux_inode_init_security(struct inode *inode, struct inode *dir, sbsec = selinux_superblock(dir->i_sb); + if (!selinux_initialized() || + !(sbsec->flags & SBLABEL_MNT)) + return -EOPNOTSUPP; + newsid = crsec->create_sid; newsclass = inode_mode_to_security_class(inode->i_mode); rc = selinux_determine_inode_label(crsec, dir, qstr, newsclass, &newsid); @@ -2988,10 +2992,6 @@ static int selinux_inode_init_security(struct inode *inode, struct inode *dir, isec->initialized = LABEL_INITIALIZED; } - if (!selinux_initialized() || - !(sbsec->flags & SBLABEL_MNT)) - return -EOPNOTSUPP; - xattr = lsm_get_xattr_slot(xattrs, xattr_count); if (xattr) { rc = security_sid_to_context_force(newsid, diff --git a/security/selinux/ss/mls.c b/security/selinux/ss/mls.c index 3cd36e2015fa..40c62600650e 100644 --- a/security/selinux/ss/mls.c +++ b/security/selinux/ss/mls.c @@ -160,9 +160,6 @@ bool mls_level_isvalid(const struct policydb *p, const struct mls_level *l) { const char *name; const struct level_datum *levdatum; - struct ebitmap_node *node; - u32 bit; - int rc; if (!l->sens || l->sens > p->p_levels.nprim) return false; @@ -176,21 +173,14 @@ bool mls_level_isvalid(const struct policydb *p, const struct mls_level *l) return false; /* - * Validate that all bits set in l->cat are also be set in - * levdatum->level->cat and no bit in l->cat is larger than - * p->p_cats.nprim. + * l is valid iff every bit in l->cat is set in levdatum->level.cat + * and no bit in l->cat is larger than p->p_cats.nprim. + * policydb_index() has already verified that every bit set in + * levdatum->level.cat names a defined category, so containment is + * sufficient here. */ - rc = ebitmap_contains(&levdatum->level.cat, &l->cat, - p->p_cats.nprim); - if (!rc) - return false; - - ebitmap_for_each_positive_bit(&levdatum->level.cat, node, bit) { - if (!sym_name(p, SYM_CATS, bit)) - return false; - } - - return true; + return ebitmap_contains(&levdatum->level.cat, &l->cat, + p->p_cats.nprim); } bool mls_range_isvalid(const struct policydb *p, const struct mls_range *r) diff --git a/security/selinux/ss/policydb.c b/security/selinux/ss/policydb.c index ead504a639e3..8a32666c0ba2 100644 --- a/security/selinux/ss/policydb.c +++ b/security/selinux/ss/policydb.c @@ -665,6 +665,23 @@ static int cat_index(void *key, void *datum, void *datap) return 0; } +static int sens_cat_index_check(void *key, void *datum, void *datap) +{ + struct policydb *p = datap; + struct level_datum *levdatum = datum; + struct ebitmap_node *node; + u32 bit; + + ebitmap_for_each_positive_bit(&levdatum->level.cat, node, bit) { + if (bit >= p->p_cats.nprim || !sym_name(p, SYM_CATS, bit)) { + pr_err("SELinux: sensitivity %s allows undefined category %u\n", + (const char *)key, bit + 1); + return -EINVAL; + } + } + return 0; +} + /* clang-format off */ static int (*const index_f[SYM_NUM])(void *key, void *datum, void *datap) = { common_index, @@ -719,6 +736,7 @@ static inline void symtab_hash_eval(struct symtab *s) static int policydb_index(struct policydb *p) { int i, rc; + u32 v; if (p->mls_enabled) pr_debug( @@ -769,6 +787,30 @@ static int policydb_index(struct policydb *p) if (rc) goto out; } + + /* + * A sparse class value is absorbed by policydb_class_isvalid() and + * its siblings, but no such predicate exists for booleans: every + * user of bool_val_to_struct[] walks it by index and dereferences + * each entry -- cond_evaluate_expr(), the two getters and + * security_set_bools() -- so an unclaimed one has no consumer that + * can tolerate it. + */ + for (v = 0; v < p->p_bools.nprim; v++) { + if (!p->bool_val_to_struct[v]) { + pr_err("SELinux: boolean %u is declared but not defined\n", + v + 1); + rc = -EINVAL; + goto out; + } + } + + if (p->mls_enabled) { + rc = hashtab_map(&p->p_levels.table, sens_cat_index_check, p); + if (rc) + goto out; + } + rc = 0; out: return rc; @@ -1154,7 +1196,18 @@ int str_read(char **strp, gfp_t flags, struct policy_file *fp, u32 len) return 0; } -static int perm_read(struct policydb *p, struct symtab *s, struct policy_file *fp) +/* + * Bitmap of the permission values a symtab has claimed. Values are 1-based + * and bounded by SEL_VEC_MAX, the width of an access vector, so the whole set + * fits in a u32 and the callers reject an nprim past that width. + */ +static u32 perm_claimed_mask(u32 nprim) +{ + return nprim ? U32_MAX >> (SEL_VEC_MAX - nprim) : 0; +} + +static int perm_read(struct policydb *p, struct symtab *s, + struct policy_file *fp, u32 *claimed) { char *key = NULL; struct perm_datum *perdatum; @@ -1175,6 +1228,13 @@ static int perm_read(struct policydb *p, struct symtab *s, struct policy_file *f rc = -EINVAL; if (perdatum->value < 1 || perdatum->value > SEL_VEC_MAX) goto bad; + /* indexes an nprim-sized array in security_get_permissions() */ + if (perdatum->value > s->nprim) + goto bad; + /* two permissions cannot share one slot of that array */ + if (*claimed & (1U << (perdatum->value - 1))) + goto bad; + *claimed |= 1U << (perdatum->value - 1); rc = str_read(&key, GFP_KERNEL, fp, len); if (rc) @@ -1195,7 +1255,7 @@ static int common_read(struct policydb *p, struct symtab *s, struct policy_file char *key = NULL; struct common_datum *comdatum; __le32 buf[4]; - u32 i, len, nel; + u32 i, len, nel, claimed = 0; int rc; comdatum = kzalloc_obj(*comdatum); @@ -1222,17 +1282,28 @@ static int common_read(struct policydb *p, struct symtab *s, struct policy_file if (rc) goto bad; comdatum->permissions.nprim = le32_to_cpu(buf[2]); + /* no permission value can reach a slot past SEL_VEC_MAX */ + rc = -EINVAL; + if (comdatum->permissions.nprim > SEL_VEC_MAX) + goto bad; rc = str_read(&key, GFP_KERNEL, fp, len); if (rc) goto bad; for (i = 0; i < nel; i++) { - rc = perm_read(p, &comdatum->permissions, fp); + rc = perm_read(p, &comdatum->permissions, fp, &claimed); if (rc) goto bad; } + rc = -EINVAL; + if (claimed != perm_claimed_mask(comdatum->permissions.nprim)) { + pr_err("SELinux: common %s does not define every permission it declares\n", + key); + goto bad; + } + hash_eval(&comdatum->permissions.table, "common_permissions", key); rc = symtab_insert(s, key, comdatum); @@ -1366,7 +1437,7 @@ static int class_read(struct policydb *p, struct symtab *s, struct policy_file * char *key = NULL; struct class_datum *cladatum; __le32 buf[6]; - u32 i, len, len2, ncons, nel, val; + u32 i, len, len2, ncons, nel, val, claimed = 0, inherited = 0; int rc; cladatum = kzalloc_obj(*cladatum); @@ -1399,6 +1470,10 @@ static int class_read(struct policydb *p, struct symtab *s, struct policy_file * if (rc) goto bad; cladatum->permissions.nprim = le32_to_cpu(buf[3]); + /* no permission value can reach a slot past SEL_VEC_MAX */ + rc = -EINVAL; + if (cladatum->permissions.nprim > SEL_VEC_MAX) + goto bad; ncons = le32_to_cpu(buf[5]); @@ -1419,13 +1494,36 @@ static int class_read(struct policydb *p, struct symtab *s, struct policy_file * cladatum->comkey); goto bad; } + + /* + * security_get_permissions() maps the common's permissions + * into an array sized by this class's nprim, so a class must + * declare at least as many as the common it inherits. + */ + if (cladatum->permissions.nprim < + cladatum->comdatum->permissions.nprim) { + pr_err("SELinux: class %s has fewer permissions than common %s\n", + key, cladatum->comkey); + goto bad; + } } for (i = 0; i < nel; i++) { - rc = perm_read(p, &cladatum->permissions, fp); + rc = perm_read(p, &cladatum->permissions, fp, &claimed); if (rc) goto bad; } + /* the class's own permissions must claim the slots the common leaves */ + if (cladatum->comdatum) + inherited = cladatum->comdatum->permissions.nprim; + rc = -EINVAL; + if (claimed != (perm_claimed_mask(cladatum->permissions.nprim) & + ~perm_claimed_mask(inherited))) { + pr_err("SELinux: class %s does not define every permission it declares\n", + key); + goto bad; + } + hash_eval(&cladatum->permissions.table, "class_permissions", key); rc = read_cons_helper(p, &cladatum->constraints, ncons, 0, fp); diff --git a/security/selinux/ss/services.c b/security/selinux/ss/services.c index 2d828548f3db..7afce975436e 100644 --- a/security/selinux/ss/services.c +++ b/security/selinux/ss/services.c @@ -2221,7 +2221,9 @@ void selinux_policy_cancel(struct selinux_load_state *load_state) oldpolicy = rcu_dereference_protected(state->policy, lockdep_is_held(&state->policy_mutex)); - sidtab_cancel_convert(oldpolicy->sidtab); + /* a first load has no outgoing policy and converted nothing */ + if (oldpolicy) + sidtab_cancel_convert(oldpolicy->sidtab); selinux_policy_free(load_state->policy); kfree(load_state->convert_data); } @@ -3302,6 +3304,7 @@ int security_get_classes(struct selinux_policy *policy, char ***classes, u32 *nclasses) { struct policydb *policydb; + u32 i; int rc; policydb = &policy->policydb; @@ -3314,16 +3317,29 @@ int security_get_classes(struct selinux_policy *policy, rc = hashtab_map(&policydb->p_classes.table, get_classes_callback, *classes); - if (rc) { - u32 i; + if (rc) + goto err; - for (i = 0; i < *nclasses; i++) - kfree((*classes)[i]); - kfree(*classes); + /* + * The class symtab may be sparse, which policydb_class_isvalid() exists + * to absorb; the callback fills this array by value, so an unclaimed + * one leaves a NULL that sel_make_classes() hands to sel_make_dir(). + */ + for (i = 0; i < *nclasses; i++) { + if (!(*classes)[i]) { + rc = -EINVAL; + goto err; + } } out: return rc; + +err: + for (i = 0; i < *nclasses; i++) + kfree((*classes)[i]); + kfree(*classes); + return rc; } static int get_permissions_callback(void *k, void *d, void *args) diff --git a/sound/soc/amd/yc/acp6x-mach.c b/sound/soc/amd/yc/acp6x-mach.c index 14b970f16020..ae90e9f513d1 100644 --- a/sound/soc/amd/yc/acp6x-mach.c +++ b/sound/soc/amd/yc/acp6x-mach.c @@ -731,6 +731,13 @@ static const struct dmi_system_id yc_acp_quirk_table[] = { DMI_MATCH(DMI_BOARD_NAME, "8E35"), } }, + { + .driver_data = &acp6x_card, + .matches = { + DMI_MATCH(DMI_BOARD_VENDOR, "HP"), + DMI_MATCH(DMI_BOARD_NAME, "8F06"), + } + }, { .driver_data = &acp6x_card, .matches = { diff --git a/sound/soc/codecs/Kconfig b/sound/soc/codecs/Kconfig index d43475a63ac2..e4d139108d72 100644 --- a/sound/soc/codecs/Kconfig +++ b/sound/soc/codecs/Kconfig @@ -2157,6 +2157,7 @@ config SND_SOC_TAC5XX2_SDW tristate "Texas Instruments TAC5XX2 SoundWire Smart Amplifier" depends on SOUNDWIRE depends on SND_SOC_SDCA + select REGMAP_SOUNDWIRE_MBQ help This option enables support for Texas Instruments TAC5XX2 family of SoundWire Smart Amplifiers. This includes TAC5572, TAC5672, diff --git a/sound/soc/codecs/rt1320-sdw.c b/sound/soc/codecs/rt1320-sdw.c index 1e930b27c67a..42acb6cef77a 100644 --- a/sound/soc/codecs/rt1320-sdw.c +++ b/sound/soc/codecs/rt1320-sdw.c @@ -1899,7 +1899,7 @@ static int rt1320_io_init(struct device *dev, struct sdw_slave *slave) dev_dbg(dev, "%s amp func_status=0x%x\n", __func__, amp_func_status); /* initialization write */ - if ((amp_func_status & FUNCTION_NEEDS_INITIALIZATION)) { + if ((amp_func_status & FUNCTION_NEEDS_INITIALIZATION) || !rt1320->first_hw_init) { switch (rt1320->dev_id) { case RT1320_DEV_ID: if (rt1320->version_id < RT1320_VC) diff --git a/sound/soc/codecs/rt5645.c b/sound/soc/codecs/rt5645.c index e9819653b30d..e7e39671a576 100644 --- a/sound/soc/codecs/rt5645.c +++ b/sound/soc/codecs/rt5645.c @@ -3496,6 +3496,10 @@ static int rt5645_probe(struct snd_soc_component *component) if (!rt5645->eq_param) return -ENOMEM; + /* no machine driver to call rt5645_set_jack_detect(), so detect here */ + if (!rt5645->pdata.jd_mode && rt5645->gpiod_hp_det) + rt5645_irq(0, rt5645); + return 0; } diff --git a/sound/soc/codecs/tas2781-i2c.c b/sound/soc/codecs/tas2781-i2c.c index 209067e98e1f..95d875c44b91 100644 --- a/sound/soc/codecs/tas2781-i2c.c +++ b/sound/soc/codecs/tas2781-i2c.c @@ -1544,8 +1544,8 @@ static ssize_t acoustic_ctl_write(struct file *file, if (src[0] > max_pkg_len && src[0] != count) { dev_err(priv->dev, "pkg(%u), max(%u), count(%u) mismatch.\n", src[0], max_pkg_len, (unsigned int)count); - ret = 0; - goto exit; + kfree(src); + return 0; } switch (src[1]) { @@ -1559,14 +1559,14 @@ static ssize_t acoustic_ctl_write(struct file *file, break; default: dev_err(priv->dev, "%s Wrong code %02x.\n", __func__, src[1]); - ret = 0; - goto exit; + kfree(src); + return 0; } if (len < 1) { dev_err(priv->dev, "pkg fmt invalid %02x.\n", len); - ret = 0; - goto exit; + kfree(src); + return 0; } for (j = 0; j < priv->ndev; j++) @@ -1576,8 +1576,8 @@ static ssize_t acoustic_ctl_write(struct file *file, } if (j >= priv->ndev) { dev_err(priv->dev, "no such device 0x%02x.\n", src[2]); - ret = 0; - goto exit; + kfree(src); + return 0; } reg = TASDEVICE_REG(src[3], src[4], src[5]); @@ -1608,7 +1608,7 @@ static ssize_t acoustic_ctl_write(struct file *file, dev_err(priv->dev, "i2c communication error.\n"); else ret = count; -exit: + kfree(src); return ret; } diff --git a/sound/soc/intel/boards/sof_es8336.c b/sound/soc/intel/boards/sof_es8336.c index e9ee752d5ec3..9b016136c639 100644 --- a/sound/soc/intel/boards/sof_es8336.c +++ b/sound/soc/intel/boards/sof_es8336.c @@ -847,6 +847,16 @@ static const struct platform_device_id board_ids[] = { SOF_ES8336_SPEAKERS_EN_GPIO1_QUIRK | SOF_ES8336_JD_INVERTED), }, + { + .name = "nvl_es83x6_c1_h02", + .driver_data = (kernel_ulong_t)(SOF_ES8336_SSP_CODEC(1) | + SOF_NO_OF_HDMI_CAPTURE_SSP(2) | + SOF_HDMI_CAPTURE_1_SSP(0) | + SOF_HDMI_CAPTURE_2_SSP(2) | + SOF_SSP_HDMI_CAPTURE_PRESENT | + SOF_ES8336_SPEAKERS_EN_GPIO1_QUIRK | + SOF_ES8336_JD_INVERTED), + }, { } }; MODULE_DEVICE_TABLE(platform, board_ids); diff --git a/sound/soc/intel/boards/sof_rt5682.c b/sound/soc/intel/boards/sof_rt5682.c index 649378957b20..7899f7ffd99b 100644 --- a/sound/soc/intel/boards/sof_rt5682.c +++ b/sound/soc/intel/boards/sof_rt5682.c @@ -915,6 +915,13 @@ static const struct platform_device_id board_ids[] = { /* SSP 0 and SSP 2 are used for HDMI IN */ SOF_SSP_MASK_HDMI_CAPTURE(0x5)), }, + { + .name = "nvl_rt5682_c1_h02", + .driver_data = (kernel_ulong_t)(SOF_RT5682_MCLK_EN | + SOF_SSP_PORT_CODEC(1) | + /* SSP 0 and SSP 2 are used for HDMI IN */ + SOF_SSP_MASK_HDMI_CAPTURE(0x5)), + }, { } }; MODULE_DEVICE_TABLE(platform, board_ids); diff --git a/sound/soc/intel/boards/sof_ssp_amp.c b/sound/soc/intel/boards/sof_ssp_amp.c index 729c0cd7c19c..b80d24966cbd 100644 --- a/sound/soc/intel/boards/sof_ssp_amp.c +++ b/sound/soc/intel/boards/sof_ssp_amp.c @@ -222,6 +222,12 @@ static const struct platform_device_id board_ids[] = { /* SSP 0 and SSP 2 are used for HDMI IN */ SOF_HDMI_PLAYBACK_PRESENT), }, + { + .name = "nvl_lt6911_hdmi_ssp", + .driver_data = (kernel_ulong_t)(SOF_SSP_MASK_HDMI_CAPTURE(0x5) | + /* SSP 0 and SSP 2 are used for HDMI IN */ + SOF_HDMI_PLAYBACK_PRESENT), + }, { } }; MODULE_DEVICE_TABLE(platform, board_ids); diff --git a/sound/soc/intel/common/soc-acpi-intel-nvl-match.c b/sound/soc/intel/common/soc-acpi-intel-nvl-match.c index a7f4097d426c..8f6e987c791e 100644 --- a/sound/soc/intel/common/soc-acpi-intel-nvl-match.c +++ b/sound/soc/intel/common/soc-acpi-intel-nvl-match.c @@ -16,12 +16,24 @@ static const struct snd_soc_acpi_codecs nvl_essx_83x6 = { .codecs = { "ESSX8316", "ESSX8326", "ESSX8336"}, }; +static const struct snd_soc_acpi_codecs nvl_lt6911_hdmi = { + .num_codecs = 1, + .codecs = {"INTC10B0"} +}; + static const struct snd_soc_acpi_codecs nvl_rt5682_rt5682s_hp = { .num_codecs = 2, .codecs = {RT5682_ACPI_HID, RT5682S_ACPI_HID}, }; struct snd_soc_acpi_mach snd_soc_acpi_intel_nvl_machines[] = { + { + .comp_ids = &nvl_essx_83x6, + .drv_name = "nvl_es83x6_c1_h02", + .machine_quirk = snd_soc_acpi_codec_list, + .quirk_data = &nvl_lt6911_hdmi, + .sof_tplg_filename = "sof-nvl-es83x6-ssp1-hdmi-ssp02.tplg", + }, { .comp_ids = &nvl_essx_83x6, .drv_name = "sof-essx8336", @@ -30,6 +42,13 @@ struct snd_soc_acpi_mach snd_soc_acpi_intel_nvl_machines[] = { SND_SOC_ACPI_TPLG_INTEL_SSP_MSB | SND_SOC_ACPI_TPLG_INTEL_DMIC_NUMBER, }, + { + .comp_ids = &nvl_rt5682_rt5682s_hp, + .drv_name = "nvl_rt5682_c1_h02", + .machine_quirk = snd_soc_acpi_codec_list, + .quirk_data = &nvl_lt6911_hdmi, + .sof_tplg_filename = "sof-nvl-rt5682-ssp1-hdmi-ssp02.tplg", + }, { .comp_ids = &nvl_rt5682_rt5682s_hp, .drv_name = "sof_rt5682", @@ -37,6 +56,12 @@ struct snd_soc_acpi_mach snd_soc_acpi_intel_nvl_machines[] = { .tplg_quirk_mask = SND_SOC_ACPI_TPLG_INTEL_SSP_NUMBER | SND_SOC_ACPI_TPLG_INTEL_SSP_MSB, }, + /* place amp/hdmi-in only boards in the end of table */ + { + .id = "INTC10B0", + .drv_name = "nvl_lt6911_hdmi_ssp", + .sof_tplg_filename = "sof-nvl-hdmi-ssp02.tplg", + }, {}, }; EXPORT_SYMBOL_GPL(snd_soc_acpi_intel_nvl_machines); diff --git a/sound/soc/xilinx/xlnx_formatter_pcm.c b/sound/soc/xilinx/xlnx_formatter_pcm.c index 8f7a76758535..f15336197ed3 100644 --- a/sound/soc/xilinx/xlnx_formatter_pcm.c +++ b/sound/soc/xilinx/xlnx_formatter_pcm.c @@ -281,8 +281,7 @@ static irqreturn_t xlnx_mm2s_irq_handler(int irq, void *arg) { u32 val; void __iomem *reg; - struct device *dev = arg; - struct xlnx_pcm_drv_data *adata = dev_get_drvdata(dev); + struct xlnx_pcm_drv_data *adata = arg; reg = adata->mmio + XLNX_MM2S_OFFSET + XLNX_AUD_STS; val = readl(reg); @@ -300,8 +299,7 @@ static irqreturn_t xlnx_s2mm_irq_handler(int irq, void *arg) { u32 val; void __iomem *reg; - struct device *dev = arg; - struct xlnx_pcm_drv_data *adata = dev_get_drvdata(dev); + struct xlnx_pcm_drv_data *adata = arg; reg = adata->mmio + XLNX_S2MM_OFFSET + XLNX_AUD_STS; val = readl(reg); @@ -637,7 +635,7 @@ static int xlnx_formatter_pcm_probe(struct platform_device *pdev) } ret = devm_request_irq(dev, aud_drv_data->mm2s_irq, xlnx_mm2s_irq_handler, 0, - "xlnx_formatter_pcm_mm2s_irq", dev); + "xlnx_formatter_pcm_mm2s_irq", aud_drv_data); if (ret) { dev_err(dev, "xlnx audio mm2s irq request failed\n"); goto clk_err; @@ -664,7 +662,7 @@ static int xlnx_formatter_pcm_probe(struct platform_device *pdev) ret = devm_request_irq(dev, aud_drv_data->s2mm_irq, xlnx_s2mm_irq_handler, 0, "xlnx_formatter_pcm_s2mm_irq", - dev); + aud_drv_data); if (ret) { dev_err(dev, "xlnx audio s2mm irq request failed\n"); goto clk_err; diff --git a/tools/testing/selftests/bpf/progs/dynptr_fail.c b/tools/testing/selftests/bpf/progs/dynptr_fail.c index 344fb2aa0813..29c6361d8820 100644 --- a/tools/testing/selftests/bpf/progs/dynptr_fail.c +++ b/tools/testing/selftests/bpf/progs/dynptr_fail.c @@ -1635,6 +1635,36 @@ static int callback(__u32 index, void *data) return 0; } +/* A commuted add should preserve the parent id of a dynptr data slice. */ +SEC("?raw_tp") +__failure __msg("invalid mem access 'scalar'") +int dynptr_slice_commuted_invalidate(void *ctx) +{ + struct bpf_dynptr ptr; + __u32 *slice, *derived; + + bpf_ringbuf_reserve_dynptr(&ringbuf, sizeof(__u32), 0, &ptr); + + slice = bpf_dynptr_data(&ptr, 0, sizeof(__u32)); + if (!slice) + goto done; + + asm volatile ("%[dst] = 0;" + "%[dst] += %[src];" + "%[src] = 0;" + : [dst]"=&r"(derived), [src]"+r"(slice) + : + : "memory"); + + bpf_ringbuf_discard_dynptr(&ptr, 0); + val = *derived; + return 0; + +done: + bpf_ringbuf_discard_dynptr(&ptr, 0); + return 0; +} + /* If the dynptr is written into in a callback function, its data * slices should be invalidated as well. */ diff --git a/tools/testing/selftests/bpf/progs/mem_rdonly_untrusted.c b/tools/testing/selftests/bpf/progs/mem_rdonly_untrusted.c index 5b4453747c23..f166fff8f217 100644 --- a/tools/testing/selftests/bpf/progs/mem_rdonly_untrusted.c +++ b/tools/testing/selftests/bpf/progs/mem_rdonly_untrusted.c @@ -226,4 +226,21 @@ int null_check(void *ctx) return 0; } +SEC("socket") +__success +__retval(1) +int ldx_is_ok_commuted_addr(void *ctx) +{ + int v, *p, *derived; + + v = 1; + p = bpf_rdonly_cast(&v, 0); + asm volatile ("%[dst] = 0;" + "%[dst] += %[src];" + : [dst]"=&r"(derived) + : [src]"r"(p) + : "memory"); + return *derived; +} + char _license[] SEC("license") = "GPL"; diff --git a/tools/testing/selftests/bpf/progs/verifier_basic_stack.c b/tools/testing/selftests/bpf/progs/verifier_basic_stack.c index fb62e09f2114..d3df7a9f1d8c 100644 --- a/tools/testing/selftests/bpf/progs/verifier_basic_stack.c +++ b/tools/testing/selftests/bpf/progs/verifier_basic_stack.c @@ -97,4 +97,45 @@ __naked void misaligned_read_from_stack(void) " ::: __clobber_all); } +SEC("socket") +__description("stack pointer arithmetic preserves frame number") +__failure __msg("R7 invalid mem access 'scalar'") +__naked void stack_ptr_arith_preserves_frameno(void) +{ + asm volatile ("\ + r3 = 0; \ + *(u64 *)(r10 - 8) = r3; \ + r1 = %[map_hash_8b] ll; \ + r2 = r10; \ + r2 += -8; \ + call %[bpf_map_lookup_elem]; \ + if r0 != 0 goto +2; \ + r0 = 0; \ + exit; \ + r1 = r0; \ + r2 = 0; \ + r3 = 0; \ + call stack_ptr_arith_preserves_frameno_subprog;\ + r0 = 0; \ + exit; \ + ": + : __imm(bpf_map_lookup_elem), + __imm_addr(map_hash_8b) + : __clobber_all); +} + +static __used __naked void stack_ptr_arith_preserves_frameno_subprog(void) +{ + asm volatile ("\ + *(u64 *)(r10 - 8) = r1; \ + r6 = -8; \ + r6 += r10; \ + *(u64 *)(r6 + 0) = r2; \ + r7 = *(u64 *)(r10 - 8); \ + *(u64 *)(r7 + 0) = r3; \ + r0 = 0; \ + exit; \ + "::: __clobber_all); +} + char _license[] SEC("license") = "GPL"; diff --git a/tools/testing/selftests/ftrace/test.d/dynevent/add_remove_eprobe.tc b/tools/testing/selftests/ftrace/test.d/dynevent/add_remove_eprobe.tc index c300eb020262..e2322693d0c3 100644 --- a/tools/testing/selftests/ftrace/test.d/dynevent/add_remove_eprobe.tc +++ b/tools/testing/selftests/ftrace/test.d/dynevent/add_remove_eprobe.tc @@ -1,16 +1,16 @@ #!/bin/sh # SPDX-License-Identifier: GPL-2.0 # description: Generic dynamic event - add/remove eprobe events -# requires: dynamic_events events/syscalls/sys_enter_openat ". []":README +# requires: dynamic_events events/syscalls/sys_enter_chdir ". []":README echo 0 > events/enable clear_dynamic_events SYSTEM="syscalls" -EVENT="sys_enter_openat" +EVENT="sys_enter_chdir" FIELD="filename" -EPROBE="eprobe_open" +EPROBE="eprobe_chdir" OPTIONS="file=+0(\$filename):ustring" echo "e:$EPROBE $SYSTEM/$EVENT $OPTIONS" >> dynamic_events @@ -18,20 +18,14 @@ grep -q "$EPROBE" dynamic_events test -d events/eprobes/$EPROBE echo 1 > events/eprobes/$EPROBE/enable -ls +cd /sys/kernel/tracing echo 0 > events/eprobes/$EPROBE/enable -content=`grep '^ *ls-' trace | grep 'file='` -nocontent=`grep '^ *ls-' trace | grep 'file=' | grep -v -e '"/' -e '"."' -e '(fault)' ` || true - +content=`grep -e 'file="/sys/kernel/tracing"\|(fault)' trace` if [ -z "$content" ]; then exit_fail fi -if [ ! -z "$nocontent" ]; then - exit_fail -fi - echo "-:$EPROBE" >> dynamic_events ! grep -q "$EPROBE" dynamic_events diff --git a/tools/testing/selftests/net/mptcp/mptcp_join.sh b/tools/testing/selftests/net/mptcp/mptcp_join.sh index c0aeffd5cb71..7dc91fac4917 100755 --- a/tools/testing/selftests/net/mptcp/mptcp_join.sh +++ b/tools/testing/selftests/net/mptcp/mptcp_join.sh @@ -584,7 +584,7 @@ check_transfer() mv "$tmpfile" "$out" tmpfile="" fi - cmp -l "$in" "$out" | while read -r i a b; do + while read -r i a b; do local sum=$((0${a} + 0${b})) if [ $check_invert -eq 0 ] || [ $sum -ne $((0xff)) ]; then fail_test "$what does not match (in, out):" @@ -595,7 +595,7 @@ check_transfer() else print_info "$what has inverted byte at ${i}" fi - done + done < <(cmp -l "$in" "$out") return 0 } diff --git a/tools/testing/selftests/net/tls.c b/tools/testing/selftests/net/tls.c index cbdd3ea28b99..a0e72e402748 100644 --- a/tools/testing/selftests/net/tls.c +++ b/tools/testing/selftests/net/tls.c @@ -835,6 +835,43 @@ TEST_F(tls, send_and_splice) EXPECT_EQ(memcmp(mem_send, mem_recv, send_len), 0); } +TEST_F(tls, splice_onto_full_record) +{ + char mem_send[4608]; + char mem_recv[4608]; + int frag_len = 100; + int nfrags, i, off; + int p[2]; + + memrnd(mem_send, sizeof(mem_send)); + ASSERT_GE(pipe(p), 0); + + for (nfrags = 16; nfrags <= 44; nfrags++) { + for (i = 0, off = 0; i < nfrags; i++, off += frag_len) { + EXPECT_EQ(write(p[1], mem_send + off, frag_len), frag_len); + EXPECT_EQ(splice(p[0], NULL, self->fd, NULL, frag_len, + SPLICE_F_MORE), frag_len); + } + + EXPECT_EQ(send(self->fd, mem_send + off, 1, MSG_MORE), 1); + off++; + + EXPECT_EQ(write(p[1], mem_send + off, frag_len), frag_len); + EXPECT_EQ(splice(p[0], NULL, self->fd, NULL, frag_len, + SPLICE_F_MORE), frag_len); + off += frag_len; + + EXPECT_EQ(send(self->fd, mem_send + off, 1, 0), 1); + off++; + + EXPECT_EQ(recv(self->cfd, mem_recv, off, MSG_WAITALL), off); + EXPECT_EQ(memcmp(mem_send, mem_recv, off), 0); + } + + close(p[0]); + close(p[1]); +} + TEST_F(tls, splice_to_pipe) { int send_len = TLS_PAYLOAD_MAX_LEN; @@ -1802,6 +1839,63 @@ TEST_F(tls, recv_efault) } #define TLS_RECORD_TYPE_HANDSHAKE 0x16 + +TEST_F(tls_basic, recvmsg_nopad_retry_iov) +{ + char payload[32]; + char first_iov[sizeof(payload)]; + char later_iov[sizeof(payload) * 2]; + char expected_later_iov[sizeof(later_iov)]; + char cbuf[CMSG_SPACE(sizeof(char))]; + struct tls_crypto_info_keys tls13; + struct iovec iov[] = { + { .iov_base = first_iov, .iov_len = sizeof(first_iov) }, + { .iov_base = later_iov, .iov_len = sizeof(later_iov) }, + }; + struct msghdr msg = { + .msg_iov = iov, + .msg_iovlen = ARRAY_SIZE(iov), + .msg_control = cbuf, + .msg_controllen = sizeof(cbuf), + }; + int one = 1; + int ret; + int i; + + if (self->notls) + SKIP(return, "no TLS support"); + + tls_crypto_info_init(TLS_1_3_VERSION, TLS_CIPHER_AES_GCM_128, + &tls13, 0); + + ret = setsockopt(self->fd, SOL_TLS, TLS_TX, &tls13, tls13.len); + ASSERT_EQ(ret, 0); + + ret = setsockopt(self->cfd, SOL_TLS, TLS_RX, &tls13, tls13.len); + ASSERT_EQ(ret, 0); + + ret = setsockopt(self->cfd, SOL_TLS, TLS_RX_EXPECT_NO_PAD, + &one, sizeof(one)); + ASSERT_EQ(ret, 0); + + for (i = 0; i < sizeof(payload); i++) + payload[i] = 0x40 + i; + memset(first_iov, 0xa5, sizeof(first_iov)); + memset(later_iov, 0x5a, sizeof(later_iov)); + memset(expected_later_iov, 0x5a, sizeof(expected_later_iov)); + + /* A control record forces optimistic TLS 1.3 RX to retry. */ + ret = tls_send_cmsg(self->fd, TLS_RECORD_TYPE_HANDSHAKE, + payload, sizeof(payload), 0); + ASSERT_EQ(ret, sizeof(payload)); + + ret = recvmsg(self->cfd, &msg, 0); + ASSERT_EQ(ret, sizeof(payload)); + EXPECT_EQ(memcmp(first_iov, payload, sizeof(payload)), 0); + EXPECT_EQ(memcmp(later_iov, expected_later_iov, + sizeof(later_iov)), 0); +} + /* key_update, length 1, update_not_requested */ static const char key_update_msg[] = "\x18\x00\x00\x01\x00"; static void tls_send_keyupdate(struct __test_metadata *_metadata, int fd) diff --git a/tools/testing/selftests/sched_ext/numa.bpf.c b/tools/testing/selftests/sched_ext/numa.bpf.c index 78cc49a7f9a6..6b4515c28aa0 100644 --- a/tools/testing/selftests/sched_ext/numa.bpf.c +++ b/tools/testing/selftests/sched_ext/numa.bpf.c @@ -34,7 +34,8 @@ static bool is_cpu_idle(s32 cpu, int node) s32 BPF_STRUCT_OPS(numa_select_cpu, struct task_struct *p, s32 prev_cpu, u64 wake_flags) { - int node = __COMPAT_scx_bpf_cpu_node(scx_bpf_task_cpu(p)); + s32 task_cpu = scx_bpf_task_cpu(p); + int node = __COMPAT_scx_bpf_cpu_node(task_cpu); s32 cpu; /* @@ -48,6 +49,16 @@ s32 BPF_STRUCT_OPS(numa_select_cpu, cpu = __COMPAT_scx_bpf_pick_any_cpu_node(p->cpus_ptr, node, __COMPAT_SCX_PICK_IDLE_IN_NODE); + /* + * @task_cpu may be outside of p->cpus_ptr if @p's affinity + * changed while it was sleeping. This means it's possible for + * p->cpus_ptr to not include any CPUs from @node. + * If we failed to find a cpu in @node, check if @task_cpu + * is outside of p->cpus_ptr and just return @prev_cpu if it is. + */ + if (cpu < 0 && !bpf_cpumask_test_cpu(task_cpu, p->cpus_ptr)) + return prev_cpu; + if (is_cpu_idle(cpu, node)) scx_bpf_error("CPU %d should be marked as busy", cpu); diff --git a/tools/testing/selftests/tc-testing/tc-tests/qdiscs/depth.json b/tools/testing/selftests/tc-testing/tc-tests/qdiscs/depth.json new file mode 100644 index 000000000000..48f8683f0d1a --- /dev/null +++ b/tools/testing/selftests/tc-testing/tc-tests/qdiscs/depth.json @@ -0,0 +1,74 @@ +[ + { + "id": "ad43", + "name": "Create qdisc hierarchy at maximum depth", + "category": [ + "qdisc", + "depth" + ], + "plugins": { + "requires": "nsPlugin" + }, + "setup": [ + "$TC qdisc add dev $DUMMY root handle 1: htb default 1", + "$TC class add dev $DUMMY parent 1: classid 1:1 htb rate 10mbit", + "$TC qdisc add dev $DUMMY parent 1:1 handle 2: htb default 1", + "$TC class add dev $DUMMY parent 2: classid 2:1 htb rate 10mbit", + "$TC qdisc add dev $DUMMY parent 2:1 handle 3: htb default 1", + "$TC class add dev $DUMMY parent 3: classid 3:1 htb rate 10mbit", + "$TC qdisc add dev $DUMMY parent 3:1 handle 4: htb default 1", + "$TC class add dev $DUMMY parent 4: classid 4:1 htb rate 10mbit", + "$TC qdisc add dev $DUMMY parent 4:1 handle 5: htb default 1", + "$TC class add dev $DUMMY parent 5: classid 5:1 htb rate 10mbit", + "$TC qdisc add dev $DUMMY parent 5:1 handle 6: htb default 1", + "$TC class add dev $DUMMY parent 6: classid 6:1 htb rate 10mbit", + "$TC qdisc add dev $DUMMY parent 6:1 handle 7: htb default 1", + "$TC class add dev $DUMMY parent 7: classid 7:1 htb rate 10mbit" + ], + "cmdUnderTest": "$TC qdisc add dev $DUMMY parent 7:1 handle 8: htb default 1", + "expExitCode": "0", + "verifyCmd": "$TC qdisc show dev $DUMMY", + "matchPattern": "qdisc htb 8: parent 7:1", + "matchCount": "1", + "teardown": [ + "$TC qdisc del dev $DUMMY root" + ] + }, + { + "id": "c519", + "name": "Reject qdisc hierarchy above maximum depth", + "category": [ + "qdisc", + "depth" + ], + "plugins": { + "requires": "nsPlugin" + }, + "setup": [ + "$TC qdisc add dev $DUMMY root handle 1: htb default 1", + "$TC class add dev $DUMMY parent 1: classid 1:1 htb rate 10mbit", + "$TC qdisc add dev $DUMMY parent 1:1 handle 2: htb default 1", + "$TC class add dev $DUMMY parent 2: classid 2:1 htb rate 10mbit", + "$TC qdisc add dev $DUMMY parent 2:1 handle 3: htb default 1", + "$TC class add dev $DUMMY parent 3: classid 3:1 htb rate 10mbit", + "$TC qdisc add dev $DUMMY parent 3:1 handle 4: htb default 1", + "$TC class add dev $DUMMY parent 4: classid 4:1 htb rate 10mbit", + "$TC qdisc add dev $DUMMY parent 4:1 handle 5: htb default 1", + "$TC class add dev $DUMMY parent 5: classid 5:1 htb rate 10mbit", + "$TC qdisc add dev $DUMMY parent 5:1 handle 6: htb default 1", + "$TC class add dev $DUMMY parent 6: classid 6:1 htb rate 10mbit", + "$TC qdisc add dev $DUMMY parent 6:1 handle 7: htb default 1", + "$TC class add dev $DUMMY parent 7: classid 7:1 htb rate 10mbit", + "$TC qdisc add dev $DUMMY parent 7:1 handle 8: htb default 1", + "$TC class add dev $DUMMY parent 8: classid 8:1 htb rate 10mbit" + ], + "cmdUnderTest": "$TC qdisc add dev $DUMMY parent 8:1 handle 9: htb default 1", + "expExitCode": "2", + "verifyCmd": "$TC qdisc show dev $DUMMY", + "matchPattern": "qdisc htb 9:", + "matchCount": "0", + "teardown": [ + "$TC qdisc del dev $DUMMY root" + ] + } +]