linux/drivers
Sergey Senozhatsky bd175a1d84 zram: sleepable entry locking
Patch series "zsmalloc/zram: there be preemption", v10.

Currently zram runs compression and decompression in non-preemptible
sections, e.g.

    zcomp_stream_get()     // grabs CPU local lock
    zcomp_compress()

or

    zram_slot_lock()       // grabs entry spin-lock
    zcomp_stream_get()     // grabs CPU local lock
    zs_map_object()        // grabs rwlock and CPU local lock
    zcomp_decompress()

Potentially a little troublesome for a number of reasons.

For instance, this makes it impossible to use async compression algorithms
or/and H/W compression algorithms, which can wait for OP completion or
resource availability.  This also restricts what compression algorithms
can do internally, for example, zstd can allocate internal state memory
for C/D dictionaries:

do_fsync()
 do_writepages()
  zram_bio_write()
   zram_write_page()                          // become non-preemptible
    zcomp_compress()
     zstd_compress()
      ZSTD_compress_usingCDict()
       ZSTD_compressBegin_usingCDict_internal()
        ZSTD_resetCCtx_usingCDict()
         ZSTD_resetCCtx_internal()
          zstd_custom_alloc()                 // memory allocation

Not to mention that the system can be configured to maximize compression
ratio at a cost of CPU/HW time (e.g.  lz4hc or deflate with very high
compression level) so zram can stay in non-preemptible section (even under
spin-lock or/and rwlock) for an extended period of time.  Aside from
compression algorithms, this also restricts what zram can do.  One
particular example is zram_write_page() zsmalloc handle allocation, which
has an optimistic allocation (disallowing direct reclaim) and a
pessimistic fallback path, which then forces zram to compress the page one
more time.

This series changes zram to not directly impose atomicity restrictions on
compression algorithms (and on itself), which makes zram write() fully
preemptible; zram read(), sadly, is not always preemptible yet.  There are
still indirect atomicity restrictions imposed by zsmalloc().  One notable
example is object mapping API, which returns with: a) local CPU lock held
b) zspage rwlock held

First, zsmalloc's zspage lock is converted from rwlock to a special type
of RW-lookalike look with some extra guarantees/features.  Second, a new
handle mapping is introduced which doesn't use per-CPU buffers (and hence
no local CPU lock), does fewer memcpy() calls, but requires users to
provide a pointer to temp buffer for object copy-in (when needed).  Third,
zram is converted to the new zsmalloc mapping API and thus zram read()
becomes preemptible.


This patch (of 19):

Concurrent modifications of meta table entries is now handled by per-entry
spin-lock.  This has a number of shortcomings.

First, this imposes atomic requirements on compression backends.  zram can
call both zcomp_compress() and zcomp_decompress() under entry spin-lock,
which implies that we can use only compression algorithms that don't
schedule/sleep/wait during compression and decompression.  This, for
instance, makes it impossible to use some of the ASYNC compression
algorithms (H/W compression, etc.) implementations.

Second, this can potentially trigger watchdogs.  For example, entry
re-compression with secondary algorithms is performed under entry
spin-lock.  Given that we chain secondary compression algorithms and that
some of them can be configured for best compression ratio (and worst
compression speed) zram can stay under spin-lock for quite some time.

Having a per-entry mutex (or, for instance, a rw-semaphore) significantly
increases sizeof() of each entry and hence the meta table.  Therefore
entry locking returns back to bit locking, as before, however, this time
also preempt-rt friendly, because if waits-on-bit instead of
spinning-on-bit.  Lock owners are also now permitted to schedule, which is
a first step on the path of making zram non-atomic.

Link: https://lkml.kernel.org/r/20250303022425.285971-1-senozhatsky@chromium.org
Link: https://lkml.kernel.org/r/20250303022425.285971-2-senozhatsky@chromium.org
Signed-off-by: Sergey Senozhatsky <senozhatsky@chromium.org>
Cc: Hillf Danton <hdanton@sina.com>
Cc: Kairui Song <ryncsn@gmail.com>
Cc: Minchan Kim <minchan@kernel.org>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Yosry Ahmed <yosry.ahmed@linux.dev>
Signed-off-by: Andrew Morton <akpm@linux-foundation.org>
2025-03-16 22:06:33 -07:00
..
accel An reset signal polarity fix for the jd9365da-h3 panel, a folio handling 2025-02-21 09:16:35 +10:00
accessibility
acpi ACPI: platform_profile: Add support for hidden choices 2025-03-04 20:45:34 +01:00
amba
android binderfs: fix use-after-free in binder_devices 2025-02-20 15:20:11 +01:00
ata ata fixes for 6.14-rc5 2025-03-01 08:59:29 -08:00
atm
auxdisplay auxdisplay for v6.14-1 2025-01-24 08:03:52 -08:00
base drivers/base/memory: simplify outputting of valid_zones_show() 2025-03-16 22:05:56 -07:00
bcma
block zram: sleepable entry locking 2025-03-16 22:06:33 -07:00
bluetooth bluetooth: btusb: Initialize .owner field of force_poll_sync_fops 2025-02-27 16:50:05 -05:00
bus bus: simple-pm-bus: fix forced runtime PM use 2025-02-20 15:22:02 +01:00
cache
cdrom treewide: const qualify ctl_tables where applicable 2025-01-28 13:48:37 +01:00
cdx cdx: Fix possible UAF error in driver_override_show() 2025-02-20 15:19:07 +01:00
char Char/Misc/IIO driver fixes for 6.14-rc6 2025-03-09 09:07:54 -10:00
clk The various patchsets are summarized below. Plus of course many 2025-01-26 18:36:23 -08:00
clocksource irqchip/jcore-aic, clocksource/drivers/jcore: Fix jcore-pit interrupt request 2025-02-17 23:27:49 +01:00
comedi
connector
counter
cpufreq amd-pstate fixes 2/6/25 2025-02-06 20:39:43 +01:00
cpuidle More power management updates for 6.14-rc1 2025-01-30 15:10:34 -08:00
crypto crypto: ccp: Add external API interface for PSP module initialization 2025-02-14 18:39:19 -05:00
cxl cxl changes for v6.14 2025-01-29 11:23:22 -08:00
dax
dca
devfreq
dio
dma dmaengine fixes for v6.14 2025-03-02 10:08:45 -08:00
dma-buf
dpll
edac EDAC/qcom: Correct interrupt enable register configuration 2025-02-14 20:36:11 +01:00
eisa
extcon
firewire Driver core and debugfs updates 2025-01-28 12:25:12 -08:00
firmware EFI fixes for v6.14 #2 2025-02-28 08:47:21 -08:00
fpga
fsi
gnss
gpio gpio: rcar: Fix missing of_node_put() call 2025-03-06 15:51:27 +01:00
gpu mm/rmap: convert make_device_exclusive_range() to make_device_exclusive() 2025-03-16 22:05:57 -07:00
greybus
hid hid-for-linus-2025030501 2025-03-05 07:46:59 -10:00
hsi
hte
hv treewide: const qualify ctl_tables where applicable 2025-01-28 13:48:37 +01:00
hwmon hwmon: fix a NULL vs IS_ERR_OR_NULL() check in xgene_hwmon_probe() 2025-03-03 06:04:34 -08:00
hwspinlock
hwtracing intel_th: pci: Add Panther Lake-P/U support 2025-02-20 09:35:57 +01:00
i2c i2c: amd-asf: Fix EOI register write to enable successive interrupts 2025-02-26 23:28:41 +01:00
i3c I3C for 6.14 2025-01-24 15:48:01 -08:00
idle intel_idle: Handle older CPUs, which stop the TSC in deeper C states, correctly 2025-02-28 22:04:26 +01:00
iio iio: filter: admv8818: Force initialization of SDO 2025-02-08 12:46:32 +00:00
infiniband RDMA/bnxt_re: Fix the page details for the srq created by kernel consumers 2025-02-23 06:57:56 -05:00
input platform-drivers-x86 for v6.14-1 2025-01-24 07:18:39 -08:00
interconnect
iommu iommu/vt-d: Fix suspicious RCU usage 2025-02-28 12:19:01 +01:00
ipack
irqchip irqchip/qcom-pdc: Workaround hardware register bug on X1E80100 2025-02-21 09:47:06 +01:00
isdn
leds Driver core and debugfs updates 2025-01-28 12:25:12 -08:00
macintosh The various patchsets are summarized below. Plus of course many 2025-01-26 18:36:23 -08:00
mailbox
mcb
md - dm-vdo: add missing spin_lock_init 2025-02-24 16:29:48 -08:00
media [GIT PULL for v6.14] media updates 2025-02-01 09:15:01 -08:00
memory
memstick Char/Misc/IIO driver updates for 6.14-rc1 2025-01-27 16:51:51 -08:00
message
mfd mfd: syscon: Restore device_node_to_regmap() for non-syscon nodes 2025-02-11 14:53:39 +00:00
misc Revert "drivers/card_reader/rtsx_usb: Restore interrupt based detection" 2025-02-27 12:24:53 -08:00
mmc mmc: mtk-sd: Fix register settings for hs400(es) mode 2025-02-03 13:34:50 +01:00
most
mtd Fix writes on SST flashes 2025-02-19 14:38:47 +01:00
mux
net mctp i3c: handle NULL header address 2025-03-06 10:33:07 +01:00
nfc
ntb
nubus
nvdimm
nvme nvme-tcp: fix signedness bug in nvme_tcp_init_connection() 2025-03-05 10:37:01 -08:00
nvmem
of Revert "of: reserved-memory: Fix using wrong number of cells to get property 'alignment'" 2025-02-26 13:39:28 -06:00
opp Driver core and debugfs updates 2025-01-28 12:25:12 -08:00
parisc
parport
pci pci-v6.14-fixes-3 2025-02-14 16:49:07 -08:00
pcmcia
peci
perf treewide: const qualify ctl_tables where applicable 2025-01-28 13:48:37 +01:00
phy phy: tegra: xusb: reset VBUS & ID OVERRIDE 2025-02-14 18:03:05 +05:30
pinctrl pinctrl: pinconf-generic: Print unsigned value if a format is registered 2025-02-06 10:13:15 +01:00
platform ACPI fix for 6.14-rc6 2025-03-07 12:17:42 -10:00
pmdomain pmdomain: airoha: Fix compilation error with Clang-20 and Thumb2 mode 2025-01-21 10:45:24 +01:00
pnp
power power: supply: axp20x_battery: Fix fault handling for AXP717 2025-02-03 12:41:18 +01:00
powercap Merge branch 'pm-powercap' 2025-02-07 12:43:58 +01:00
pps
ps3
ptp ptp: vmclock: Remove goto-based cleanup logic 2025-02-11 10:20:52 +01:00
pwm Driver core and debugfs updates 2025-01-28 12:25:12 -08:00
rapidio rapidio: add check for rio_add_net() in rio_scan_alloc_net() 2025-03-05 21:36:19 -08:00
ras
regulator regulator: core: let dt properties override driver init_data 2025-02-11 16:29:01 +00:00
remoteproc
reset soc: driver updates for 6.14 2025-01-24 14:56:59 -08:00
rpmsg
rtc RTC for 6.13 2025-01-30 17:50:02 -08:00
s390 Smaller than usual with no fixes from any subtree. 2025-02-20 10:19:54 -08:00
sbus
scsi scsi: core: Clear driver private data when retrying request 2025-02-20 21:20:58 -05:00
sh
siox
slimbus slimbus: messaging: Free transaction ID in delayed interrupt scenario 2025-02-20 15:19:51 +01:00
soc soc: loongson: loongson2_guts: Add check for devm_kstrdup() 2025-02-20 22:29:05 +01:00
soundwire soundwire updates for 6.14 2025-01-29 14:38:19 -08:00
spi spi: sn-f-ospi: Fix division by zero 2025-02-06 11:33:51 +00:00
spmi
ssb
staging Driver core and debugfs updates 2025-01-28 12:25:12 -08:00
target Merge branch '6.14/scsi-queue' into 6.14/scsi-fixes 2025-02-03 16:28:51 -05:00
tc
tee tee: optee: Fix supplicant wait loop 2025-02-14 15:17:34 +01:00
thermal thermal: gov_power_allocator: Update total_weight on bind and cdev updates 2025-02-25 12:30:45 +01:00
thunderbolt Driver core and debugfs updates 2025-01-28 12:25:12 -08:00
tty Serial driver fixes for 6.14-rc3 2025-02-16 12:50:44 -08:00
ufs scsi: ufs: core: bsg: Fix crash when arpmb command fails 2025-02-20 22:18:24 -05:00
uio Char/Misc/IIO driver updates for 6.14-rc1 2025-01-27 16:51:51 -08:00
usb usb: typec: ucsi: Fix NULL pointer access 2025-03-06 16:55:46 +01:00
vdpa virtio: features, fixes, cleanups 2025-01-27 15:26:06 -08:00
vfio VFIO updates for v6.14-rc1 2025-01-28 14:16:46 -08:00
vhost vhost: return task creation error instead of NULL 2025-03-01 02:52:52 -05:00
video fb_defio: do not use deprecated page->mapping, index fields 2025-03-16 22:06:11 -07:00
virt Char/Misc/IIO driver fixes for 6.14-rc6 2025-03-09 09:07:54 -10:00
virtio virtio: features, fixes, cleanups 2025-01-27 15:26:06 -08:00
w1
watchdog linux-watchdog 6.14-rc1 tag 2025-01-25 16:19:10 -08:00
xen xen: branch for v6.14-rc3 2025-02-14 08:15:17 -08:00
zorro
Kconfig
Makefile