linux/lib
Gabriel Krisman Bertazi ff9571a4de sbitmap: Try each queue to wake up at least one waiter
commit 26edb30dd1 upstream.

Jan reported the new algorithm as merged might be problematic if the
queue being awaken becomes empty between the waitqueue_active inside
sbq_wake_ptr check and the wake up.  If that happens, wake_up_nr will
not wake up any waiter and we loose too many wake ups.  In order to
guarantee progress, we need to wake up at least one waiter here, if
there are any.  This now requires trying to wake up from every queue.

Instead of walking through all the queues with sbq_wake_ptr, this call
moves the wake up inside that function.  In a previous version of the
patch, I found that updating wake_index several times when walking
through queues had a measurable overhead.  This ensures we only update
it once, at the end.

Fixes: 4f8126bb23 ("sbitmap: Use single per-bitmap counting to wake up queued tags")
Reported-by: Jan Kara <jack@suse.cz>
Signed-off-by: Gabriel Krisman Bertazi <krisman@suse.de>
Reviewed-by: Jan Kara <jack@suse.cz>
Link: https://lore.kernel.org/r/20221115224553.23594-4-krisman@suse.de
Signed-off-by: Jens Axboe <axboe@kernel.dk>
Signed-off-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
2023-03-10 09:34:34 +01:00
..
842
crypto
dim
fonts
kunit kunit: alloc_string_stream_fragment error handling bug fix 2023-01-12 12:02:41 +01:00
livepatch
lz4
lzo
math
mpi lib/mpi: Fix buffer overrun when SG is too long 2023-03-10 09:32:52 +01:00
pldmfw
raid6
reed_solomon
test_fortify
vdso
xz
zlib_deflate
zlib_dfltcc
zlib_inflate
zstd
.gitignore
argv_split.c
ashldi3.c
ashrdi3.c
asn1_decoder.c
asn1_encoder.c
assoc_array.c
atomic64_test.c
atomic64.c
audit.c
base64.c
bcd.c
bch.c
bitfield_kunit.c
bitmap.c
bitrev.c
bootconfig-data.S
bootconfig.c
bsearch.c
btree.c
bucket_locks.c
bug.c cpuidle: lib/bug: Disable rcu_is_watching() during WARN/BUG 2023-03-10 09:33:47 +01:00
build_OID_registry
buildid.c
bust_spinlocks.c
check_signature.c
checksum.c
clz_ctz.c
clz_tab.c
cmdline_kunit.c
cmdline.c
cmpdi2.c
compat_audit.c
cpu_rmap.c
cpumask_kunit.c
cpumask.c
crc-ccitt.c
crc-itu-t.c
crc-t10dif.c
crc4.c
crc7.c
crc8.c
crc16.c
crc32.c
crc32defs.h
crc32test.c
crc64-rocksoft.c
crc64.c
ctype.c
debug_info.c
debug_locks.c
debugobjects.c
dec_and_lock.c
decompress_bunzip2.c
decompress_inflate.c
decompress_unlz4.c
decompress_unlzma.c
decompress_unlzo.c
decompress_unxz.c
decompress_unzstd.c
decompress.c
devmem_is_allowed.c
devres.c
digsig.c
dump_stack.c
dynamic_debug.c
dynamic_queue_limits.c
earlycpio.c
errname.c printf: fix errname.c list 2023-03-10 09:33:27 +01:00
error-inject.c
errseq.c
extable.c
fault-inject-usercopy.c
fault-inject.c
fdt_addresses.c
fdt_empty_tree.c
fdt_ro.c
fdt_rw.c
fdt_strerror.c
fdt_sw.c
fdt_wip.c
fdt.c
find_bit_benchmark.c
find_bit.c
flex_proportions.c
fortify_kunit.c
gen_crc32table.c
gen_crc64table.c
genalloc.c
generic-radix-tree.c
glob.c
globtest.c
hexdump.c
hweight.c
idr.c
inflate.c
interval_tree_test.c
interval_tree.c
iomap_copy.c
iomap.c
iommu-helper.c
iov_iter.c
irq_poll.c
irq_regs.c
is_signed_type_kunit.c
is_single_threaded.c
kasprintf.c
Kconfig
Kconfig.debug
Kconfig.kasan
Kconfig.kcsan
Kconfig.kfence
Kconfig.kgdb
Kconfig.kmsan
Kconfig.ubsan
kfifo.c
klist.c
kobject_uevent.c
kobject.c kobject: Fix slab-out-of-bounds in fill_kobj_path() 2023-03-10 09:33:30 +01:00
kstrtox.c
kstrtox.h
libcrc32c.c
linear_ranges.c
list_debug.c
list_sort.c
list-test.c
llist.c
locking-selftest-hardirq.h
locking-selftest-mutex.h
locking-selftest-rlock-hardirq.h
locking-selftest-rlock-softirq.h
locking-selftest-rlock.h
locking-selftest-rsem.h
locking-selftest-rtmutex.h
locking-selftest-softirq.h
locking-selftest-spin-hardirq.h
locking-selftest-spin-softirq.h
locking-selftest-spin.h
locking-selftest-wlock-hardirq.h
locking-selftest-wlock-softirq.h
locking-selftest-wlock.h
locking-selftest-wsem.h
locking-selftest.c
lockref.c lockref: stop doing cpu_relax in the cmpxchg loop 2023-02-01 08:34:34 +01:00
logic_iomem.c
logic_pio.c
lru_cache.c
lshrdi3.c
Makefile
maple_tree.c maple_tree: fix mas_empty_area_rev() lower bound validation 2023-02-09 11:28:23 +01:00
memcat_p.c
memcpy_kunit.c
memory-notifier-error-inject.c
memregion.c
memweight.c
muldi3.c
net_utils.c
netdev-notifier-error-inject.c
nlattr.c netlink: prevent potential spectre v1 gadgets 2023-02-01 08:34:43 +01:00
nmi_backtrace.c
notifier-error-inject.c
notifier-error-inject.h
objagg.c
of-reconfig-notifier-error-inject.c
oid_registry.c
once.c
overflow_kunit.c
packing.c
parman.c
parser.c
pci_iomap.c
percpu_counter.c
percpu_test.c
percpu-refcount.c
plist.c
pm-notifier-error-inject.c
polynomial.c
radix-tree.c
random32.c
ratelimit.c
rbtree_test.c
rbtree.c
ref_tracker.c
refcount.c
rhashtable.c
sbitmap.c sbitmap: Try each queue to wake up at least one waiter 2023-03-10 09:34:34 +01:00
scatterlist.c
seq_buf.c
sg_pool.c
sg_split.c
show_mem.c
siphash.c
slub_kunit.c
smp_processor_id.c
sort.c
stackdepot.c
stackinit_kunit.c
stmp_device.c
string_helpers.c
string.c
strncpy_from_user.c
strnlen_user.c
syscall.c
test_bitmap.c
test_bitops.c
test_bits.c
test_blackhole_dev.c
test_bpf.c
test_debug_virtual.c
test_dynamic_debug.c
test_firmware.c
test_fprobe.c
test_fpu.c
test_free_pages.c
test_hash.c
test_hexdump.c
test_hmm_uapi.h
test_hmm.c
test_ida.c
test_kmod.c
test_kprobes.c
test_linear_ranges.c
test_list_sort.c
test_lockup.c
test_maple_tree.c maple_tree: fix mas_empty_area_rev() lower bound validation 2023-02-09 11:28:23 +01:00
test_memcat_p.c
test_meminit.c
test_min_heap.c
test_module.c
test_objagg.c
test_parman.c
test_printf.c
test_ref_tracker.c
test_rhashtable.c
test_scanf.c
test_siphash.c
test_sort.c
test_static_key_base.c
test_static_keys.c
test_string.c
test_strscpy.c
test_sysctl.c
test_ubsan.c
test_user_copy.c
test_uuid.c
test_vmalloc.c
test_xarray.c
test-kstrtox.c
test-string_helpers.c
textsearch.c
timerqueue.c
trace_readwrite.c
ts_bm.c
ts_fsm.c
ts_kmp.c
ubsan.c panic: Consolidate open-coded panic_on_warn checks 2023-01-24 07:24:41 +01:00
ubsan.h
ucmpdi2.c
ucs2_string.c
usercopy.c uaccess: Add speculation barrier to copy_from_user() 2023-02-25 11:25:41 +01:00
uuid.c
vsprintf.c
win_minmax.c
xarray.c
xxhash.c