mirror of
https://github.com/torvalds/linux.git
synced 2026-10-23 15:32:04 +02:00
c0a44d1745
Add an implementation of xor_gen() using AVX-512.
It uses 512-bit vectors, i.e. ZMM registers. It also uses the
vpternlogq instruction to do three-input XORs when applicable.
It's enabled on x86_64 CPUs that have AVX512F && !PREFER_YMM. In
practice that means:
- AMD Zen 4 and later (client and server)
- Intel Sapphire Rapids and later (server)
- Intel Rocket Lake (client)
- Intel Nova Lake and later (client)
The !PREFER_YMM condition excludes the older AVX-512 implementations in
Intel Skylake Server and Intel Ice Lake. They could run this code, but
they're known to have overly-eager downclocking when ZMM registers are
used. This is the same policy that the crypto and CRC code uses.
Results from xor_benchmark on AMD Ryzen 9 9950X (Zen 5):
Before:
# xor_benchmark: 4096 bytes 16384 bytes
# xor_benchmark: 4 disks: 256 GB/s 185 GB/s
# xor_benchmark: 5 disks: 233 GB/s 165 GB/s
# xor_benchmark: 6 disks: 243 GB/s 174 GB/s
# xor_benchmark: 7 disks: 244 GB/s 176 GB/s
# xor_benchmark: 8 disks: 265 GB/s 179 GB/s
# xor_benchmark: 10 disks: 249 GB/s 175 GB/s
# xor_benchmark: 12 disks: 219 GB/s 173 GB/s
# xor_benchmark: 15 disks: 206 GB/s 177 GB/s
# xor_benchmark: 16 disks: 212 GB/s 180 GB/s
# xor_benchmark: 32 disks: 193 GB/s 177 GB/s
After:
# xor_benchmark: 4096 bytes 16384 bytes
# xor_benchmark: 4 disks: 492 GB/s 202 GB/s
# xor_benchmark: 5 disks: 436 GB/s 191 GB/s
# xor_benchmark: 6 disks: 466 GB/s 193 GB/s
# xor_benchmark: 7 disks: 491 GB/s 194 GB/s
# xor_benchmark: 8 disks: 513 GB/s 205 GB/s
# xor_benchmark: 10 disks: 454 GB/s 199 GB/s
# xor_benchmark: 12 disks: 330 GB/s 204 GB/s
# xor_benchmark: 15 disks: 266 GB/s 198 GB/s
# xor_benchmark: 16 disks: 243 GB/s 177 GB/s
# xor_benchmark: 32 disks: 187 GB/s 165 GB/s
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Borislav Petkov (AMD) <bp@alien8.de>
Reviewed-by: David Laight <david.laight.linux@gmail.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Acked-by: Christoph Hellwig <hch@lst.de>
Link: https://patch.msgid.link/20260901145740.42337-9-ebiggers@kernel.org
|
||
|---|---|---|
| .. | ||
| raid6 | ||
| xor | ||
| .kunitconfig | ||
| Kconfig | ||
| Makefile | ||