github-mirror/linux
1
0
Fork 0
mirror of https://github.com/torvalds/linux.git synced 2026-11-07 06:17:02 +01:00
Code Issues Actions Packages Projects Releases Wiki Activity Graph

crypto: zstd - Avoid redundant cstream initialization

Browse Source 
zstd_compress() initializes the shared workspace as a CStream before
entering the walk loop. If the first source and destination fragments
each span the whole request it then hands off to zstd_compress_one(),
which initializes that same ctx->wksp as a CCtx, discarding the CStream
setup without having compressed a byte. zswap always takes this one-shot
path when storing, so every page it stores paid for both.

Neither is cheap: zstd_init_cstream() redoes the cwksp layout, zeroes the
ZSTD_CCtx, probes for BMI2 through ZSTD_cpuid(), then resets the session
and parameters and replays ten validated ZSTD_CCtx_setParameter() calls.

Defer the CStream initialization to the first walk iteration that needs
it, guarded by a flag because that iteration can be reached more than
once. The first inner iteration either takes the one-shot path and
returns or initializes the CStream, so the trailing zstd_end_stream()
cannot pick up the stale context left in ctx->cctx by an earlier request.
Unlike the old call site the new one runs with the walk's fragments
mapped, so it has to release them before failing.

For a 4 KB crypto_acomp benchmark for compression, twelve runs of nine
30K operation rounds, on the bare-metal host the median per-round mean
request time fell from 52,283 ns to 51,038 ns (2.4%). In the one-vCPU
KVM guest it fell from 16,675 ns to 15,050 ns (9.8%).
The larger improvement in guest is because of the pair of CPUID
instructions in ZSTD_cpuid() that the removed initialization runs.

Signed-off-by: Usama Arif <usama.arif@linux.dev>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This commit is contained in:
Usama Arif 2026-08-25 15:06:01 -07:00 committed by Herbert Xu
parent ac50823889
commit 0db478c2e2
1 changed files with 14 additions and 6 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

20
crypto/zstd.c
View File

@ -96,6 +96,7 @@ static int zstd_compress_one(struct acomp_req *req, struct zstd_ctx *ctx,
static int zstd_compress(struct acomp_req *req)
{
bool stream_initialized = false;
struct crypto_acomp_stream *s;
unsigned int pos, scur, dcur;
unsigned int total_out = 0;
@ -115,12 +116,6 @@ static int zstd_compress(struct acomp_req *req)
if (ret)
goto out;
ctx->cctx = zstd_init_cstream(&ctx->params, 0, ctx->wksp, ctx->wksp_size);
if (!ctx->cctx) {
ret = -EINVAL;
goto out;
}
do {
dcur = acomp_walk_next_dst(&walk);
if (!dcur) {
@ -142,6 +137,19 @@ static int zstd_compress(struct acomp_req *req)
goto out;
}
if (!stream_initialized) {
ctx->cctx = zstd_init_cstream(&ctx->params, 0,
ctx->wksp, ctx->wksp_size);
if (!ctx->cctx) {
/* Release in the reverse of the map order. */
acomp_walk_done_src(&walk, 0);
acomp_walk_done_dst(&walk, 0);
ret = -EINVAL;
goto out;
}
stream_initialized = true;
}
if (scur) {
inbuf.pos = 0;
inbuf.src = walk.src.virt.addr;