drbg_kcapi_seed() calls drbg_uninstantiate() only to free drbg->jent and
set drbg->instantiated = false. However, the latter is necessary only
because drbg_kcapi_seed() sets drbg->instantiated = true too early. Fix
that, then just inline the freeing of drbg->jent.
Then, simplify the actual "uninstantiate" in drbg_kcapi_exit(). Just
free drbg->jent (note that this is a no-op on error and null pointers),
then memzero_explicit() the entire drbg_state.
Note that in reality the memzero_explicit() is redundant, since the
crypto_rng API zeroizes the memory anyway. But the way SP800-90A is
worded, it's easy to imagine that someone assessing conformance with it
would be looking for code in drbg.c that says it does an "Uninstantiate"
and does the zeroization. So it's probably worth keeping it somewhat
explicit, even though that means double zeroization in practice.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Fold drbg_prepare_hrng() into its only caller.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Clearly separate the code for the "reseed" and "instantiate" cases,
since what they need to do is quite different.
Note that the mutex guard makes the mutex start being held during the
call to drbg_uninstantiate(), which is fine.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Fold drbg_instantiate() into its only caller.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Put the DRBG implementation of the rng_alg methods in the order in which
they're called (cra_init => set_ent => seed => generate => cra_exit) so
that it's easier to understand the flow.
Also rename drbg_kcapi_random to drbg_kcapi_generate, and
drbg_kcapi_cleanup to drbg_kcapi_exit, so they match the method names.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Simplify drbg_generate_long() to use a more straightforward loop, and
then fold it into its only caller. No functional change.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Use straightforward (buffer, len) parameters instead of struct
drbg_string or lists of strings. This simplifies the code considerably.
For now struct drbg_string is still used in crypto_drbg_ctr_df(), so
move its definition to crypto/df_sp80090a.h.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Currently some of the steps of "Instantiate the DRBG" occur in a
convoluted way in different places in the call stack:
drbg_instantiate()
=> drbg_seed(reseed=0) sets the raw HMAC key drbg_state::C to its
correct initial value, and it sets the state value
drbg_state::V to an *incorrect* initial value.
=> drbg_hmac_update(reseed=0) overwrites drbg_state::V with the
correct initial value, then prepares the hmac_sha512_key
drbg_state::key from the initial raw HMAC key drbg_state::C.
Later, each time the HMAC key is updated, drbg_hmac_update() also uses
drbg_state::C to temporarily store the new raw key.
Simplify all of this by:
- Making drbg_instantiate() set the correct initial values of
drbg_state::V and drbg_state::key.
- Converting drbg_hmac_update() to generate the raw key in a
temporary on-stack array instead of drbg_state::C.
- Removing drbg_state::C.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Move the MODULE_ALIAS_CRYPTO lines in the middle of the file to the end
so that they are in the usual place and next to the other one.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Set rng_alg::seed to different functions for the prediction-resistant
and non-prediction-resistant algorithms, so that the function does not
need to parse the algorithm name to figure out which algorithm it is.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Now that none of the information in struct drbg_core is used, remove it.
The null-ity of the pointer drbg_state::core was used to keep track of
whether the DRBG has been instantiated. Replace it with a boolean.
No functional change.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Since the HMAC algorithm is now fixed at HMAC-SHA512, just use the
HMAC-SHA512 library API. This is simpler and more efficient.
Remove error-handling code that is no longer needed.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Now that the sizes of V and C are known at compile time, embed them into
struct drbg_state rather than using separate allocations.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Since only one drbg_core remains, the state length, block length, and
security strength are now fixed values. Moreover, the maximum request
length, maximum additional data length, and maximum number of requests
were all already fixed values.
Simplify the code by just using #defines for all these fixed values.
In drbg_seed_from_random(), take advantage of the constant to define the
array size. Remove assertions that are no longer useful.
In the case of drbg_blocklen() and drbg_statelen(), replace these with a
single value DRBG_STATE_LEN, as for HMAC_DRBG they are the same thing.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Now that there's only one set of state operations, use direct calls to
those operations.
No change in behavior. In particular, drbg_alloc_state() doesn't change
behavior, because the only remaining drbg_core uses HMAC_DRBG.
drbg_uninstantiate() doesn't change behavior, because a NULL d_ops
implied NULL priv_data which makes a drbg_fini_hash_kernel() a no-op.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Now that "drbg_pr_hmac_sha512" and "drbg_nopr_hmac_sha512" are the only
crypto_rng algorithms left in crypto/drbg.c, simplify the algorithm
registration logic to register these more directly without relying on
the drbg_cores[] array (which will be removed).
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Remove support for the HMAC-SHA256 and HMAC-SHA384 variants of
HMAC_DRBG, leaving only the HMAC-SHA512 variant of HMAC_DRBG.
HMAC-SHA512 is already the default. The default did used to be
HMAC-SHA256, but several years ago it was upgraded to HMAC-SHA512 "to
support compliance with SP800-90B and SP800-90C". Given that the point
of crypto/drbg.c is compliance with those standards, and there's also no
technical reason to prefer HMAC-SHA384 in this situation even if
acceptable, there's really no point in offering anything else.
Note: now that only HMAC-SHA512 remains, a lot of unnecessary
abstractions can be removed. A later commit will do that. This commit
just straightforwardly removes the HMAC-SHA256 and HMAC-SHA384 code.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Synchronize the drbg_nopr_hmac_sha512 test vector with the first test
vector from the latest ACVP json files, so that both of the DRBG test
vectors are pulled from a consistent source.
Note that the new test vector has a nonempty personalization string.
That should be helpful as well: Some FIPS labs require this, due to
their interpretation of SP800-90A 11.3.2 which says that a
"representative" value of the personalization string must be tested.
It also now does an explicit reseed, which makes it clearer that the
requirement to test "Reseed" is met, without having to interpret the
additional input processing as covering that.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Add a test for drbg_pr_hmac_sha512, which previously didn't have a test.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Now that the menuconfig CRYPTO_DRBG_MENU has no options in it other than
the hidden symbol CRYPTO_DRBG, remove it and move CRYPTO_DRBG to its
parent menu. Give CRYPTO_DRBG an appropriate prompt and help text.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Remove the support for HASH_DRBG. It's likely unused code, seeing as
HMAC_DRBG is always enabled and prioritized over it unless
NETLINK_CRYPTO is used to change the algorithm priorities.
There's also no compelling reason to support more than one of
[HMAC_DRBG, HASH_DRBG, CTR_DRBG]. By definition, callers cannot tell
any difference in their outputs. And all are FIPS-certifiable, which is
the only point of the kernel's NIST DRBGs anyway.
Switching to HASH_DRBG doesn't seem all that compelling, either. For
one, it's more complex than HMAC_DRBG.
Thus, let's just drop HASH_DRBG support and focus on HMAC_DRBG.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Acked-by: Geert Uytterhoeven <geert@linux-m68k.org> # m68k
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Remove the support for CTR_DRBG. It's likely unused code, seeing as
HMAC_DRBG is always enabled and prioritized over it unless
NETLINK_CRYPTO is used to change the algorithm priorities.
There's also no compelling reason to support more than one of
[HMAC_DRBG, HASH_DRBG, CTR_DRBG]. By definition, callers cannot tell
any difference in their outputs. And all are FIPS-certifiable, which is
the only point of the kernel's NIST DRBGs anyway.
Switching to CTR_DRBG doesn't seem all that compelling, either. While
it's often the fastest NIST DRBG, it has several disadvantages:
- CTR_DRBG uses AES. Some platforms don't have AES acceleration at all,
causing a fallback to the table-based AES code which is very slow and
can be vulnerable to cache-timing attacks. In contrast, HMAC_DRBG
uses primitives that are consistently constant-time.
- CTR_DRBG is usually considered to be somewhat less cryptographically
robust than HMAC_DRBG. Granted, HMAC_DRBG isn't all that great
either, e.g. given the negative result from Woodage & Shumow (2018)
(https://eprint.iacr.org/2018/349.pdf), but that can be worked around.
- CTR_DRBG is more complex than HMAC_DRBG, risking bugs. Indeed, while
reviewing the CTR_DRBG code, I found two bugs, including one where it
can return success while leaving the output buffer uninitialized.
- The kernel's implementation of CTR_DRBG uses an "ctr(aes)"
crypto_skcipher and relies on it returning the next counter value.
That's fragile, and indeed historically many "ctr(aes)"
crypto_skcipher implementations haven't done that. E.g. see
commit 511306b2d0 ("crypto: arm/aes-ce - update IV after partial final CTR block"),
commit fa5fd3afc7 ("crypto: arm64/aes-blk - update IV after partial final CTR block"),
commit 371731ec21 ("crypto: atmel-aes - Fix saving of IV for CTR mode"),
commit 25baaf8e2c ("crypto: crypto4xx - fix ctr-aes missing output IV"),
commit 334d37c9e2 ("crypto: caam - update IV using HW support"),
commit 0a4491d3fe ("crypto: chelsio - count incomplete block in IV"),
commit e8e3c1ca57 ("crypto: s5p - update iv after AES-CBC op end").
I.e., there were many years where the kernel's CTR_DRBG code (if it
were to have actually been used) repeated outputs on some platforms.
AES-CTR also uses a 128-bit counter, which creates overflow edge cases
that are sometimes gotten wrong. E.g. see commit 009b30ac74
("crypto: vmx - CTR: always increment IV as quadword").
So, while switching to CTR_DRBG for performance reasons isn't completely
out of the question (notably BoringSSL uses it), it would take quite a
bit more work to create a solid implementation of it in the kernel,
including a more solid implementation of AES-CTR itself (in lib/crypto/,
with a scalar bit-sliced fallback, etc). Since HMAC_DRBG has always
been the default NIST DRBG variant in the kernel and is in a better
state, let's just standardize on it for now.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Acked-by: Geert Uytterhoeven <geert@linux-m68k.org> # m68k
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The inclusion of <crypto/internal/cipher.h> and the import of the
internal crypto namespace became unnecessary in commit ba0570bdf1
("crypto: drbg - Replace AES cipher calls with library calls").
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
include/crypto/drbg.h no longer contains anything that is used
externally to crypto/drbg.c. Therefore, fold it into crypto/drbg.c.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
FIPS 140-2 required that a continuous test for repeated outputs be done
on both "Approved RNGs" and "Non-Approved RNGs".
That's apparently why crypto/drbg.c does such a test on the bytes it
pulls from get_random_bytes(), despite get_random_bytes() being a
"Non-Approved RNG" that is credited with zero entropy for FIPS purposes.
(From FIPS's point of view, the "Approved RNG" is jitterentropy.)
FIPS 140-3 "modernized" the continuous RNG test requirements. They're
now a bit more sophisticated, requiring both an "Adaptive Proportion
Test" and a "Repetition Count Test".
At the same time, FIPS 140-3 doesn't require continuous RNG tests on
"Non-Approved RNGs" if a "vetted conditioning component" is used. The
SP800-90A DRBGs are exactly such a vetted conditioning component, by
their design. (In the case of HASH_DRBG and CTR_DRBG, the derivation
function does have to be implemented. But the kernel does that.)
In other words: from FIPS 140-3's point of view, get_random_bytes()
still produces zero entropy, but the way the DRBG combines those bytes
with the jitterentropy bytes preserves all the "approved" entropy from
jitterentropy. Thus no test for get_random_bytes() is required.
Seeing as FIPS 140-2 certificates stopped being issued in 2021 in favor
of FIPS 140-3, this means this code is obsolete. Remove it.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Fold the contents of the inline functions crypto_drbg_get_bytes_addtl(),
crypto_drbg_get_bytes_addtl_test(), and crypto_drbg_reset_test() into
their only caller in drbg_cavs_test(). It ends up being much simpler.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
This commented-out code doesn't compile. Even if it did, it wouldn't
actually do what it was apparently intended to do, seeing as the "test"
for "drbg_pr_hmac_sha512" and "drbg_pr_ctr_aes256" is alg_test_null().
Just delete it to avoid keeping broken code around, and so that there
isn't any perceived need to try to update it as the DRBG code is
refactored.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The kconfig symbol CRYPTO_DRBG_HMAC is always enabled when
CRYPTO_DRBG_MENU is enabled, and all checks for CRYPTO_DRBG_HMAC are in
code conditional on CRYPTO_DRBG_MENU. Thus, the only purpose of the
CRYPTO_DRBG_HMAC symbol is to select CRYPTO_HMAC and CRYPTO_SHA512.
Move those two selections to CRYPTO_DRBG_MENU, remove the checks for
CRYPTO_DRBG_HMAC, and remove the CRYPTO_DRBG_HMAC symbol itself.
Note that this also fixes an issue where CRYPTO_HMAC and CRYPTO_SHA512
were unnecessarily being forced to built-in when CRYPTO_DRBG=m.
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
When fips_enabled=1, it seems to have been intended for one of the
algorithms defined in crypto/drbg.c to be the highest priority "stdrng"
algorithm, so that it is what is used by "stdrng" users.
However, the code only boosts the priority to 400, which is less than
the priority 500 used in drivers/crypto/caam/caamprng.c. Thus, the CAAM
RNG could be used instead.
Fix this by boosting the priority by 2000 instead of 200.
Fixes: 541af946fe ("crypto: drbg - SP800-90A Deterministic Random Bit Generator")
Cc: stable@vger.kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
On 64-bit kernels, drbg_max_addtl() returns 2**35 bytes. That's too
large, for two reasons:
1. SP800-90A says the maximum limit is 2**35 *bits*, not 2**35 bytes.
So the implemented limit has confused bits and bytes.
2. When drbg_kcapi_hash() calls crypto_shash_update() on the additional
information string, the length is implicitly cast to 'unsigned int'.
That truncates the additional information string to U32_MAX bytes.
Fix the maximum additional information string length to always be
U32_MAX - 1, causing an error to be returned for any longer lengths.
Fixes: 541af946fe ("crypto: drbg - SP800-90A Deterministic Random Bit Generator")
Cc: stable@vger.kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Fix drbg_healthcheck_sanity() to correctly check the return value of
drbg_generate(). drbg_generate() returns 0 on success, or a negative
errno value on failure. drbg_healthcheck_sanity() incorrectly assumed
that it returned a positive value on success.
This didn't make the sanity check fail, but it made it ineffective.
Fixes: cde001e4c3 ("crypto: rng - RNGs must return 0 in success case")
Cc: stable@vger.kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
drbg_cpu_to_be32() is being used to do a plain write to a byte array,
which doesn't have any alignment guarantee. This can cause a misaligned
write. Replace it with the correct function, put_unaligned_be32().
Fixes: 72f3e00dd6 ("crypto: drbg - replace int2byte with cpu_to_be")
Cc: stable@vger.kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
drbg_ctr_generate() sometimes returns success when it fails, leaving the
output buffer uninitialized. Fix it.
Fixes: cde001e4c3 ("crypto: rng - RNGs must return 0 in success case")
Cc: stable@vger.kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Document the crypto engine on Glymur platform.
Signed-off-by: Harshal Dev <harshal.dev@oss.qualcomm.com>
Reviewed-by: Krzysztof Kozlowski <krzysztof.kozlowski@oss.qualcomm.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Add entry for ATECC608B. Update the ATECC508A comment for consistency.
Acked-by: Rob Herring (Arm) <robh@kernel.org>
Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Tested on hardware with an ATECC608B at 0x60. The device binds
successfully, passes the driver's sanity check, and registers the
ecdh-nist-p256 KPP algorithm.
The hardware ECDH path was also exercised using a minimal KPP test
module, covering private key generation, public key derivation, and
shared secret computation.
Signed-off-by: Thorsten Blum <thorsten.blum@linux.dev>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Sashiko notes:
> is the stack variable data left uninitialized when taking the else branch?
> Since data.tio_en is later evaluated unconditionally, could stack garbage
> cause it to evaluate to true, leading to erroneous attempts to allocate
> pages and initialize SEV-TIO on unsupported hardware?
If the firmware is too old to support SEV_INIT_EX, data is left
uninitialized but used in the debug logging about whether TIO is enabled or
not.
Fixes: 4be423572d ("crypto/ccp: Implement SEV-TIO PCIe IDE (phase1)")
Reported-by: Sashiko
Assisted-by: Gemini:gemini-3.1-pro-preview
Link: https://sashiko.dev/#/patchset/20260324161301.1353976-1-tycho%40kernel.org
Signed-off-by: Tycho Andersen (AMD) <tycho@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Sashiko notes:
> if __snp_alloc_firmware_pages() returns NULL under memory pressure, is it
> safe to pass it directly to page_address()?
>
> On architectures without HASHED_PAGE_VIRTUAL, page_address(NULL) might
> compute a deterministic but invalid, non-zero virtual address. The
> subsequent if (tio_status) check would then evaluate to true, and
> sev_tsm_init_locked() would dereference the invalid pointer.
Indeed, page_address(NULL) will return non-NULL garbage here. Fix this by
checking the page allocation itself for NULL, not the resulting virtual
address.
Fixes: 4be423572d ("crypto/ccp: Implement SEV-TIO PCIe IDE (phase1)")
Reported-by: Sashiko
Assisted-by: Gemini:gemini-3.1-pro-preview
Link: https://sashiko.dev/#/patchset/20260324161301.1353976-1-tycho%40kernel.org
Signed-off-by: Tycho Andersen (AMD) <tycho@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Sashiko notes:
> regarding the bounds check in snp_filter_reserved_mem_regions()
> called via walk_iomem_res_desc(): does the check
> if ((range_list->num_elements * 16 + 8) > PAGE_SIZE)
> allow an off-by-one heap buffer overflow?
>
> If range_list->num_elements is 255, 255 * 16 + 8 = 4088, which is <= 4096.
> Writing range->base (8 bytes) fills 4088-4095, but writing range->page_count
> (4 bytes) would write to 4096-4099, overflowing the kzalloc-allocated
> PAGE_SIZE buffer.
Fix this by accounting for the entry about to be written to, in addition to
the entries that are already allocated.
Fixes: 1ca5614b84 ("crypto: ccp: Add support to initialize the AMD-SP for SEV-SNP")
Reported-by: Sashiko
Assisted-by: Gemini:gemini-3.1-pro-preview
Link: https://sashiko.dev/#/patchset/20260324161301.1353976-1-tycho%40kernel.org
Signed-off-by: Tycho Andersen (AMD) <tycho@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Before SNP x86 shutdown [1], all HV_FIXED pages were always leaked on
module unload. Now pages can be reclaimed if they are freed before SNP
shutdown.
The SFS driver does sfs_dev_destroy() -> snp_free_hv_fixed_pages(), marking
the command buffer as free. But this happens after sev_dev_destroy() in
psp_dev_destroy(), so the pages are always leaked.
Rearrange psp_dev_destroy() to destroy things in the reverse order from
psp_init(), so that any dependencies can be unwound accordingly. This lets
SFS free the page and the subsequent SNP shutdown release it.
This was identified with use of Chris Mason's review-prompts:
https://github.com/masoncl/review-prompts
[1]: https://lore.kernel.org/all/20260324161301.1353976-1-tycho@kernel.org/
Fixes: 648dbccc03 ("crypto: ccp - Add AMD Seamless Firmware Servicing (SFS) driver")
Reported-by: review-prompts
Assisted-by: Claude:claude-4.6-opus
Suggested-by: Tom Lendacky <thomas.lendacky@amd.com>
Signed-off-by: Tycho Andersen (AMD) <tycho@kernel.org>
Reviewed-by: Ashish Kalra <ashish.kalra@amd.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Document the QCE crypto engine on Qualcomm Eliza SoC, fully compatible
with earlier generations.
Signed-off-by: Krzysztof Kozlowski <krzysztof.kozlowski@oss.qualcomm.com>
Reviewed-by: Harshal Dev <harshal.dev@oss.qualcomm.com>
Reviewed-by: Kuldeep Singh <kuldeep.singh@oss.qualcomm.com>
Acked-by: Rob Herring (Arm) <robh@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
The current implementation of the heartbeat error injection uses
adf_disable_arb_thd() to stop a specific accelerator engine thread
from processing requests. This does not reliably prevent the device
from generating responses.
Fix the error injection by disabling the device arbiter through
exit_arb() instead. This properly simulates a device failure by
stopping all arbitration, which results in missing responses for
sent requests.
Remove the now unused adf_disable_arb_thd() function and its
declaration.
Fixes: e2b67859ab ("crypto: qat - add heartbeat error simulator")
Signed-off-by: Damian Muszynski <damian.muszynski@intel.com>
Reviewed-by: Ahsan Atta <ahsan.atta@intel.com>
Reviewed-by: Giovanni Cabiddu <giovanni.cabiddu@intel.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
Document the crypto engine on the Milos platform.
Signed-off-by: Alexander Koskovich <akoskovich@pm.me>
Reviewed-by: Krzysztof Kozlowski <krzysztof.kozlowski@oss.qualcomm.com>
Reviewed-by: Kuldeep Singh <kuldeep.singh@oss.qualcomm.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
There is already an 'if ASYMMETRIC_KEY_TYPE' condition wrapping
FIPS_SIGNATURE_SELFTEST, making the 'depends on' statement a
duplicate dependency (dead code).
I propose leaving the outer 'if ASYMMETRIC_KEY_TYPE...endif' and removing
the individual 'depends on' statement.
This dead code was found by kconfirm, a static analysis tool for Kconfig.
Signed-off-by: Julian Braha <julianbraha@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
rhashtable_insert_rehash() allocates a new bucket table
with GFP_ATOMIC, as it is called from an RCU read-side
critical section.
If rhashtable_rehash_attach() then fails, the new table
is freed via kvfree(). This is unsafe, since kvfree() may
fall back to vfree() for vmalloc-backed allocations, which
can sleep and trigger:
BUG: sleeping function called from invalid context
Add bucket_table_free_atomic(), which uses kvfree_atomic()
so the table can be freed safely from non-sleeping context.
Signed-off-by: Uladzislau Rezki (Sony) <urezki@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
kvmalloc() now supports non-sleeping GFP flags, including
the vmalloc fallback path. This means it may return vmalloc
memory even for GFP_ATOMIC and GFP_NOWAIT allocations.
Freeing such memory with kvfree() may then end up calling
vfree(), which is not safe for non-sleeping contexts.
Introduce kvfree_atomic() helper for such cases. It mirrors
kvfree(), but uses vfree_atomic() for vmalloced memory.
Signed-off-by: Uladzislau Rezki (Sony) <urezki@gmail.com>
Acked-by: Vlastimil Babka (SUSE) <vbabka@kernel.org>
Acked-by: Harry Yoo (Oracle) <harry@kernel.org>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
rhashtable_free_and_destroy() is a single-shot teardown routine:
cancel_work_sync() has already quiesced the deferred rehash worker, and
the function's documented contract requires the caller to guarantee no
other concurrent access to the rhashtable. Under those conditions
ht->mutex is not protecting anything -- taking it is a leftover from
the original teardown path.
That leftover is actively harmful: it closes a circular lock-class
dependency with fs_reclaim. The deferred rehash worker takes ht->mutex
and then allocates GFP_KERNEL memory in bucket_table_alloc(),
establishing
&ht->mutex -> fs_reclaim
After commit b32c4a2136 ("xattr: add rhashtable-based simple_xattr
infrastructure") introduced simple_xattr_ht_free(), which calls
rhashtable_free_and_destroy(), the simple_xattrs teardown became
reachable from evict() under the dcache shrinker. The subsequent
per-subsystem adaptations made the reverse edge concrete in three
independent code paths:
* commit 52b364fed6 ("shmem: adapt to rhashtable-based simple_xattrs with lazy allocation")
* commit 5bd97f5c5f ("kernfs: adapt to rhashtable-based simple_xattrs with lazy allocation")
* commit 50704c391f ("pidfs: adapt to rhashtable-based simple_xattrs")
Any of the three closes the cycle
fs_reclaim -> &ht->mutex
which lockdep reports as follows. This particular splat was observed
organically on a workstation kernel built from vfs-7.1-rc1.xattr at
~35h uptime under normal mixed workload, with CONFIG_PROVE_LOCKING=y.
The path happens to go through kernfs:
WARNING: possible circular locking dependency detected
7.0.0-faeab166167f-with-fixes-v1+ #191 Tainted: G U
kswapd0/243 is trying to acquire lock:
ffff8882e475c0f8 (&ht->mutex){+.+.}-{4:4},
at: rhashtable_free_and_destroy+0x36/0x740
but task is already holding lock:
ffffffffa8ad1d00 (fs_reclaim){+.+.}-{0:0},
at: balance_pgdat+0x995/0x1600
the existing dependency chain (in reverse order) is:
-> #1 (fs_reclaim){+.+.}-{0:0}:
__lock_acquire+0x506/0xbf0
lock_acquire.part.0+0xc7/0x280
fs_reclaim_acquire+0xd9/0x130
__kvmalloc_node_noprof+0xcd/0xb40
bucket_table_alloc.isra.0+0x5a/0x440
rhashtable_rehash_alloc+0x4e/0xd0
rht_deferred_worker+0x14b/0x440
process_one_work+0x8fd/0x16a0
worker_thread+0x601/0xff0
kthread+0x36b/0x470
ret_from_fork+0x5bf/0x910
ret_from_fork_asm+0x1a/0x30
-> #0 (&ht->mutex){+.+.}-{4:4}:
check_prev_add+0xdb/0xce0
validate_chain+0x554/0x780
__lock_acquire+0x506/0xbf0
lock_acquire.part.0+0xc7/0x280
__mutex_lock+0x1b2/0x2550
rhashtable_free_and_destroy+0x36/0x740
kernfs_put.part.0+0x119/0x570
evict+0x3b6/0x9c0
__dentry_kill+0x181/0x540
shrink_dentry_list+0x135/0x440
prune_dcache_sb+0xdb/0x150
super_cache_scan+0x2ff/0x520
do_shrink_slab+0x35a/0xee0
shrink_slab_memcg+0x457/0x950
shrink_slab+0x43b/0x550
shrink_one+0x31a/0x6f0
shrink_many+0x31e/0xc80
shrink_node+0xeb3/0x14a0
balance_pgdat+0x8ed/0x1600
kswapd+0x2f3/0x530
kthread+0x36b/0x470
ret_from_fork+0x5bf/0x910
ret_from_fork_asm+0x1a/0x30
Possible unsafe locking scenario:
CPU0 CPU1
---- ----
lock(fs_reclaim);
lock(&ht->mutex);
lock(fs_reclaim);
lock(&ht->mutex);
Note that lockdep tracks lock classes, not instances: the two
&ht->mutex sites are on different rhashtable objects (the deferred
worker was triggered by some unrelated rhashtable growth), but because
rhashtable_init() uses a single static lockdep key for all rhashtables,
this is a real class-level cycle. Once reported, lockdep disables
itself for the remainder of the boot, masking any subsequent locking
bugs.
Drop the mutex. After cancel_work_sync() the rehash worker is quiesced
and, per this function's contract, no other concurrent access is
possible; the tables are therefore owned exclusively by this function
and can be walked without any lock held.
Switch the table walks from rht_dereference() (which requires
ht->mutex to be held under CONFIG_PROVE_RCU) to rcu_dereference_raw(),
which has no lockdep annotation. rht_ptr_exclusive() already uses
rcu_dereference_protected(p, 1) and needs no change.
This is the only place in lib/rhashtable.c where &ht->mutex is
acquired from a path reachable under fs_reclaim; the deferred worker
is the only other site and it is the forward edge. Removing the
acquisition here therefore eliminates the class cycle for all three
subsystems that use simple_xattrs, not just the one in the splat
above. No locking-semantics change is introduced for correct users;
incorrect users would already be racing with rehash worker completion
regardless of the mutex.
Synthetic reproduction of the splat within a few-minute window was
unsuccessful across several attempts (tmpfs and kernfs zombies via
cgroupfs with open-fd-through-rmdir, with and without swap, up to
~60k reclaim-path executions of simple_xattr_ht_free() in a single
run), consistent with the rare coincidence-of-edges profile of the
bug: the forward edge is already registered in /proc/lockdep on any
idle system via rht_deferred_worker, but the reverse edge requires
evict() to complete kernfs_put()'s final release inside the fs_reclaim
critical section, which in my attempts was ordered against rather than
interleaved with the worker.
Fixes: b32c4a2136 ("xattr: add rhashtable-based simple_xattr infrastructure")
Signed-off-by: Mikhail Gavrilov <mikhail.v.gavrilov@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
- sh: Fix fallout from ZERO_PAGE consolidation
-----BEGIN PGP SIGNATURE-----
iQIzBAABCAAdFiEEYv+KdYTgKVaVRgAGdCY7N/W1+RMFAmn3X0MACgkQdCY7N/W1
+RNzqhAAnKdBTXt7Ks1qYfCA1zrpTUPSLkJeyX/rjfmFHJkSoH9onmUhxOXPc5jp
TT1Jy8YjN8GEPd2Qis5e2FDM+yvF8wMfScQRaSE52WR8bXddQLTjQBK7+nyyPJNu
xJWymOufB7wooD9Qsg4jf7zmTcODMskT/F9i5EQxYA8bSt4/EVMKdhNF8k4goDwT
vpw4TYmWYj/tDndWe5zZNkok4kT6TZDQ/shiaqSSq5N0YpCBkKrsNP19zrxs1iQM
UCu/M5UvWkj6JukBuYQAZE642a7jvpvNCR74/l3DtF4kTTUoSYFozQ0LshXRVI94
46vpGZXRVsxiCjGNPB+XWLY9cJ9j2T8vVBuBfabRZjlllxOT9ygQ4n5pRVAwx2jW
VJP6w1AnCUea0fZ0BP2yOxFee48b4QIP2YufArAMKy2mtkAiwx3iDIxi0u43uZyG
TuyYQ6hKRFDoE157qYRoizMeBWf18FCsPg1hd7cWIu8E2xxTJaUss/qgwZAqLcFU
W2/RiUDLfFgyHCY9CM9go+ZgzbRWGUEItSLPv3AUgBlbFPscByjQzISU0ed6XjQ9
XMNKcIow6lwRkFpWf9CDcRqNn+9ey551tQvCyDsStp8h99V6buyX1nvndupDspZz
+l49H7GVwyIA9eqoTucCMNn5isx+KAsa2KeSjOyup9o0eGYr0Os=
=6ywF
-----END PGP SIGNATURE-----
Merge tag 'sh-for-v7.1-tag2' of git://git.kernel.org/pub/scm/linux/kernel/git/glaubitz/sh-linux
Pull sh fix from John Paul Adrian Glaubitz:
"The ZERO_PAGE consolidation in v7.1, introduced a regression on sh
which made these systems unbootable.
The problem was that on sh, the initial boot parameters were
previously referenced as an array and after 6215d9f447 ("arch, mm:
consolidate empty_zero_page"), they were referenced as a pointer which
caused wrong code generation and boot hang.
This changes the declaration back to being an array which fixes the
boot hang"
* tag 'sh-for-v7.1-tag2' of git://git.kernel.org/pub/scm/linux/kernel/git/glaubitz/sh-linux:
sh: Fix fallout from ZERO_PAGE consolidation
-----BEGIN PGP SIGNATURE-----
iQFPBAABCAA5FiEEe7vIQRWZI0iWSE3xu+CwddJFiJoFAmn3BZgbFIAAAAAABAAO
bWFudTIsMi41KzEuMTIsMiwyAAoJELvgsHXSRYiabzcIAIDyPybWZ/bRup/KfWGE
GknYLaUA3nw9ZpZQdQ0cJ+jGO6pfoUqacO0cZ/5ppdcXblKz22AFItUnpsd44M9H
92QjwNJoT6vjgzaSWCDE/6TeE0y27HjyhBlIYjs9mTRTb8sIH+mzfDS58FZponb/
RZMGmcJvS4La+VzaetAxzyx+cX7J9W/7zDfvz/qPMpm8tysGe4z8H7/54c5x/U+c
NvuEXaxPaKEa62x5e+IFG6G67NszbJnfRrLGn/UMwp+SS5WUId6D9kmjtT70yJlg
hbdU6zw6kI+DIPxRidnpI/iJqcA08r9UuC/TrZwcjaM6sythev5UvOxZXMRVp19w
qXU=
=yAe1
-----END PGP SIGNATURE-----
Merge tag 'slab-for-7.1-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/vbabka/slab
Pull slab fixes from Vlastimil Babka:
- Stable fixes for CONFIG_SMP=n where _nolock() allocations in NMI both
at kmalloc and page allocator levels are not properly protected by
the spin_trylock() semantics on !SMP (Harry Yoo)
* tag 'slab-for-7.1-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/vbabka/slab:
mm/slab: return NULL early from kmalloc_nolock() in NMI on UP
mm/page_alloc: return NULL early from alloc_frozen_pages_nolock() in NMI on UP
