| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
Add support for the AES/SM4 crypto engine included in the Offload and
Crypto Subsystem (OCS) of the Intel Keem Bay SoC, thus enabling
hardware-acceleration for the following transformations:
- ecb(aes), cbc(aes), ctr(aes), cts(cbc(aes)), gcm(aes) and cbc(aes);
supported for 128-bit and 256-bit keys.
- ecb(sm4), cbc(sm4), ctr(sm4), cts(cbc(sm4)), gcm(sm4) and cbc(sm4);
supported for 128-bit keys.
The driver passes crypto manager self-tests, including the extra tests
(CRYPTO_MANAGER_EXTRA_TESTS=y).
Signed-off-by: Mike Healy <[email protected]>
Co-developed-by: Daniele Alessandrelli <[email protected]>
Signed-off-by: Daniele Alessandrelli <[email protected]>
Acked-by: Mark Gross <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Add device-tree bindings for Intel Keem Bay Offload and Crypto Subsystem
(OCS) AES crypto driver.
Signed-off-by: Daniele Alessandrelli <[email protected]>
Acked-by: Mark Gross <[email protected]>
Reviewed-by: Rob Herring <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Geert reports that builds where CONFIG_CRYPTO_AEGIS128_SIMD is not set
may still emit references to crypto_aegis128_update_simd(), which
cannot be satisfied and therefore break the build. These references
only exist in functions that can be optimized away, but apparently,
the compiler is not always able to prove this.
So add some explicit checks for CONFIG_CRYPTO_AEGIS128_SIMD to help the
compiler figure this out.
Tested-by: Geert Uytterhoeven <[email protected]>
Signed-off-by: Ard Biesheuvel <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
The macro use will already have a semicolon.
Signed-off-by: Tom Rix <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
CMP $0,%reg can't set overflow flag, so we can use shorter TEST %reg,%reg
instruction when only zero and sign flags are checked (E,L,LE,G,GE conditions).
Signed-off-by: Uros Bizjak <[email protected]>
Cc: Herbert Xu <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: "H. Peter Anvin" <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
CMP $0,%reg can't set overflow flag, so we can use shorter TEST %reg,%reg
instruction when only zero and sign flags are checked (E,L,LE,G,GE conditions).
Signed-off-by: Uros Bizjak <[email protected]>
Cc: Herbert Xu <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: "H. Peter Anvin" <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
CMP $0,%reg can't set overflow flag, so we can use shorter TEST %reg,%reg
instruction when only zero and sign flags are checked (E,L,LE,G,GE conditions).
Signed-off-by: Uros Bizjak <[email protected]>
Cc: Herbert Xu <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: "H. Peter Anvin" <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
This patch fixes a few sparse warnings that were missed in the
last round.
Signed-off-by: Herbert Xu <[email protected]>
|
|
This patch adds a dependency for KEYSTONE on HAS_IOMEM and OF to
prevent COMPILE_TEST build failures.
Reported-by: kernel test robot <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
This patch fixes a missing prototype warning on blake2s_selftest.
Reported-by: kernel test robot <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
ARM Cortex-A57 and Cortex-A72 cores running in 32-bit mode are affected
by silicon errata #1742098 and #1655431, respectively, where the second
instruction of a AES instruction pair may execute twice if an interrupt
is taken right after the first instruction consumes an input register of
which a single 32-bit lane has been updated the last time it was modified.
This is not such a rare occurrence as it may seem: in counter mode, only
the least significant 32-bit word is incremented in the absence of a
carry, which makes our counter mode implementation susceptible to these
errata.
So let's shuffle the counter assignments around a bit so that the most
recent updates when the AES instruction pair executes are 128-bit wide.
[0] ARM-EPM-049219 v23 Cortex-A57 MPCore Software Developers Errata Notice
[1] ARM-EPM-012079 v11.0 Cortex-A72 MPCore Software Developers Errata Notice
Cc: <[email protected]> # v5.4+
Signed-off-by: Ard Biesheuvel <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
ecdh_set_secret() casts a void* pointer to a const u64* in order to
feed it into ecc_is_key_valid(). This is not generally permitted by
the C standard, and leads to actual misalignment faults on ARMv6
cores. In some cases, these are fixed up in software, but this still
leads to performance hits that are entirely avoidable.
So let's copy the key into the ctx buffer first, which we will do
anyway in the common case, and which guarantees correct alignment.
Cc: <[email protected]>
Signed-off-by: Ard Biesheuvel <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Rework the setting of DMA cache parameters, program more appropriate
values and explicitly set sharability domain.
Signed-off-by: Gilad Ben-Yossef <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
'pci_set_dma_mask()' + 'pci_set_consistent_dma_mask()' can be replaced by
an equivalent 'dma_set_mask_and_coherent()' which is much less verbose.
Signed-off-by: Christophe JAILLET <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
'pci_set_dma_mask()' + 'pci_set_consistent_dma_mask()' can be replaced by
an equivalent 'dma_set_mask_and_coherent()' which is much less verbose.
Signed-off-by: Christophe JAILLET <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
'pci_set_dma_mask()' + 'pci_set_consistent_dma_mask()' can be replaced by
an equivalent 'dma_set_mask_and_coherent()' which is much less verbose.
Signed-off-by: Christophe JAILLET <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
In preparation to enable -Wimplicit-fallthrough for Clang, fix multiple
warnings by explicitly adding multiple break statements instead of
letting the code fall through to the next case.
Link: https://github.com/KSPP/linux/issues/115
Signed-off-by: Gustavo A. R. Silva <[email protected]>
Acked-by: Gilad Ben-Yossef <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
WireGuard and IPsec both typically operate on input blocks that are
~1420 bytes in size, given the default Ethernet MTU of 1500 bytes and
the overhead of the VPN metadata.
Many aead and sckipher implementations are optimized for power-of-2
block sizes, and whether they perform well when operating on 1420
byte blocks cannot be easily extrapolated from the performance on
power-of-2 block size. So let's add 1420 bytes explicitly, and round
it up to the next blocksize multiple of the algo in question if it
does not support 1420 byte blocks.
Signed-off-by: Ard Biesheuvel <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
When working on crypto algorithms, being able to run tcrypt quickly
without booting an entire Linux installation can be very useful. For
instance, QEMU/kvm can be used to boot a kernel from the command line,
and having tcrypt.ko builtin would allow tcrypt to be executed to run
benchmarks, or to run tests for algorithms that need to be instantiated
from templates, without the need to make it past the point where the
rootfs is mounted.
So let's relax the requirement that tcrypt can only be built as a module
when CONFIG_EXPERT is enabled.
Signed-off-by: Ard Biesheuvel <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Commit c4741b2305979 ("crypto: run initcalls for generic implementations
earlier") converted tcrypt.ko's module_init() to subsys_initcall(), but
this was unintentional: tcrypt.ko currently cannot be built into the core
kernel, and so the subsys_initcall() gets converted into module_init()
under the hood. Given that tcrypt.ko does not implement a generic version
of a crypto algorithm that has to be available early during boot, there
is no point in running the tcrypt init code earlier than implied by
module_init().
However, for crypto development purposes, we will lift the restriction
that tcrypt.ko must be built as a module, and when builtin, it makes sense
for tcrypt.ko (which does its work inside the module init function) to run
as late as possible. So let's switch to late_initcall() instead.
Signed-off-by: Ard Biesheuvel <[email protected]>
Reviewed-by: Eric Biggers <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Move HiSilicon TRNG V2 driver into 'drivers/crypto/hisilicon/trng'
with some updating on 'MAINTAINERS'.
Signed-off-by: Weili Qian <[email protected]>
Reviewed-by: Zaibo Xu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
This patch adds support for pseudo random number generator(PRNG)
in Crypto subsystem.
Signed-off-by: Weili Qian <[email protected]>
Reviewed-by: Zaibo Xu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Move existing char/hw_random/hisi-trng-v2.c to crypto/hisilicon/trng.c.
Signed-off-by: Weili Qian <[email protected]>
Reviewed-by: Zaibo Xu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Driver of HiSilicon true random number generator(TRNG)
is removed from 'drivers/char/hw_random'.
Both 'Kunpeng 920' and 'Kunpeng 930' chips have TRNG,
however, PRNG is only supported by 'Kunpeng 930'.
So, this driver is moved to 'drivers/crypto/hisilicon/trng/'
in the next to enable the two's TRNG better.
Signed-off-by: Weili Qian <[email protected]>
Reviewed-by: Zaibo Xu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
This patch fixes a coulpe of sparse endianness warnings.
Signed-off-by: Herbert Xu <[email protected]>
|
|
This patch fixes a sparse endianness warning in sha256-spe.
Signed-off-by: Herbert Xu <[email protected]>
|
|
This patch fixes a number of endianness warnings in the mips/octeon
code.
Signed-off-by: Herbert Xu <[email protected]>
|
|
Condition !A || A && B is equivalent to !A || B.
Generated by: scripts/coccinelle/misc/excluded_middle.cocci
Fixes: b76f0ea01312 ("coccinelle: misc: add excluded_middle.cocci script")
CC: Denis Efremov <[email protected]>
Reported-by: kernel test robot <[email protected]>
Signed-off-by: kernel test robot <[email protected]>
Signed-off-by: Julia Lawall <[email protected]>
Signed-off-by: Giovanni Cabiddu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Partial hash was being copied into the final result buffer without the
entire message block processed. Depending on how the end user processes
this result buffer, errors vary from result buffer corruption to result
buffer poisoing. Fix this issue by ensuring that only the final hash value
is copied into the result buffer.
Reviewed-by: Bjorn Andersson <[email protected]>
Signed-off-by: Thara Gopinath <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Add support Qualcomm Crypto Engine accelerated encryption and
authentication algorithms on sdm845.
Reviewed-by: Bjorn Andersson <[email protected]>
Signed-off-by: Thara Gopinath <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Wiring the SIMD code into the generic driver has the unfortunate side
effect that the tcrypt testing code cannot distinguish them, and will
therefore not use the latter to fuzz test the former, as it does for
other algorithms.
So let's refactor the code a bit so we can register two implementations:
aegis128-generic and aegis128-simd.
Signed-off-by: Ard Biesheuvel <[email protected]>
Reviewed-by: Ondrej Mosnacek <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Instead of calculating the tag and returning it to the caller on
decryption, use a SIMD compare and min across vector to perform
the comparison. This is slightly more efficient, and removes the
need on the caller's part to wipe the tag from memory if the
decryption failed.
While at it, switch to unsigned int when passing cryptlen and
assoclen - we don't support input sizes where it matters anyway.
Signed-off-by: Ard Biesheuvel <[email protected]>
Reviewed-by: Ondrej Mosnacek <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Avoid copying the tail block via a stack buffer if the total size
exceeds a single AEGIS block. In this case, we can use overlapping
loads and stores and NEON permutation instructions instead, which
leads to a modest performance improvement on some cores (< 5%),
and is slightly cleaner. Note that we still need to use a stack
buffer if the entire input is smaller than 16 bytes, given that
we cannot use 16 byte NEON loads and stores safely in this case.
Signed-off-by: Ard Biesheuvel <[email protected]>
Reviewed-by: Ondrej Mosnacek <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
The AEGIS spec mentions explicitly that the security guarantees hold
only if the resulting plaintext and tag of a failed decryption are
withheld. So ensure that we abide by this.
While at it, drop the unused struct aead_request *req parameter from
crypto_aegis128_process_crypt().
Reviewed-by: Ondrej Mosnacek <[email protected]>
Signed-off-by: Ard Biesheuvel <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
This patch fixes the following smatch warnings:
drivers/crypto/allwinner/sun8i-ce/sun8i-ce-hash.c:412
sun8i_ce_hash_run() warn: possible memory leak of 'result'
Note: "buf" is leaked as well.
Furthermore, in case of ENOMEM, crypto_finalize_hash_request() was not
called which was an error.
Fixes: 56f6d5aee88d ("crypto: sun8i-ce - support hash algorithms")
Reported-by: kernel test robot <[email protected]>
Reported-by: Dan Carpenter <[email protected]>
Signed-off-by: Corentin Labbe <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
There are a couple of spelling mistakes in two crypto Kconfig files.
Fix these.
Signed-off-by: Colin Ian King <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Add support for QAT 4xxx devices.
Signed-off-by: Giovanni Cabiddu <[email protected]>
Reviewed-by: Fiona Trahe <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Add an hook to initialize the vector routing table with the default
values before MSIx is enabled.
The new function set_msix_rttable() is called only if present in the
struct adf_hw_device_data of the device. This is to allow for QAT
devices that do not support that functionality.
Signed-off-by: Giovanni Cabiddu <[email protected]>
Reviewed-by: Fiona Trahe <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Introduce support for devices that require multiple firmware images.
If a device requires more than a firmware image to operate, load the
image to the appropriate Acceleration Engine (AE).
Signed-off-by: Giovanni Cabiddu <[email protected]>
Reviewed-by: Fiona Trahe <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
The pm_runtime_enable will increase power disable depth.
Thus a pairing decrement is needed on the error handling
path to keep it balanced according to context.
Fixes: f7b2b5dd6a62a ("crypto: omap-aes - add error check for pm_runtime_get_sync")
Signed-off-by: Zhang Qilong <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
The patch 'irqchip/gic-v3-its: Balance initial LPI affinity across CPUs'
set the IRQ to an uncentain CPU. If an IRQ is bound to the CPU used by the
thread which is sending request, the throughput will be just half.
So allocate a 'work_queue' and set as 'WQ_UNBOUND' to do the back half work
on some different CPUS.
Signed-off-by: Yang Shen <[email protected]>
Reviewed-by: Zaibo Xu <[email protected]>
Reviewed-by: Zhou Wang <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
This patch moves the curve25519_selftest into curve25519.h so
we don't get a warning from gcc complaining about a missing
prototype.
Reported-by: kernel test robot <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Currently <crypto/sha.h> contains declarations for both SHA-1 and SHA-2,
and <crypto/sha3.h> contains declarations for SHA-3.
This organization is inconsistent, but more importantly SHA-1 is no
longer considered to be cryptographically secure. So to the extent
possible, SHA-1 shouldn't be grouped together with any of the other SHA
versions, and usage of it should be phased out.
Therefore, split <crypto/sha.h> into two headers <crypto/sha1.h> and
<crypto/sha2.h>, and make everyone explicitly specify whether they want
the declarations for SHA-1, SHA-2, or both.
This avoids making the SHA-1 declarations visible to files that don't
want anything to do with SHA-1. It also prepares for potentially moving
sha1.h into a new insecure/ or dangerous/ directory.
Signed-off-by: Eric Biggers <[email protected]>
Acked-by: Ard Biesheuvel <[email protected]>
Acked-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Clang warns:
drivers/crypto/amcc/crypto4xx_core.c:921:60: warning: operator '?:' has
lower precedence than '|'; '|' will be evaluated first
[-Wbitwise-conditional-parentheses]
(crypto_tfm_alg_type(req->tfm) == CRYPTO_ALG_TYPE_AEAD) ?
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ ^
drivers/crypto/amcc/crypto4xx_core.c:921:60: note: place parentheses
around the '|' expression to silence this warning
(crypto_tfm_alg_type(req->tfm) == CRYPTO_ALG_TYPE_AEAD) ?
^
)
drivers/crypto/amcc/crypto4xx_core.c:921:60: note: place parentheses
around the '?:' expression to evaluate it first
(crypto_tfm_alg_type(req->tfm) == CRYPTO_ALG_TYPE_AEAD) ?
^
(
1 warning generated.
It looks like this should have been a logical OR so that
PD_CTL_HASH_FINAL gets added to the w bitmask if crypto_tfm_alg_type
is either CRYPTO_ALG_TYPE_AHASH or CRYPTO_ALG_TYPE_AEAD. Change the
operator so that everything works properly.
Fixes: 4b5b79998af6 ("crypto: crypto4xx - fix stalls under heavy load")
Link: https://github.com/ClangBuiltLinux/linux/issues/1198
Signed-off-by: Nathan Chancellor <[email protected]>
Reviewed-by: Christian Lamparter <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Wang Qing reports that IS_ERR_OR_NULL() should be matched with
PTR_ERR_OR_ZERO(), not PTR_ERR().
As it turns out, the error path always returns an error code,
i.e. NULL is never returned.
Update the code accordingly - s/IS_ERR_OR_NULL/IS_ERR.
Reported-by: Wang Qing <[email protected]>
Signed-off-by: Horia Geantă <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Instead of copying the calculated authentication tag to memory and
calling crypto_memneq() to verify it, use vector bytewise compare and
min across vector instructions to decide whether the tag is valid. This
is more efficient, and given that the tag is only transiently held in a
NEON register, it is also safer, given that calculated tags for failed
decryptions should be withheld.
Signed-off-by: Ard Biesheuvel <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Fix aead auth setting key process error. if use soft shash function, driver
need to use digest size replace of the user input key length.
Signed-off-by: Kai Ye <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Based on lessons learnt from optimizing the 32-bit version of this driver,
we can simplify the arm64 version considerably, by reordering the final
two stores when the last block is not a multiple of 64 bytes. This removes
the need to use permutation instructions to calculate the elements that are
clobbered by the final overlapping store, given that the store of the
penultimate block now follows it, and that one carries the correct values
for those elements already.
While at it, simplify the overlapping loads as well, by calculating the
address of the final overlapping load upfront, and switching to this
address for every load that would otherwise extend past the end of the
source buffer.
There is no impact on performance, but the resulting code is substantially
smaller and easier to follow.
Cc: Eric Biggers <[email protected]>
Cc: "Jason A . Donenfeld" <[email protected]>
Signed-off-by: Ard Biesheuvel <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Add support for the QAT gen4 devices in the firmware loader.
Signed-off-by: Jack Xu <[email protected]>
Reviewed-by: Giovanni Cabiddu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Add support for broadcasting mode in firmware loader to enable the next
generation of QAT devices.
Signed-off-by: Jack Xu <[email protected]>
Co-developed-by: Wojciech Ziemba <[email protected]>
Signed-off-by: Wojciech Ziemba <[email protected]>
Reviewed-by: Giovanni Cabiddu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|