Age | Commit message (Collapse) | Author | Files | Lines |
|
Add support for shared ustore mode support. This is required by the next
generation of QAT devices to share the same fw image across engines.
Signed-off-by: Jack Xu <[email protected]>
Co-developed-by: Wojciech Ziemba <[email protected]>
Signed-off-by: Wojciech Ziemba <[email protected]>
Reviewed-by: Giovanni Cabiddu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Introduce new API, qat_uclo_set_cfg_ae_mask(), to allow the load of the
firmware image to a subset of Acceleration Engines (AEs). This is
required by the next generation of QAT devices to be able to load
different firmware images to the device.
Signed-off-by: Jack Xu <[email protected]>
Reviewed-by: Giovanni Cabiddu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Add firmware control unit (FCU) CSRs to chip info so the firmware
authentication code is common between all devices.
Signed-off-by: Jack Xu <[email protected]>
Co-developed-by: Wojciech Ziemba <[email protected]>
Signed-off-by: Wojciech Ziemba <[email protected]>
Reviewed-by: Giovanni Cabiddu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Add support for CSS3K, which uses RSA3K as image signature algorithm,
to support the next generation of QAT devices.
Signed-off-by: Jack Xu <[email protected]>
Co-developed-by: Wojciech Ziemba <[email protected]>
Signed-off-by: Wojciech Ziemba <[email protected]>
Reviewed-by: Giovanni Cabiddu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Use ae_mask to decide which Accelerator Engine (AE) to target in AE
related operations, instead of a sequential loop, to skip AEs that are
fused out.
Signed-off-by: Jack Xu <[email protected]>
Co-developed-by: Wojciech Ziemba <[email protected]>
Signed-off-by: Wojciech Ziemba <[email protected]>
Reviewed-by: Giovanni Cabiddu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Add null pointer check when freeing the memory for firmware.
Signed-off-by: Jack Xu <[email protected]>
Co-developed-by: Wojciech Ziemba <[email protected]>
Signed-off-by: Wojciech Ziemba <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Add misc control CSR to chip info since the CSR offset will be different
in the next generation of QAT devices.
Signed-off-by: Jack Xu <[email protected]>
Co-developed-by: Wojciech Ziemba <[email protected]>
Signed-off-by: Wojciech Ziemba <[email protected]>
Reviewed-by: Giovanni Cabiddu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Add the wake up event to chip info since this value will be different
in the next generation of QAT devices.
Signed-off-by: Jack Xu <[email protected]>
Co-developed-by: Wojciech Ziemba <[email protected]>
Signed-off-by: Wojciech Ziemba <[email protected]>
Reviewed-by: Giovanni Cabiddu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Add global clock enable CSR to the chip info since the CSR offset
will be different in the next generation of QAT devices.
Signed-off-by: Jack Xu <[email protected]>
Co-developed-by: Wojciech Ziemba <[email protected]>
Signed-off-by: Wojciech Ziemba <[email protected]>
Reviewed-by: Giovanni Cabiddu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Add reset CSR offset and mask to chip info since they are different
in new QAT devices. This also simplifies the reset/clrReset functions
by using the reset mask.
Signed-off-by: Jack Xu <[email protected]>
Co-developed-by: Wojciech Ziemba <[email protected]>
Signed-off-by: Wojciech Ziemba <[email protected]>
Reviewed-by: Giovanni Cabiddu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Add the local memory size to the chip info since the size of this memory
will be different in the next generation of QAT devices.
Signed-off-by: Jack Xu <[email protected]>
Co-developed-by: Wojciech Ziemba <[email protected]>
Signed-off-by: Wojciech Ziemba <[email protected]>
Reviewed-by: Giovanni Cabiddu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Add support for local memory lm2 and lm3 which is introduced in the next
generation of QAT devices.
Signed-off-by: Jack Xu <[email protected]>
Co-developed-by: Wojciech Ziemba <[email protected]>
Signed-off-by: Wojciech Ziemba <[email protected]>
Reviewed-by: Giovanni Cabiddu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Introduce the next neighbor (NN) capability in chip_info as NN registers
are not supported in certain SKUs of QAT.
Signed-off-by: Jack Xu <[email protected]>
Co-developed-by: Wojciech Ziemba <[email protected]>
Signed-off-by: Wojciech Ziemba <[email protected]>
Reviewed-by: Giovanni Cabiddu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Modify condition in qat_uclo_wr_mimage() to use a capability of the
device (sram_visible), rather than the device ID, so the check is not
specific to devices of the same type.
Signed-off-by: Jack Xu <[email protected]>
Co-developed-by: Wojciech Ziemba <[email protected]>
Signed-off-by: Wojciech Ziemba <[email protected]>
Reviewed-by: Giovanni Cabiddu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Introduce the chip info structure which contains device specific
information. The initialization path has been split between common and
hardware specific in order to facilitate the introduction of the next
generation hardware.
Signed-off-by: Jack Xu <[email protected]>
Co-developed-by: Wojciech Ziemba <[email protected]>
Signed-off-by: Wojciech Ziemba <[email protected]>
Reviewed-by: Giovanni Cabiddu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Replace long expressions with local variables in the functions
qat_uclo_wr_uimage_page(), qat_uclo_init_globals() and
qat_uclo_init_umem_seg() to improve readability.
Signed-off-by: Jack Xu <[email protected]>
Co-developed-by: Wojciech Ziemba <[email protected]>
Signed-off-by: Wojciech Ziemba <[email protected]>
Reviewed-by: Giovanni Cabiddu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Refactor qat_uclo_set_ae_mode() by moving the logic that sets the AE
modes to a separate function, qat_hal_set_modes().
Signed-off-by: Jack Xu <[email protected]>
Co-developed-by: Wojciech Ziemba <[email protected]>
Signed-off-by: Wojciech Ziemba <[email protected]>
Reviewed-by: Giovanni Cabiddu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Move the definition of ICP_QAT_AE_OFFSET, ICP_QAT_CAP_OFFSET,
LOCAL_TO_XFER_REG_OFFSET and ICP_QAT_EP_OFFSET from qat_hal.c to
icp_qat_hal.h to avoid the definition of generation specific constants
in qat_hal.c.
Signed-off-by: Jack Xu <[email protected]>
Co-developed-by: Wojciech Ziemba <[email protected]>
Signed-off-by: Wojciech Ziemba <[email protected]>
Reviewed-by: Giovanni Cabiddu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Include the offset of GLOBAL_CSR directly into the enum hal_global_csr
and remove the macros SET_GLB_CSR/GET_GLB_CSR to simplify the global CSR
access.
Signed-off-by: Jack Xu <[email protected]>
Co-developed-by: Wojciech Ziemba <[email protected]>
Signed-off-by: Wojciech Ziemba <[email protected]>
Reviewed-by: Giovanni Cabiddu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Change the API and the behaviour of the qat_hal_start() function.
With this change, the function starts under the hood all acceleration
engines (AEs) and there is no longer need to call it for each engine.
Signed-off-by: Jack Xu <[email protected]>
Co-developed-by: Wojciech Ziemba <[email protected]>
Signed-off-by: Wojciech Ziemba <[email protected]>
Reviewed-by: Giovanni Cabiddu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Change micro word data mask since the Acceleration Engine (AE)
instruction codes have been changed in the new generation QAT devices.
Signed-off-by: Jack Xu <[email protected]>
Co-developed-by: Wojciech Ziemba <[email protected]>
Signed-off-by: Wojciech Ziemba <[email protected]>
Reviewed-by: Giovanni Cabiddu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Change type for ctx_mask from unsigned char to unsigned long to avoid
type casting.
Signed-off-by: Jack Xu <[email protected]>
Co-developed-by: Wojciech Ziemba <[email protected]>
Signed-off-by: Wojciech Ziemba <[email protected]>
Reviewed-by: Giovanni Cabiddu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Improve the way micro instructions (FW code) are uploaded to Accelerator
Engines (AEs). If code starts at PC zero (absolute addressing), read
uwords with no relative address. Otherwise, use relative addressing to
the page region.
Signed-off-by: Jack Xu <[email protected]>
Co-developed-by: Wojciech Ziemba <[email protected]>
Signed-off-by: Wojciech Ziemba <[email protected]>
Reviewed-by: Giovanni Cabiddu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Rename the function qat_uclo_del_uof_obj() in qat_uclo_del_obj() since
it frees the memory allocated for all firmware objects.
Signed-off-by: Jack Xu <[email protected]>
Co-developed-by: Wojciech Ziemba <[email protected]>
Signed-off-by: Wojciech Ziemba <[email protected]>
Reviewed-by: Giovanni Cabiddu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Introduce additional parenthesis to resolve a warninga reported by
checkpatch.
Signed-off-by: Jack Xu <[email protected]>
Co-developed-by: Wojciech Ziemba <[email protected]>
Signed-off-by: Wojciech Ziemba <[email protected]>
Reviewed-by: Giovanni Cabiddu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Remove unnecessary parenthesis across the firmware loader.
Signed-off-by: Jack Xu <[email protected]>
Co-developed-by: Wojciech Ziemba <[email protected]>
Signed-off-by: Wojciech Ziemba <[email protected]>
Reviewed-by: Giovanni Cabiddu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Change message in error path of qat_uclo_check_image_compat() to report
an incompatible firmware image that contains a neighbor register table.
Signed-off-by: Jack Xu <[email protected]>
Co-developed-by: Wojciech Ziemba <[email protected]>
Signed-off-by: Wojciech Ziemba <[email protected]>
Reviewed-by: Giovanni Cabiddu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Do not mask the AE number with the AE mask when accessing the AE local
CSRs. Bit 12 of the local CSR address is the start of AE number so just
take out the AE mask here.
Signed-off-by: Jack Xu <[email protected]>
Reviewed-by: Giovanni Cabiddu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
The return value of qat_hal_rd_ae_csr() is always a CSR value and never
a status and should not be stored in the status variable of
qat_hal_put_rel_rd_xfer().
This removes the assignment as qat_hal_rd_ae_csr() is not expected to
fail.
A more comprehensive handling of the theoretical corner case which could
result in a fail will be submitted in a separate patch.
Fixes: 8c9478a400b7 ("crypto: qat - reduce stack size with KASAN")
Signed-off-by: Jack Xu <[email protected]>
Reviewed-by: Giovanni Cabiddu <[email protected]>
Reviewed-by: Fiona Trahe <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Implement infrastructure for the Multiple Object File (MOF) format
in the firmware loader. This will allow to load a specific firmware
image contained inside an MOF file.
This patch is based on earlier work done by Pingchao Yang.
Signed-off-by: Giovanni Cabiddu <[email protected]>
Reviewed-by: Jack Xu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
This patch fixes all the sparse warnings in cavium/nitrox:
- Fix endianness warnings by adding the correct markers to unions.
- Add missing header inclusions for prototypes.
- Move nitrox_sriov_configure prototype into the isr header file.
Signed-off-by: Herbert Xu <[email protected]>
|
|
Change all lower case pci in comments to be upper case PCI.
Suggested-by: Andy Shevchenko <[email protected]>
Signed-off-by: Adam Guerin <[email protected]>
Reviewed-by: Giovanni Cabiddu <[email protected]>
Reviewed-by: Andy Shevchenko <[email protected]>
Signed-off-by: Giovanni Cabiddu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
The current NEON based ChaCha implementation for ARM is optimized for
multiples of 4x the ChaCha block size (64 bytes). This makes sense for
block encryption, but given that ChaCha is also often used in the
context of networking, it makes sense to consider arbitrary length
inputs as well.
For example, WireGuard typically uses 1420 byte packets, and performing
ChaCha encryption involves 5 invocations of chacha_4block_xor_neon()
and 3 invocations of chacha_block_xor_neon(), where the last one also
involves a memcpy() using a buffer on the stack to process the final
chunk of 1420 % 64 == 12 bytes.
Let's optimize for this case as well, by letting chacha_4block_xor_neon()
deal with any input size between 64 and 256 bytes, using NEON permutation
instructions and overlapping loads and stores. This way, the 140 byte
tail of a 1420 byte input buffer can simply be processed in one go.
This results in the following performance improvements for 1420 byte
blocks, without significant impact on power-of-2 input sizes. (Note
that Raspberry Pi is widely used in combination with a 32-bit kernel,
even though the core is 64-bit capable)
Cortex-A8 (BeagleBone) : 7%
Cortex-A15 (Calxeda Midway) : 21%
Cortex-A53 (Raspberry Pi 3) : 3%
Cortex-A72 (Raspberry Pi 4) : 19%
Cc: Eric Biggers <[email protected]>
Cc: "Jason A . Donenfeld" <[email protected]>
Signed-off-by: Ard Biesheuvel <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Remove cast for mailbox CSR in adf_admin.c as it is not needed.
Suggested-by: Andy Shevchenko <[email protected]>
Signed-off-by: Adam Guerin <[email protected]>
Reviewed-by: Giovanni Cabiddu <[email protected]>
Reviewed-by: Andy Shevchenko <[email protected]>
Signed-off-by: Giovanni Cabiddu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
The extra tests in the manager actually require the manager to be
selected too. Otherwise the linker gives errors like:
ld: arch/x86/crypto/chacha_glue.o: in function `chacha_simd_stream_xor':
chacha_glue.c:(.text+0x422): undefined reference to `crypto_simd_disabled_for_test'
Fixes: 2343d1529aff ("crypto: Kconfig - allow tests to be disabled when manager is disabled")
Signed-off-by: Jason A. Donenfeld <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
At the time xts fallback tfm allocation fails the device struct
hasn't been enabled yet in the caam xts tfm's private context.
Fix this by using the device struct from xts algorithm's private context
or, when not available, by replacing dev_err with pr_err.
Fixes: 9d9b14dbe077 ("crypto: caam/jr - add fallback for XTS with more than 8B IV")
Fixes: 83e8aa912138 ("crypto: caam/qi - add fallback for XTS with more than 8B IV")
Fixes: 36e2d7cfdcf1 ("crypto: caam/qi2 - add fallback for XTS with more than 8B IV")
Signed-off-by: Horia Geantă <[email protected]>
Reviewed-by: Iuliana Prodan <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
'hisi_qm_init' initializes configuration of QM.
To improve code readability, split it into two pieces.
Signed-off-by: Weili Qian <[email protected]>
Reviewed-by: Zhou Wang <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
'qm_eq_ctx_cfg' initializes configuration of EQ and AEQ,
split it into two pieces to improve code readability.
Signed-off-by: Weili Qian <[email protected]>
Reviewed-by: Zhou Wang <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
'qm_qp_ctx_cfg' initializes configuration of SQ and CQ,
split it into two pieces to improve code readability.
Signed-off-by: Weili Qian <[email protected]>
Reviewed-by: Zhou Wang <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Replace 'sprintf' with 'scnprintf' to avoid overrun.
Signed-off-by: Weili Qian <[email protected]>
Reviewed-by: Zhou Wang <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Since 'qm_set_sqctype' always returns 0, change it as 'void'.
Signed-off-by: Weili Qian <[email protected]>
Reviewed-by: Zhou Wang <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Since 'qm_create_debugfs_file' always returns 0, change it as 'void'.
Signed-off-by: Weili Qian <[email protected]>
Reviewed-by: Zhou Wang <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
The returns of 'qm_get_hw_error_status' and 'qm_get_dev_err_status'
are values from the hardware registers, which should not be defined
as 'int', so update as 'u32'.
Signed-off-by: Weili Qian <[email protected]>
Reviewed-by: Zhou Wang <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Some numbers are replaced by macros to avoid incomprehension.
Signed-off-by: Weili Qian <[email protected]>
Reviewed-by: Zhou Wang <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Clean up the check for irq. dev_err() is superfluous as
platform_get_irq() already prints an error. Check for zero
would indicate a bug. Remove curly braces to conform to
styling requirements.
Signed-off-by: Nigel Christian <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Loading the module deadlocks since:
-local cbc(aes) implementation needs a fallback and
-crypto API tries to find one but the request_module() resolves back to
the same module
Fix this by changing the module alias for cbc(aes) and
using the NEED_FALLBACK flag when requesting for a fallback algorithm.
Fixes: 00b99ad2bac2 ("crypto: arm/aes-neonbs - Use generic cbc encryption path")
Signed-off-by: Horia Geantă <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
A semicolon is not needed after a switch statement.
Signed-off-by: Tom Rix <[email protected]>
Signed-off-by: Tom Rix <[email protected]>
Signed-off-by: Giovanni Cabiddu <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
A semicolon is not needed after a switch statement.
Signed-off-by: Tom Rix <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
PAC pointer authentication signs the return address against the value
of the stack pointer, to prevent stack overrun exploits from corrupting
the control flow. However, this requires that the AUTIASP is issued with
SP holding the same value as it held when the PAC value was generated.
The Poly1305 NEON code got this wrong, resulting in crashes on PAC
capable hardware.
Fixes: f569ca164751 ("crypto: arm64/poly1305 - incorporate OpenSSL/CRYPTOGAMS ...")
Signed-off-by: Ard Biesheuvel <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|
|
Commit 3f69cc60768b ("crypto: af_alg - Allow arbitrarily long algorithm
names") made the kernel start accepting arbitrarily long algorithm names
in sockaddr_alg. However, the actual length of the salg_name field
stayed at the original 64 bytes.
This is broken because the kernel can access indices >= 64 in salg_name,
which is undefined behavior -- even though the memory that is accessed
is still located within the sockaddr structure. It would only be
defined behavior if the array were properly marked as arbitrary-length
(either by making it a flexible array, which is the recommended way
these days, or by making it an array of length 0 or 1).
We can't simply change salg_name into a flexible array, since that would
break source compatibility with userspace programs that embed
sockaddr_alg into another struct, or (more commonly) declare a
sockaddr_alg like 'struct sockaddr_alg sa = { .salg_name = "foo" };'.
One solution would be to change salg_name into a flexible array only
when '#ifdef __KERNEL__'. However, that would keep userspace without an
easy way to actually use the longer algorithm names.
Instead, add a new structure 'sockaddr_alg_new' that has the flexible
array field, and expose it to both userspace and the kernel.
Make the kernel use it correctly in alg_bind().
This addresses the syzbot report
"UBSAN: array-index-out-of-bounds in alg_bind"
(https://syzkaller.appspot.com/bug?extid=92ead4eb8e26a26d465e).
Reported-by: [email protected]
Fixes: 3f69cc60768b ("crypto: af_alg - Allow arbitrarily long algorithm names")
Cc: <[email protected]> # v4.12+
Signed-off-by: Eric Biggers <[email protected]>
Signed-off-by: Herbert Xu <[email protected]>
|