aboutsummaryrefslogtreecommitdiff
path: root/drivers/infiniband/core/rw.c
AgeCommit message (Collapse)AuthorFilesLines
2020-08-23treewide: Use fallthrough pseudo-keywordGustavo A. R. Silva1-1/+0
Replace the existing /* fall through */ comments and its variants with the new pseudo-keyword macro fallthrough[1]. Also, remove unnecessary fall-through markings when it is the case. [1] https://www.kernel.org/doc/html/v5.7/process/deprecated.html?highlight=fallthrough#implicit-switch-case-fall-through Signed-off-by: Gustavo A. R. Silva <[email protected]>
2020-04-15RDMA/rw: use DIV_ROUND_UP to calculate nr_opsMax Gurtovoy1-1/+1
Don't open-code DIV_ROUND_UP() kernel macro. Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Max Gurtovoy <[email protected]> Signed-off-by: Leon Romanovsky <[email protected]> Signed-off-by: Jason Gunthorpe <[email protected]>
2020-03-10RDMA/rw: map P2P memory correctly for signature operationsMax Gurtovoy1-6/+6
Since RDMA rw API support operations with P2P memory sg list, make sure to map/unmap the scatter list for signature operation correctly. Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Max Gurtovoy <[email protected]> Signed-off-by: Jason Gunthorpe <[email protected]>
2020-02-20RDMA/rw: Fix error flow during RDMA context initializationMax Gurtovoy1-11/+20
In case the SGL was mapped for P2P DMA operation, we must unmap it using pci_p2pdma_unmap_sg during the error unwind of rdma_rw_ctx_init() Fixes: 7f73eac3a713 ("PCI/P2PDMA: Introduce pci_p2pdma_unmap_sg()") Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Max Gurtovoy <[email protected]> Reviewed-by: Leon Romanovsky <[email protected]> Reviewed-by: Logan Gunthorpe <[email protected]> Signed-off-by: Jason Gunthorpe <[email protected]>
2019-10-22RDMA/rw: Support threshold for registration vs scattering to local pagesYamin Friedman1-10/+15
If there are more scatter entries than the recommended limit provided by the ib device, UMR registration is used. This will provide optimal performance when performing large RDMA READs over devices that advertise the threshold capability. With ConnectX-5 running NVMeoF RDMA with FIO single QP 128KB writes: Without use of cap: 70Gb/sec With use of cap: 84Gb/sec Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Yamin Friedman <[email protected]> Reviewed-by: Or Gerlitz <[email protected]> Signed-off-by: Leon Romanovsky <[email protected]> Reviewed-by: Christoph Hellwig <[email protected]> Signed-off-by: Jason Gunthorpe <[email protected]>
2019-08-16PCI/P2PDMA: Introduce pci_p2pdma_unmap_sg()Logan Gunthorpe1-2/+4
Add pci_p2pdma_unmap_sg() to the two places that call pci_p2pdma_map_sg(). This is a prep patch to introduce correct mappings for p2pdma transactions that go through the root complex. Link: https://lore.kernel.org/r/[email protected] Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Logan Gunthorpe <[email protected]> Signed-off-by: Bjorn Helgaas <[email protected]> Reviewed-by: Christoph Hellwig <[email protected]>
2019-06-28Merge tag 'v5.2-rc6' into rdma.git for-nextJason Gunthorpe1-9/+1
For dependencies in next patches. Resolve conflicts: - Use uverbs_get_cleared_udata() with new cq allocation flow - Continue to delete nes despite SPDX conflict - Resolve list appends in mlx5_command_str() - Use u16 for vport_rule stuff - Resolve list appends in struct ib_client Signed-off-by: Jason Gunthorpe <[email protected]>
2019-06-24RDMA/rw: Use IB_WR_REG_MR_INTEGRITY for PI handoverIsrael Rukshin1-97/+70
Replace the old signature handover API with the new one. The new API simplifes PI handover code complexity for ULPs and improve performance. For RW API it will reduce the maximum number of work requests per task and the need of dealing with multiple MRs (and their registrations and invalidations) per task. All the mappings and registration of the data and the protection buffers is done by the LLD using a single WR and a special MR type (IB_MR_TYPE_INTEGRITY) for the PI handover operation. The setup of the tested benchmark (using iSER ULP): - 2 servers with 24 cores (1 initiator and 1 target) - ConnectX-4/ConnectX-5 adapters - 24 target sessions with 1 LUN each - ramdisk backstore - PI active Performance results running fio (24 jobs, 128 iodepth) using write_generate=1 and read_verify=1 (w/w.o patch): bs IOPS(read) IOPS(write) ---- ---------- ---------- 512 1243.3K/1182.3K 1725.1K/1680.2K 4k 571233/528835 743293/748259 32k 72388/71086 71789/93573 Using write_generate=0 and read_verify=0 (w/w.o patch): bs IOPS(read) IOPS(write) ---- ---------- ---------- 512 1572.1K/1427.2K 1823.5K/1724.3K 4k 921992/916194 753772/768267 32k 75052/73960 73180/95484 There is a performance degradation when writing big block sizes. Degradation is caused by the complexity of combining multiple indirections and perform RDMA READ operation from it. This will be fixed in the following patches by reducing the indirections if possible. Signed-off-by: Israel Rukshin <[email protected]> Reviewed-by: Max Gurtovoy <[email protected]> Reviewed-by: Christoph Hellwig <[email protected]> Signed-off-by: Jason Gunthorpe <[email protected]>
2019-06-24RDMA/rw: Introduce rdma_rw_inv_key helperIsrael Rukshin1-8/+17
This is a preparation for adding new signature API to the rw-API. Signed-off-by: Israel Rukshin <[email protected]> Suggested-by: Christoph Hellwig <[email protected]> Reviewed-by: Max Gurtovoy <[email protected]> Reviewed-by: Christoph Hellwig <[email protected]> Signed-off-by: Jason Gunthorpe <[email protected]>
2019-06-24RDMA/core: Rename signature qp create flag and signature device capabilityIsrael Rukshin1-2/+2
Rename IB_QP_CREATE_SIGNATURE_EN to IB_QP_CREATE_INTEGRITY_EN and IB_DEVICE_SIGNATURE_HANDOVER to IB_DEVICE_INTEGRITY_HANDOVER. Signed-off-by: Israel Rukshin <[email protected]> Reviewed-by: Max Gurtovoy <[email protected]> Reviewed-by: Christoph Hellwig <[email protected]> Signed-off-by: Jason Gunthorpe <[email protected]>
2019-06-24RDMA/core: Add an integrity MR pool supportIsrael Rukshin1-2/+2
This is a preparation for adding new signature API to the rw-API. Signed-off-by: Israel Rukshin <[email protected]> Reviewed-by: Max Gurtovoy <[email protected]> Reviewed-by: Christoph Hellwig <[email protected]> Reviewed-by: Sagi Grimberg <[email protected]> Signed-off-by: Jason Gunthorpe <[email protected]>
2019-06-05treewide: Replace GPLv2 boilerplate/reference with SPDX - rule 288Thomas Gleixner1-9/+1
Based on 1 normalized pattern(s): this program is free software you can redistribute it and or modify it under the terms and conditions of the gnu general public license version 2 as published by the free software foundation this program is distributed in the hope it will be useful but without any warranty without even the implied warranty of merchantability or fitness for a particular purpose see the gnu general public license for more details extracted by the scancode license scanner the SPDX license identifier GPL-2.0-only has been chosen to replace the boilerplate/reference in 263 file(s). Signed-off-by: Thomas Gleixner <[email protected]> Reviewed-by: Allison Randal <[email protected]> Reviewed-by: Alexios Zavras <[email protected]> Cc: [email protected] Link: https://lkml.kernel.org/r/[email protected] Signed-off-by: Greg Kroah-Hartman <[email protected]>
2019-05-21RDMA/rw: Add info regarding SG count failureMax Gurtovoy1-1/+2
Print the supported and wanted values for SG count during signature operation. Signed-off-by: Max Gurtovoy <[email protected]> Reviewed-by: Christoph Hellwig <[email protected]> Signed-off-by: Jason Gunthorpe <[email protected]>
2019-05-21RDMA/rw: Print the correct number of sig MRsIsrael Rukshin1-1/+1
A wrong value was printed in case of sig MR pool initialization failure. Signed-off-by: Israel Rukshin <[email protected]> Reviewed-by: Max Gurtovoy <[email protected]> Reviewed-by: Christoph Hellwig <[email protected]> Signed-off-by: Jason Gunthorpe <[email protected]>
2019-05-21RDMA/rw: Fix doc typoIsrael Rukshin1-1/+1
Use the correct function name. Signed-off-by: Israel Rukshin <[email protected]> Reviewed-by: Max Gurtovoy <[email protected]> Reviewed-by: Christoph Hellwig <[email protected]> Signed-off-by: Jason Gunthorpe <[email protected]>
2019-02-04IB/core: Remove ib_sg_dma_address() and ib_sg_dma_len()Bart Van Assche1-7/+5
Keeping single line wrapper functions is not useful. Hence remove the ib_sg_dma_address() and ib_sg_dma_len() functions. This patch does not change any functionality. Signed-off-by: Bart Van Assche <[email protected]> Signed-off-by: Jason Gunthorpe <[email protected]>
2018-10-17IB/core: Ensure we map P2P memory correctly in rdma_rw_ctx_[init|destroy]()Logan Gunthorpe1-2/+9
In order to use PCI P2P memory the pci_p2pmem_map_sg() function must be called to map the correct PCI bus address. To do this, check the first page in the scatter list to see if it is P2P memory or not. At the moment, scatter lists that contain P2P memory must be homogeneous so if the first page is P2P the entire SGL should be P2P. Signed-off-by: Logan Gunthorpe <[email protected]> Signed-off-by: Bjorn Helgaas <[email protected]> Reviewed-by: Christoph Hellwig <[email protected]> Reviewed-by: Sagi Grimberg <[email protected]>
2018-07-24RDMA/core: Simplify ib_post_(send|recv|srq_recv)() callsBart Van Assche1-2/+2
Instead of declaring and passing a dummy 'bad_wr' pointer, pass NULL as third argument to ib_post_(send|recv|srq_recv)(). Signed-off-by: Bart Van Assche <[email protected]> Signed-off-by: Jason Gunthorpe <[email protected]>
2018-07-09RDMA/rw: Fix rdma_rw_ctx_signature_init() kernel-doc headerBart Van Assche1-1/+1
Fixes: 0e353e34e1e7 ("IB/core: add RW API support for signature MRs") Signed-off-by: Bart Van Assche <[email protected]> Cc: Christoph Hellwig <[email protected]> Signed-off-by: Jason Gunthorpe <[email protected]>
2018-07-04IB/core: type promotion bug in rdma_rw_init_one_mr()Dan Carpenter1-1/+1
"nents" is an unsigned int, so if ib_map_mr_sg() returns a negative error code then it's type promoted to a high unsigned int which is treated as success. Fixes: a060b5629ab0 ("IB/core: generic RDMA READ/WRITE API") Signed-off-by: Dan Carpenter <[email protected]> Signed-off-by: Jason Gunthorpe <[email protected]>
2017-10-10IB/core: remove redundant check on prot_sg_cntColin Ian King1-14/+10
prot_sg_cnt cannot be zero as a previous check on ret (from which prot_sg_cnt is assigned) returns -ENOMEM if is it zero. Since it cannot be zero we can simplify the code by removing the non -zero check on prot_sg_cnt and redundant else statement. Detected by CoverityScan, COD#1357188 ("Logically dead code") Signed-off-by: Colin Ian King <[email protected]> Signed-off-by: Doug Ledford <[email protected]>
2017-09-05rdma core: Add rdma_rw_mr_payload()Chuck Lever1-0/+24
The amount of payload per MR depends on device capabilities and the memory registration mode in use. The new rdma_rw API hides both, making it difficult for ULPs to determine how large their transport send queues need to be. Expose the MR payload information via a new API. Signed-off-by: Chuck Lever <[email protected]> Acked-by: Doug Ledford <[email protected]> Signed-off-by: J. Bruce Fields <[email protected]>
2016-08-02IB/core, RDMA RW API: Do not exceed QP SGE send limitBart Van Assche1-8/+2
Compute the SGE limit for RDMA READ and WRITE requests in ib_create_qp(). Use that limit in the RDMA RW API implementation. Signed-off-by: Bart Van Assche <[email protected]> Cc: Christoph Hellwig <[email protected]> Cc: Sagi Grimberg <[email protected]> Cc: Steve Wise <[email protected]> Cc: Parav Pandit <[email protected]> Cc: Nicholas Bellinger <[email protected]> Cc: Laurence Oberman <[email protected]> Cc: <[email protected]> #v4.7+ Reviewed-by: Christoph Hellwig <[email protected]> Signed-off-by: Doug Ledford <[email protected]>
2016-08-02IB/core: Make rdma_rw_ctx_init() initialize all used fieldsBart Van Assche1-5/+9
Some but not all callers of rdma_rw_ctx_init() zero-initialize struct rdma_rw_ctx. Hence make rdma_rw_ctx_init() initialize all work request fields that will be read by ib_post_send(). Fixes: a060b5629ab0 ("IB/core: generic RDMA READ/WRITE API") Signed-off-by: Bart Van Assche <[email protected]> Reviewed-by: Sagi Grimberg <[email protected]> Reviewed-by: Christoph Hellwig <[email protected]> Tested-by: Steve Wise <[email protected]> Tested-by: Laurence Oberman <[email protected]> Cc: Parav Pandit <[email protected]> Cc: Nicholas Bellinger <[email protected]> Cc: <[email protected]> #v4.7+ Signed-off-by: Doug Ledford <[email protected]>
2016-05-13IB/core: Enhance ib_map_mr_sg()Bart Van Assche1-1/+1
The SRP initiator allows to set max_sectors to a value that exceeds the largest amount of data that can be mapped at once with an mlx4 HCA using fast registration and a page size of 4 KB. Hence modify ib_map_mr_sg() such that it can map partial sg-elements. If an sg-element has been mapped partially, let the caller know which fraction has been mapped by adjusting *sg_offset. Signed-off-by: Bart Van Assche <[email protected]> Tested-by: Laurence Oberman <[email protected]> Cc: Christoph Hellwig <[email protected]> Cc: Sagi Grimberg <[email protected]> Signed-off-by: Doug Ledford <[email protected]>
2016-05-13IB/core: add RW API support for signature MRsChristoph Hellwig1-5/+223
Signed-off-by: Christoph Hellwig <[email protected]> Signed-off-by: Doug Ledford <[email protected]>
2016-05-13IB/core: generic RDMA READ/WRITE APIChristoph Hellwig1-0/+509
This supports both manual mapping of lots of SGEs, as well as using MRs from the QP's MR pool, for iWarp or other cases where it's more optimal. For now, MRs are only used for iWARP transports. The user of the RDMA-RW API must allocate the QP MR pool as well as size the SQ accordingly. Thanks to Steve Wise for testing, fixing and rewriting the iWarp support, and to Sagi Grimberg for ideas, reviews and fixes. Signed-off-by: Christoph Hellwig <[email protected]> Signed-off-by: Doug Ledford <[email protected]>
Powered by cgit, Gruvboxed by Blaster4385.