aboutsummaryrefslogtreecommitdiff
path: root/drivers/infiniband/ulp/iser
AgeCommit message (Collapse)AuthorFilesLines
2015-04-15IB/iser: Rewrite bounce buffer code pathSagi Grimberg3-88/+137
In some rare cases, IO operations may be not aligned to page boundaries. This prevents iser from performing fast memory registration. In order to overcome that iser uses a bounce buffer to carry the transaction. We basically allocate a buffer in the size of the transaction and perform a copy. The buffer allocation using kmalloc is too restrictive since it requires higher order (atomic) allocations for large transactions (which may result in memory exhaustion fairly fast for some workloads). We rewrite the bounce buffer code path to allocate scattered pages and perform a copy between the transaction sg and the bounce sg. Reported-by: Alex Lyakas <alex@zadarastorage.com> Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2015-04-15IB/iser: Bump version to 1.6Sagi Grimberg1-1/+1
Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2015-04-15IB/iser: Remove code duplication for a single DMA entrySagi Grimberg1-27/+21
In singleton scatterlists, DMA memory registration code is taken both for Fastreg and FMR code paths. Move it to a function. This patch does not change any functionality. Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Adir Lev <adirl@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2015-04-15IB/iser: Pass struct iser_mem_reg to iser_fast_reg_mr and iser_reg_sig_mrSagi Grimberg1-43/+32
Instead of passing ib_sge as output variable, we pass the mem_reg pointer to have the routines fill the rkey as well. This reduces code duplication and extra assignments. This is a preparation step to unify some registration logics together. Also, pass iser_fast_reg_mr the fastreg descriptor directly. This patch does not change any functionality. Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Adir Lev <adirl@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2015-04-15IB/iser: Modify struct iser_mem_reg membersSagi Grimberg3-33/+29
No need to keep lkey, va, len variables, we can keep them as struct ib_sge. This will help when we change the memory registration logic. This patch does not change any functionality. Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Adir Lev <adirl@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2015-04-15IB/iser: Make fastreg pool cache friendlySagi Grimberg1-1/+1
Memory regions are resources that are saved in the device caches. Increase the probability for a cache hit by adding the MRU descriptor to pool head. Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2015-04-15IB/iser: Move PI context alloc/free to routinesSagi Grimberg1-55/+63
Make iser_[create|destroy]_fastreg_desc shorter, more readable and easily extendable. This patch does not change any functionality. Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Adir Lev <adirl@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2015-04-15IB/iser: Move fastreg descriptor pool get/put to helper functionsSagi Grimberg2-18/+37
Instead of open-coding connection fastreg pool get/put, we introduce iser_reg_desc[get|put] helpers. We aren't setting these static as this will be a per-device routine later on. Also, cleanup iser_unreg_rdma_mem_fastreg a bit. This patch does not change any functionality. Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Adir Lev <adirl@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2015-04-15IB/iser: Merge build page-vec into register page-vecSagi Grimberg1-58/+33
No need for these two separate. Keep it in a single routine like in the fastreg case. This will also make iser_reg_page_vec closer to iser_fast_reg_mr arguments. This is a preparation step for registration flow refactor. This patch does not change any functionality. Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Adir Lev <adirl@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2015-04-15IB/iser: Get rid of struct iser_rdma_regdSagi Grimberg4-70/+53
This struct members other than struct iser_mem_reg are unused, so remove it altogether. This patch does not change any functionality. Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Adir Lev <adirl@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2015-04-15IB/iser: Remove redundant assignments in iser_reg_page_vecSagi Grimberg1-5/+2
Buffer length was assigned twice, and no reason to set va to io_addr and then add the offset, just set va to io_addr + offset. This patch does not change any functionality. Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Adir Lev <adirl@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2015-04-15IB/iser: Move memory reg/dereg routines to iser_memory.cSagi Grimberg3-91/+88
As memory registration/de-registration methods, lets move them to their natural location. While we're at it, make iser_reg_page_vec routine static. This patch does not change any functionality. Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2015-04-15IB/iser: Don't pass ib_device to fall_to_bounce_buff routineSagi Grimberg1-6/+6
No need to pass that, we can take it from the task. In a later stage, this function will be invoked according to a device capability. This patch does not change any functionality. Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Adir Lev <adirl@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2015-04-15IB/iser: Remove a redundant struct iser_data_bufSagi Grimberg3-52/+34
No need to keep two iser_data_buf structures just in case we use mem copy. We can avoid that just by adding a pointer to the original sg. So keep only two iser_data_buf per command (data and protection) and pass the relevant data_buf to bounce buffer routine. This patch does not change any functionality. Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Adir Lev <adirl@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2015-04-15IB/iser: Remove redundant cmd_data_len calculationSagi Grimberg1-4/+1
This code was added before we had protection data length calculation (in iser_send_command), so we needed to calc the sg data length from the sg itself. This is not needed anymore. This patch does not change any functionality. Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Adir Lev <adirl@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2015-04-15IB/iser: Fix wrong calculation of protection buffer lengthSagi Grimberg1-2/+2
This length miss-calculation may cause a silent data corruption in the DIX case and cause the device to reference unmapped area. Fixes: d77e65350f2d ('libiscsi, iser: Adjust data_length to include protection information') Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2015-04-15IB/iser: Handle fastreg/local_inv completion errorsSagi Grimberg1-5/+6
Fast registration and local invalidate work requests can also fail. We should call error completion handler for them. Reported-by: Roi Dayan <roid@mellanox.com> Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2015-04-15IB/iser: Fix unload during ep_poll wrong dereferenceSagi Grimberg1-1/+1
In case the user unloaded ib_iser while ep_connect is in progress, we need to destroy the endpoint although ep_disconnect wasn't invoked (we detect this by the iser conn state != DOWN). However, if we got an REJECTED/UNREACHABLE CM event we move the connection state to DOWN which will prevent us from destroying the endpoint in the module unload stage. Fix this by setting the connection state to TERMINATING in iser_conn_error so we can still destroy the endpoint at unload stage. Reported-by: Ariel Nahum <arieln@mellanox.com> Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Doug Ledford <dledford@redhat.com>
2015-02-17IB/iser: Release the iscsi endpoint if ep_disconnect wasn't calledAriel Nahum1-0/+2
In some cases, we might reach the iser connection termination without ep_disconnect being invoked (for example if user-space daemon doesn't exists. In this case, we need to free the iscsi endpoint when we remove the iser connection. Signed-off-by: Ariel Nahum <arieln@mellanox.com> Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2015-02-17IB/iser: Fix memory regions possible leakSagi Grimberg2-15/+14
When teardown process starts during live IO, we need to keep the memory regions pool (frmr/fmr) until all in-flight tasks are properly released, since each task may return a memory region to the pool. In order to do this, we pass a destroy flag to iser_free_ib_conn_res to indicate we can destroy the device and the memory regions pool. iser_conn_release will pass it as true and also DEVICE_REMOVAL event (we need to let the device to properly remove). Also, Since we conditionally call iser_free_rx_descriptors, remove the extra check on iser_conn->rx_descs. Fixes: 5426b1711fd0 ("IB/iser: Collapse cleanup and disconnect handlers") Reported-by: Or Gerlitz <ogerlitz@mellanox.com> Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2015-02-13IB/iser: Use correct dma direction when unmapping SGsRoi Dayan3-8/+17
We always unmap SGs with the same direction instead of unmapping with the direction the mapping was done, fix that. Fixes: 9a8b08fad2ef ("IB/iser: Generalize iser_unmap_task_data and [...]") Signed-off-by: Roi Dayan <roid@mellanox.com> Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2014-12-15IB/iser: Bump version to 1.5Or Gerlitz1-1/+1
Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2014-12-15IB/iser: DIX updateSagi Grimberg2-57/+26
Following few recent Block integrity updates, we align the iSER data integrity offload settings with: - Deprecate pi_guard module param - Expose support for DIX type 0. - Use scsi_transfer_length for the transfer length - Get pi_interval, ref_tag, ref_remap, bg_type and check_mask setting from scsi_cmnd Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2014-12-15IB/iser: Micro-optimize iser_handle_wcSagi Grimberg1-1/+1
Use likely() for wc.status == IB_WC_SUCCESS Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2014-12-15IB/iser: Micro-optimize iser loggingSagi Grimberg1-8/+5
And fix a checkpatch warning. Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2014-12-15IB/iser: Use more completion queuesSagi Grimberg2-4/+12
No reason to settle with four, can use the min between device max comp vectors and number of cores. Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2014-12-15IB/iser: Remove redundant is_mr indicatorSagi Grimberg3-13/+4
It is enough to check mem_h pointer assignment, mem_h == NULL will indicate that buffer is not registered using mr. Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2014-12-15IB/iser: Centralize memory region invalidation to a functionSagi Grimberg1-16/+16
Eliminates code duplication. Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2014-12-15IB/iser: Terminate connection before cleaning inflight tasksSagi Grimberg1-1/+1
When closing the connection, we should first terminate the connection (in case it was not previously terminated) to guarantee the QP is in error state and we are done with servicing IO. Only then go ahead with tasks cleanup via iscsi_conn_stop. Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2014-12-15IB/iser: Fix race between iser connection teardown and scsi TMFsSagi Grimberg1-8/+41
In certain scenarios (target kill with live IO) scsi TMFs may race with iser RDMA teardown, which might cause NULL dereference on iser IB device handle (which might have been freed). In this case we take a conditional lock for TMFs and check the connection state (avoid introducing lock contention in the IO path). This is indeed best effort approach, but sufficient to survive multi targets sudden death while heavy IO is inflight. While we are on it, add a nice kernel-doc style documentation. Reported-by: Ariel Nahum <arieln@mellanox.com> Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2014-12-15IB/iser: Fix possible NULL derefernce ib_conn->device in session_createAriel Nahum1-4/+18
If rdma_cm error event comes after ep_poll but before conn_bind, we should protect against dereferncing the device (which may have been terminated) in session_create and conn_create (already protected) callbacks. Signed-off-by: Ariel Nahum <arieln@mellanox.com> Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2014-12-15IB/iser: Fix sparse warningsSagi Grimberg2-8/+8
Use uintptr_t to handle wr_id casting, which was found by Kbuild test robot and smatch. Also remove an internal definition of variable which potentially shadows an external one (and make sparse happy). Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2014-12-15IB/iser: Fix possible SQ overflowMax Gurtovoy2-3/+5
Fix a regression was introduced in commit 6df5a128f0fd ("IB/iser: Suppress scsi command send completions"). The sig_count was wrongly set to be static variable, thus it is possible that we won't reach to (sig_count % ISER_SIGNAL_BATCH) == 0 condition (due to races) and the send queue will be overflowed. Instead keep sig_count per connection. We don't need it to be atomic as we are safe under the iscsi session frwd_lock taken by libiscsi on the queuecommand path. Fixes: 6df5a128f0fd ("IB/iser: Suppress scsi command send completions") Signed-off-by: Max Gurtovoy <maxg@mellanox.com> Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2014-12-15IB/iser: Decrement CQ's active QPs accounting when QP creation failsSagi Grimberg1-0/+4
When creating a connection QP we choose the least used CQ and inc the number of active QPs on that. If we fail to create the QP, we need to decrement the active QPs counter. Reported-by: Roi Dayan <roid@mellanox.com> Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2014-12-15IB/iser: Collapse cleanup and disconnect handlersAriel Nahum1-8/+11
No real need to wait for TIMEWAIT_EXIT before we destroy the RDMA resources (also TIMEAWAIT_EXIT is not guarenteed to always arrive). As for the cma_id, only destroy it if the state is not DOWN where in this case, conn_release is already running and we don't want to compete. Signed-off-by: Ariel Nahum <arieln@mellanox.com> Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2014-12-15IB/iser: Fix catastrophic error flow hangSagi Grimberg1-1/+3
In case of the HCA going into catasrophic error flow, the beacon post_send is likely to fail, so surely there will be no completion for it. In this case, use a best effort approach and don't wait for beacon completion if we failed to post the send. Reported-by: Alex Tabachnik <alext@mellanox.com> Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2014-12-15IB/iser: Re-adjust CQ and QP send ring sizes to HW limitsMinh Tran3-8/+38
Re-adjust max CQEs per CQ and max send_wr per QP according to the resource limits supported by underlying hardware. Signed-off-by: Minh Tran <minhduc.tran@emulex.com> Signed-off-by: Jayamohan Kallickal <jayamohan.kallickal@emulex.com> Acked-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2014-11-24scsi: drop reason argument from ->change_queue_depthChristoph Hellwig1-1/+1
Drop the now unused reason argument from the ->change_queue_depth method. Also add a return value to scsi_adjust_queue_depth, and rename it to scsi_change_queue_depth now that it can be used as the default ->change_queue_depth implementation. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Mike Christie <michaelc@cs.wisc.edu> Reviewed-by: Hannes Reinecke <hare@suse.de>
2014-11-24scsi: avoid ->change_queue_depth indirection for queue full trackingChristoph Hellwig1-0/+1
All drivers use the implementation for ramping the queue up and down, so instead of overloading the change_queue_depth method call the implementation diretly if the driver opts into it by setting the track_queue_depth flag in the host template. Note that a few drivers validated the new queue depth in their change_queue_depth method, but as we never go over the queue depth set during slave_configure or the sysfs file this isn't nessecary and can safely be removed. Signed-off-by: Christoph Hellwig <hch@lst.de> Reviewed-by: Mike Christie <michaelc@cs.wisc.edu> Reviewed-by: Hannes Reinecke <hare@suse.de> Reviewed-by: Venkatesh Srinivas <venkateshs@google.com>
2014-10-09IB/mlx5, iser, isert: Add Signature API additionsSagi Grimberg1-26/+24
Expose more signature setting parameters. We modify the signature API to allow usage of some new execution parameters relevant to data integrity feature. This patch modifies ib_sig_domain structure by: - Deprecate DIF type in signature API (operation will be determined by the parameters alone, no DIF type awareness) - Add APPTAG check bitmask (for input domain) - Add REFTAG remap (increment) flag for each domain - Add APPTAG/REFTAG escape options for each domain The mlx5 driver is modified to follow the new parameters in HW signature setup. At the moment the callers (iser/isert) hard-code new parameters (by DIF type). In the future, callers will retrieve them from the scsi command structure. Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2014-10-09IB/iser: Centralize ib_sig_domain settingsSagi Grimberg1-18/+18
Later there will be more parameters to set, so we want to do it in a centralized place. This patch does not change any functionality. Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2014-10-09IB/iser: Set IP_CSUM as default guard typeSagi Grimberg1-2/+2
In the future this will be a per-command parameter so we can lose it, but in the mean time IP_CSUM is a lot lighter for SW layers to compute, set it as default. Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2014-10-09IB/iser: Remove redundant assignmentSagi Grimberg1-1/+0
We clear the struct before - no need to do 0 assignment. Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2014-10-09IB/iser: Bump version, add maintainerOr Gerlitz1-1/+1
Update the driver version and add Sagi Grimberg as maintainer Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2014-10-09IB/iser: Fix/add kernel-doc style description in iscsi_iser.cSagi Grimberg1-9/+160
This patch does not change any functionality. Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2014-10-09IB/iser: Add/Fix kernel doc style descriptions in iscsi_iser.hSagi Grimberg1-46/+189
- iser_hdr - iser_data_buf - iser_mem_reg - iser_regd_buf - iser_tx_desc - iser_rx_desc - iser_device - iser_pi_context - iser_conn - ib_conn - iser_comp - iscsi_iser_task - iser_global While we're at it, change nit alignments in this file This patch does not change any functionality. Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2014-10-09IB/iser: Nit - add space after __func__ in iser loggingSagi Grimberg1-8/+8
Change logging: "iser:XXXX" to "iser: XXXX" Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2014-10-09IB/iser: Change iscsi_conn_stop log level to infoAriel Nahum1-1/+1
Match to the debug level of all functions in connect/disconnect flows. Signed-off-by: Ariel Nahum <arieln@mellanox.com> Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2014-10-09IB/iser: Suppress scsi command send completionsSagi Grimberg3-7/+16
Singal completion of every 32 scsi commands and suppress all the rest. We don't do anything upon getting the completion so no need to "just consume" it. Cleanup of scsi command is done in cleanup_task callback. Still keep dataout and control send completions as we may need to cleanup there. This helps reducing the amount of interrupts/completions in the IO path. Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
2014-10-09IB/iser: Optimize completion pollingSagi Grimberg2-5/+11
Poll in batch of 16. Since we don't want it on the stack, keep under iser completion context (iser_comp). Signed-off-by: Sagi Grimberg <sagig@mellanox.com> Signed-off-by: Or Gerlitz <ogerlitz@mellanox.com> Signed-off-by: Roland Dreier <roland@purestorage.com>
Powered by cgit, Gruvboxed by Blaster4385.