| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
This patch implement a page fault handler (leaving the pages pinned as
of time being). The page fault handler handles initiator and responder
page faults for UD/RC transports, for send/receive operations, as well
as RDMA read/write initiator support.
Signed-off-by: Sagi Grimberg <[email protected]>
Signed-off-by: Shachar Raindel <[email protected]>
Signed-off-by: Haggai Eran <[email protected]>
Signed-off-by: Roland Dreier <[email protected]>
|
|
* Refactor MR registration and cleanup, and fix reg_pages accounting.
* Create a work queue to handle page fault events in a kthread context.
* Register a fault handler to get events from the core for each QP.
The registered fault handler is empty in this patch, and only a later
patch implements it.
Signed-off-by: Sagi Grimberg <[email protected]>
Signed-off-by: Shachar Raindel <[email protected]>
Signed-off-by: Haggai Eran <[email protected]>
Signed-off-by: Roland Dreier <[email protected]>
|
|
The new function allows updating the page tables of a memory region
after it was created. This can be used to handle page faults and page
invalidations.
Since mlx5_ib_update_mtt will need to work from within page invalidation,
so it must not block on memory allocation. It employs an atomic memory
allocation mechanism that is used as a fallback when kmalloc(GFP_ATOMIC) fails.
In order to reuse code from mlx5_ib_populate_pas, the patch splits
this function and add the needed parameters.
Signed-off-by: Haggai Eran <[email protected]>
Signed-off-by: Shachar Raindel <[email protected]>
Signed-off-by: Roland Dreier <[email protected]>
|
|
This patch wraps together several changes needed for on-demand paging support
in the mlx5_ib_populate_pas function, and when registering memory regions.
* Instead of accepting a UMR bit telling the function to enable all
access flags, the function now accepts the access flags themselves.
* For on-demand paging memory regions, fill the memory tables from the
correct list, and enable/disable the access flags per-page according
to whether the page is present.
* A new bit is set to enable writing of access flags when using the
firmware create_mkey command.
* Disable contig pages when on-demand paging is enabled.
In addition the patch changes the UMR code to use PTR_ALIGN instead of
our own macro.
Signed-off-by: Haggai Eran <[email protected]>
Signed-off-by: Roland Dreier <[email protected]>
|
|
* Add a handler function pointer in the mlx5_core_qp struct for page
fault events. Handle page fault events by calling the handler
function, if not NULL.
* Add on-demand paging capability query command.
* Export command for resuming QPs after page faults.
* Add various constants related to paging support.
Signed-off-by: Sagi Grimberg <[email protected]>
Signed-off-by: Shachar Raindel <[email protected]>
Signed-off-by: Haggai Eran <[email protected]>
Signed-off-by: Roland Dreier <[email protected]>
|
|
In commit 0c7aac854f52 ("net/mlx5_core: Remove unused dev cap enum
fields"), the flag MLX5_DEV_CAP_FLAG_ON_DMND_PG was removed.
Unfortunately the on-demand paging changes actually use it, so re-add
the missing flag.
Signed-off-by: Roland Dreier <[email protected]>
|
|
Add a helper function mlx5_ib_read_user_wqe to read information from
user-space owned work queues. The function will be used in a later
patch by the page-fault handling code in mlx5_ib.
Signed-off-by: Haggai Eran <[email protected]>
[ Add stub for ib_umem_copy_from() for CONFIG_INFINIBAND_USER_MEM=n
- Roland ]
Signed-off-by: Roland Dreier <[email protected]>
|
|
The current UMR interface doesn't allow partial updates to a memory
region's page tables. This patch changes the interface to allow that.
It also changes the way the UMR operation validates the memory
region's state. When set, IB_SEND_UMR_FAIL_IF_FREE will cause the UMR
operation to fail if the MKEY is in the free state. When it is
unchecked the operation will check that it isn't in the free state.
Signed-off-by: Haggai Eran <[email protected]>
Signed-off-by: Shachar Raindel <[email protected]>
Signed-off-by: Roland Dreier <[email protected]>
|
|
These enumerations are not used so remove them.
Signed-off-by: Eli Cohen <[email protected]>
Signed-off-by: David S. Miller <[email protected]>
|
|
Signed-off-by: Al Viro <[email protected]>
Acked-by: Eli Cohen <[email protected]>
Signed-off-by: David S. Miller <[email protected]>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband
Pull infiniband/RDMA updates from Roland Dreier:
- large set of iSER initiator improvements
- hardware driver fixes for cxgb4, mlx5 and ocrdma
- small fixes to core midlayer
* tag 'rdma-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/roland/infiniband: (47 commits)
RDMA/cxgb4: Fix ntuple calculation for ipv6 and remove duplicate line
RDMA/cxgb4: Add missing neigh_release in find_route
RDMA/cxgb4: Take IPv6 into account for best_mtu and set_emss
RDMA/cxgb4: Make c4iw_wr_log_size_order static
IB/core: Fix XRC race condition in ib_uverbs_open_qp
IB/core: Clear AH attr variable to prevent garbage data
RDMA/ocrdma: Save the bit environment, spare unncessary parenthesis
RDMA/ocrdma: The kernel has a perfectly good BIT() macro - use it
RDMA/ocrdma: Don't memset() buffers we just allocated with kzalloc()
RDMA/ocrdma: Remove a unused-label warning
RDMA/ocrdma: Convert kernel VA to PA for mmap in user
RDMA/ocrdma: Get vlan tag from ib_qp_attrs
RDMA/ocrdma: Add default GID at index 0
IB/mlx5, iser, isert: Add Signature API additions
Target/iser: Centralize ib_sig_domain setting
IB/iser: Centralize ib_sig_domain settings
IB/mlx5: Use extended internal signature layout
IB/iser: Set IP_CSUM as default guard type
IB/iser: Remove redundant assignment
IB/mlx5: Use enumerations for PI copy mask
...
|
|
Rather than using the basic BSF layout which utilizes a pre-configured
signature settings (sufficient for current DIF implementation), we use
the extended BSF layout to expose advanced signature settings. These
settings will also be exposed to the user later.
Signed-off-by: Sagi Grimberg <[email protected]>
Signed-off-by: Roland Dreier <[email protected]>
|
|
In case input and output space parameters match, we can use a copy
mask from input and output space. Use enums for those.
Signed-off-by: Sagi Grimberg <[email protected]>
Signed-off-by: Roland Dreier <[email protected]>
|
|
This patch puts a common part as the first field of mlx5_core_qp. This field is
used to identify which resource generated an event. This is required since upcoming
new resource types such as DC targets are allocated for the same numerical space
as regular QPs and may generate the same events. By searching the resource in the
same table we can then look at the common field to identify the resource.
Signed-off-by: Eli Cohen <[email protected]>
Signed-off-by: David S. Miller <[email protected]>
|
|
Transform device capabilities related commands to use set/get macros to
manipulate command mailboxes.
Signed-off-by: Eli Cohen <[email protected]>
Signed-off-by: David S. Miller <[email protected]>
|
|
Add an auto generated header file that describes hardware registers along with
set of macros that set/get values. The macros do static checks to avoid
overflow, handle endianess, and overall provide a clean way to code commands.
Currently the header file is small and we will add structs as we make use of
the macros.
A few commands were removed from the commands enum since they are not supported
currently and will be added when support is available.
Signed-off-by: Eli Cohen <[email protected]>
Signed-off-by: David S. Miller <[email protected]>
|
|
Rearrange struct mlx5_caps so it has a "gen" field to represent the current
capabilities configured for the device. Max capabilities can also be queried
from the device. Also update capabilities struct to contain more fields as per
the latest revision if firmware specification.
Signed-off-by: Eli Cohen <[email protected]>
Signed-off-by: David S. Miller <[email protected]>
|
|
Pull networking updates from David Miller:
"Highlights:
1) Steady transitioning of the BPF instructure to a generic spot so
all kernel subsystems can make use of it, from Alexei Starovoitov.
2) SFC driver supports busy polling, from Alexandre Rames.
3) Take advantage of hash table in UDP multicast delivery, from David
Held.
4) Lighten locking, in particular by getting rid of the LRU lists, in
inet frag handling. From Florian Westphal.
5) Add support for various RFC6458 control messages in SCTP, from
Geir Ola Vaagland.
6) Allow to filter bridge forwarding database dumps by device, from
Jamal Hadi Salim.
7) virtio-net also now supports busy polling, from Jason Wang.
8) Some low level optimization tweaks in pktgen from Jesper Dangaard
Brouer.
9) Add support for ipv6 address generation modes, so that userland
can have some input into the process. From Jiri Pirko.
10) Consolidate common TCP connection request code in ipv4 and ipv6,
from Octavian Purdila.
11) New ARP packet logger in netfilter, from Pablo Neira Ayuso.
12) Generic resizable RCU hash table, with intial users in netlink and
nftables. From Thomas Graf.
13) Maintain a name assignment type so that userspace can see where a
network device name came from (enumerated by kernel, assigned
explicitly by userspace, etc.) From Tom Gundersen.
14) Automatic flow label generation on transmit in ipv6, from Tom
Herbert.
15) New packet timestamping facilities from Willem de Bruijn, meant to
assist in measuring latencies going into/out-of the packet
scheduler, latency from TCP data transmission to ACK, etc"
* git://git.kernel.org/pub/scm/linux/kernel/git/davem/net-next: (1536 commits)
cxgb4 : Disable recursive mailbox commands when enabling vi
net: reduce USB network driver config options.
tg3: Modify tg3_tso_bug() to handle multiple TX rings
amd-xgbe: Perform phy connect/disconnect at dev open/stop
amd-xgbe: Use dma_set_mask_and_coherent to set DMA mask
net: sun4i-emac: fix memory leak on bad packet
sctp: fix possible seqlock seadlock in sctp_packet_transmit()
Revert "net: phy: Set the driver when registering an MDIO bus device"
cxgb4vf: Turn off SGE RX/TX Callback Timers and interrupts in PCI shutdown routine
team: Simplify return path of team_newlink
bridge: Update outdated comment on promiscuous mode
net-timestamp: ACK timestamp for bytestreams
net-timestamp: TCP timestamping
net-timestamp: SCHED timestamp on entering packet scheduler
net-timestamp: add key to disambiguate concurrent datagrams
net-timestamp: move timestamp flags out of sk_flags
net-timestamp: extend SCM_TIMESTAMPING ancillary data struct
cxgb4i : Move stray CPL definitions to cxgb4 driver
tcp: reduce spurious retransmits due to transient SACK reneging
qlcnic: Initialize dcbnl_ops before register_netdev
...
|
|
In the event flow, we currently pass only a port number in the
void *data argument. Rather than pass a pointer to the event handlers,
we should use an "unsigned long" parameter, and pass the port number
value directly.
In the future, if necessary for some events, we can use the unsigned long
parameter to pass a pointer.
Based on a patch by Eli Cohen <[email protected]>
Signed-off-by: Jack Morgenstein <[email protected]>
Signed-off-by: Eli Cohen <[email protected]>
Signed-off-by: David S. Miller <[email protected]>
|
|
There were many places where parameters which should be u8/u16 were
integer type.
Additionally, in 2 places, a check for a non-null pointer was added
before dereferencing the pointer (this is actually a bug fix).
Signed-off-by: Jack Morgenstein <[email protected]>
Signed-off-by: Eli Cohen <[email protected]>
Signed-off-by: David S. Miller <[email protected]>
|
|
In preparation for a new mlx5 device which is VPI (i.e., ports can be
either IB or ETH), move the pci device functionality from mlx5_ib
to mlx5_core.
This involves the following changes:
1. Move mlx5_core_dev struct out of mlx5_ib_dev. mlx5_core_dev
is now an independent structure maintained by mlx5_core.
mlx5_ib_dev now has a pointer to that struct.
This requires changing a lot of places where the core_dev
struct was accessed via mlx5_ib_dev (now, this needs to
be a pointer dereference).
2. All PCI initializations are now done in mlx5_core. Thus,
it is now mlx5_core which does pci_register_device (and not
mlx5_ib, as was previously).
3. mlx5_ib now registers itself with mlx5_core as an "interface"
driver. This is very similar to the mechanism employed for
the mlx4 (ConnectX) driver. Once the HCA is initialized
(by mlx5_core), it invokes the interface drivers to do
their initializations.
4. There is a new event handler which the core registers:
mlx5_core_event(). This event handler invokes the
event handlers registered by the interfaces.
Based on a patch by Eli Cohen <[email protected]>
Signed-off-by: Jack Morgenstein <[email protected]>
Signed-off-by: Eli Cohen <[email protected]>
Signed-off-by: David S. Miller <[email protected]>
|
|
This code is beyond silly:
struct timespec ts = ktime_get_ts();
ktime_t ktime = timespec_to_ktime(ts);
Further down the code builds the delta of two ktime_t values and
converts the result to nanoseconds.
Use ktime_get_ns() and replace all the nonsense.
Signed-off-by: Thomas Gleixner <[email protected]>
Cc: Eli Cohen <[email protected]>
Signed-off-by: John Stultz <[email protected]>
|
|
The patch stores iova, pd and size during mr creation and after UMRs
that modify them. It removes the unused access flags field.
Signed-off-by: Haggai Eran <[email protected]>
Signed-off-by: Roland Dreier <[email protected]>
|
|
Add support for the block multicast loopback QP creation flag along
the proper firmware API for that.
Signed-off-by: Eli Cohen <[email protected]>
Signed-off-by: Or Gerlitz <[email protected]>
Signed-off-by: Roland Dreier <[email protected]>
|
|
This commit takes care of the generated signature error CQE generated
by the HW (if happened). The underlying mlx5 driver will handle
signature error completions and will mark the relevant memory region
as dirty.
Once the consumer gets the completion for the transaction, it must
check for signature errors on signature memory region using a new
lightweight verb ib_check_mr_status().
In case the user doesn't check for signature error (i.e. doesn't call
ib_check_mr_status() with status check IB_MR_CHECK_SIG_STATUS), the
memory region cannot be used for another signature operation
(REG_SIG_MR work request will fail).
Signed-off-by: Sagi Grimberg <[email protected]>
Signed-off-by: Roland Dreier <[email protected]>
|
|
This patch implements IB_WR_REG_SIG_MR posted by the user.
Baisically this WR involves 3 WQEs in order to prepare and properly
register the signature layout:
1. post UMR WR to register the sig_mr in one of two possible ways:
* In case the user registered a single MR for data so the UMR data segment
consists of:
- single klm (data MR) passed by the user
- BSF with signature attributes requested by the user.
* In case the user registered 2 MRs, one for data and one for protection,
the UMR consists of:
- strided block format which includes data and protection MRs and
their repetitive block format.
- BSF with signature attributes requested by the user.
2. post SET_PSV in order to set the memory domain initial
signature parameters passed by the user.
SET_PSV is not signaled and solicited CQE.
3. post SET_PSV in order to set the wire domain initial
signature parameters passed by the user.
SET_PSV is not signaled and solicited CQE.
* After this compound WR we place a small fence for next WR to come.
This patch also introduces some helper functions to set the BSF correctly
and determining the signature format selectors.
Signed-off-by: Sagi Grimberg <[email protected]>
Signed-off-by: Roland Dreier <[email protected]>
|
|
This will be useful when processing signature errors on a specific
key. The mlx5 driver will lookup the matching mlx5 memory region
structure and mark it as dirty (contains signature errors).
Signed-off-by: Sagi Grimberg <[email protected]>
Signed-off-by: Roland Dreier <[email protected]>
|
|
If user requested signature enable we initialize relevant mlx5_ib_qp
members. We mark the qp as sig_enable and we increase the effective
SQ size, but still limit the user max_send_wr to original size
computed. We also allow the create_qp routine to accept sig_enable
create flag.
Signed-off-by: Sagi Grimberg <[email protected]>
Signed-off-by: Roland Dreier <[email protected]>
|
|
Support create_mr and destroy_mr verbs. Creating ib_mr may be done
for either ib_mr that will register regular page lists like
alloc_fast_reg_mr routine, or indirect ib_mrs that can register other
(pre-registered) ib_mrs in an indirect manner.
In addition user may request signature enable, that will mean that the
created ib_mr may be attached with signature attributes (BSF, PSVs).
Currently we only allow direct/indirect registration modes.
Signed-off-by: Sagi Grimberg <[email protected]>
Signed-off-by: Roland Dreier <[email protected]>
|
|
On some architectures (for example, arm), we don't end up indirectly
pulling in the declaration of kzalloc() and kfree(), and so building
anything that includes <linux/mlx5/driver.h> breaks. Fix this by adding
an explicit include to get the declaration.
Reported-by: kbuild test robot <[email protected]>
Signed-off-by: Roland Dreier <[email protected]>
|
|
Commit c1be5232d21d ("Fix micro UAR allocator") broke binary compatibility
between libmlx5 and mlx5_ib since it defines a different value to the number
of micro UARs per page, leading to wrong calculation in libmlx5. This patch
defines struct mlx5_ib_alloc_ucontext_req_v2 as an extension to struct
mlx5_ib_alloc_ucontext_req. The extended size is determined in mlx5_ib_alloc_ucontext()
and in case of old library we use uuarn 0 which works fine -- this is
acheived due to create_user_qp() falling back from high to medium then to
low class where low class will return 0. For new libraries we use the
more sophisticated allocation algorithm.
Signed-off-by: Eli Cohen <[email protected]>
Reviewed-by: Yann Droneaud <[email protected]>
Signed-off-by: Roland Dreier <[email protected]>
|
|
Match firmware specification.
Signed-off-by: Eli Cohen <[email protected]>
Signed-off-by: Roland Dreier <[email protected]>
|
|
1. Fix derivation of sub-page index from the dma address in free_4k.
2. Fix the DMA address passed to dma_unmap_page by masking it properly.
Signed-off-by: Eli Cohen <[email protected]>
Signed-off-by: Roland Dreier <[email protected]>
|
|
Use strings to display transport service or state of QPs. Use numeric
value for MTU of a QP.
Signed-off-by: Eli Cohen <[email protected]>
Signed-off-by: Roland Dreier <[email protected]>
|
|
Implement resize CQ which is a mandatory verb in mlx5.
Signed-off-by: Eli Cohen <[email protected]>
Signed-off-by: Roland Dreier <[email protected]>
|
|
Modify CQ is used by ULPs like IPoIB to change moderation parameters. This
patch adds support in mlx5.
Signed-off-by: Eli Cohen <[email protected]>
Signed-off-by: Roland Dreier <[email protected]>
|
|
The micro UAR (uuar) allocator had a bug which resulted from the fact
that in each UAR we only have two micro UARs avaialable, those at
index 0 and 1. This patch defines iterators to aid in traversing the
list of available micro UARs when allocating a uuar.
In addition, change the logic in create_user_qp() so that if high
class allocation fails (high class means lower latency), we revert to
medium class and not to the low class.
Signed-off-by: Eli Cohen <[email protected]>
Signed-off-by: Roland Dreier <[email protected]>
|
|
The Connect-IB adapter has an inherent page size which equals 4K.
Define an new enum that equals the page shift and use it instead of
using the value 12 throughout the code.
Signed-off-by: Eli Cohen <[email protected]>
Signed-off-by: Roland Dreier <[email protected]>
|
|
Firmware spec requires reserved fields to be cleared when calling
set_hca_cap. Current code queries and copy to the set area, possibly
resulting in reserved bits not cleared. This patch copies only
writable fields to the set area.
Fix also typo - msx => max
Signed-off-by: Eli Cohen <[email protected]>
Signed-off-by: Roland Dreier <[email protected]>
|
|
Connect-IB firmware requires 4K pages to be communicated with the
driver. This patch breaks larger pages to 4K units to enable support
for architectures utilizing larger page size, such as PowerPC. This
patch also fixes several places that referred to PAGE_SHIFT instead of
explicit 12 which is the inherent page shift on Connect-IB.
Signed-off-by: Eli Cohen <[email protected]>
Signed-off-by: Roland Dreier <[email protected]>
|
|
Use asynchronous commands to execute up to eight concurrent create MR
commands. This is to fill memory caches faster so we keep consuming
from there. Also, increase timeout for shrinking caches to five
minutes.
Signed-off-by: Eli Cohen <[email protected]>
Signed-off-by: Roland Dreier <[email protected]>
|
|
It's helpful for a driver to put the pci slot name in its interrupt
names, so /proc/interrupts will show the pci slot of the device.
Signed-off-by: Sagi Grimberg <[email protected]>
Signed-off-by: Eli Cohen <[email protected]>
Signed-off-by: Roland Dreier <[email protected]>
|
|
The layout of struct health_buffer was not according to firmware
specification. Fix it to comply.
Signed-off-by: Eli Cohen <[email protected]>
Signed-off-by: Roland Dreier <[email protected]>
|
|
Checksum calculations consume CPU resources and can be significant to
the rate of resource creation/destruction.
Signed-off-by: Eli Cohen <[email protected]>
Signed-off-by: Roland Dreier <[email protected]>
|
|
In the previous QUERY_PAGES command version we used one command to get the
required amount of boot, init and post init pages. The new version uses the
op_mod field to specify whether the query is for the required amount of boot,
init or post init pages. In addition the output field size for the required
amount of pages increased from 16 to 32 bits.
In MANAGE_PAGES command the input_num_entries and output_num_entries fields
sizes changed from 16 to 32 bits and the PAS tables offset changed to 0x10.
In the pages request event the num_pages field also changed to 32 bits.
In the HCA-capabilities-layout the size and location of max_qp_mcg field has
been changed to support 24 bits.
This patch isn't compatible with firmware versions < 5; however, it turns out that the
first GA firmware we will publish will not support previous versions so this should be OK.
Signed-off-by: Moshe Lazer <[email protected]>
Signed-off-by: Eli Cohen <[email protected]>
Signed-off-by: David S. Miller <[email protected]>
|
|
Remove this code, per Dave Miller's request, since it is not being used
anywhere in the kernel.
Signed-off-by: Eli Cohen <[email protected]>
Signed-off-by: David S. Miller <[email protected]>
|
|
Introduce enbale_hca and disable_hca commands to signify when the
driver starts or ceases to operate on the device.
In addition the driver will use boot and init pages count; boot pages
is required to allow firmware to complete boot commands and the other
to complete init hca. Command interface revision is bumped to 4 to
enforce using supported firmware.
This patch breaks compatibility with old versions of firmware (< 4);
however, the first GA firmware we will publish will support version 4
so this should not be a problem.
Signed-off-by: Eli Cohen <[email protected]>
Signed-off-by: Roland Dreier <[email protected]>
|
|
Sparse reported an endianness bug in the assignment to hca_cap.uar_page_sz.
Fix the declaration of this field to be __be16 (which is what is in
the firmware spec), renaming the field to log_uar_pg_size to conform
to the spec, which fixes the endianness bug reported by sparse.
Reported-by: Fengguang Wu <[email protected]>
Signed-off-by: Moshe Lazer <[email protected]>
Signed-off-by: Or Gerlitz <[email protected]>
Signed-off-by: Roland Dreier <[email protected]>
|
|
This deals with the sparse warning:
drivers/net/ethernet/mellanox/mlx5/core/health.c:94:54: warning: incorrect type in argument 2 (different address spaces)
drivers/net/ethernet/mellanox/mlx5/core/health.c:94:54: expected void *buf
drivers/net/ethernet/mellanox/mlx5/core/health.c:94:54: got struct health_buffer [noderef] <asn:2>*health
Signed-off-by: Roland Dreier <[email protected]>
|
|
The driver is comprised of two kernel modules: mlx5_ib and mlx5_core.
This partitioning resembles what we have for mlx4, except that mlx5_ib
is the pci device driver and not mlx5_core.
mlx5_core is essentially a library that provides general functionality
that is intended to be used by other Mellanox devices that will be
introduced in the future. mlx5_ib has a similar role as any hardware
device under drivers/infiniband/hw.
Signed-off-by: Eli Cohen <[email protected]>
Signed-off-by: Jack Morgenstein <[email protected]>
Signed-off-by: Or Gerlitz <[email protected]>
[ Merge in coccinelle fixes from Fengguang Wu <[email protected]>.
- Roland ]
Signed-off-by: Roland Dreier <[email protected]>
|