aboutsummaryrefslogtreecommitdiff
path: root/drivers/net/ethernet/intel/ice/ice_xsk.c
AgeCommit message (Collapse)AuthorFilesLines
2020-08-31xsk: i40e: ice: ixgbe: mlx5: Test for dma_need_sync earlier for better ↵Magnus Karlsson1-1/+1
performance Test for dma_need_sync earlier to increase performance. xsk_buff_dma_sync_for_cpu() takes an xdp_buff as parameter and from that the xsk_buff_pool reference is dug out. Perf shows that this dereference causes a lot of cache misses. But as the buffer pool is now sent down to the driver at zero-copy initialization time, we might as well use this pointer directly, instead of going via the xsk_buff and we can do so already in xsk_buff_dma_sync_for_cpu() instead of in xp_dma_sync_for_cpu. This gets rid of these cache misses. Throughput increases with 3% for the xdpsock l2fwd sample application on my machine. Signed-off-by: Magnus Karlsson <[email protected]> Signed-off-by: Daniel Borkmann <[email protected]> Acked-by: Björn Töpel <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2020-08-31xsk: i40e: ice: ixgbe: mlx5: Rename xsk zero-copy driver interfacesMagnus Karlsson1-14/+14
Rename the AF_XDP zero-copy driver interface functions to better reflect what they do after the replacement of umems with buffer pools in the previous commit. Mostly it is about replacing the umem name from the function names with xsk_buff and also have them take the a buffer pool pointer instead of a umem. The various ring functions have also been renamed in the process so that they have the same naming convention as the internal functions in xsk_queue.h. This so that it will be clearer what they do and also for consistency. Signed-off-by: Magnus Karlsson <[email protected]> Signed-off-by: Daniel Borkmann <[email protected]> Acked-by: Björn Töpel <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2020-08-31xsk: i40e: ice: ixgbe: mlx5: Pass buffer pool to driver instead of umemMagnus Karlsson1-68/+68
Replace the explicit umem reference passed to the driver in AF_XDP zero-copy mode with the buffer pool instead. This in preparation for extending the functionality of the zero-copy mode so that umems can be shared between queues on the same netdev and also between netdevs. In this commit, only an umem reference has been added to the buffer pool struct. But later commits will add other entities to it. These are going to be entities that are different between different queue ids and netdevs even though the umem is shared between them. Signed-off-by: Magnus Karlsson <[email protected]> Signed-off-by: Daniel Borkmann <[email protected]> Acked-by: Björn Töpel <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2020-08-01ice: Misc minor fixesTony Nguyen1-2/+0
This is a collection of minor fixes including typos, white space, and style. No functional changes. Signed-off-by: Tony Nguyen <[email protected]> Tested-by: Andrew Bowers <[email protected]>
2020-07-29ice: need_wakeup flag might not be set for TxKrzysztof Kazimierczak1-8/+2
This is a port of i40e commit 705639572e8c ("i40e: need_wakeup flag might not be set for Tx"). Quoting the original commit message: "The need_wakeup flag for Tx might not be set for AF_XDP sockets that are only used to send packets. This happens if there is at least one outstanding packet that has not been completed by the hardware and we get that corresponding completion (which will not generate an interrupt since interrupts are disabled in the napi poll loop) between the time we stopped processing the Tx completions and interrupts are enabled again. In this case, the need_wakeup flag will have been cleared at the end of the Tx completion processing as we believe we will get an interrupt from the outstanding completion at a later point in time. But if this completion interrupt occurs before interrupts are enable, we lose it and should at that point really have set the need_wakeup flag since there are no more outstanding completions that can generate an interrupt to continue the processing. When this happens, user space will see a Tx queue need_wakeup of 0 and skip issuing a syscall, which means will never get into the Tx processing again and we have a deadlock." As a result, packet processing stops. This patch introduces a fix for this issue, by always setting the need_wakeup flag at the end of an interrupt processing. This ensures that the deadlock will not happen. Signed-off-by: Krzysztof Kazimierczak <[email protected]> Tested-by: Andrew Bowers <[email protected]> Signed-off-by: Tony Nguyen <[email protected]>
2020-07-01ice: replace single-element array used for C struct hackBruce Allan1-2/+4
Convert the pre-C90-extension "C struct hack" method (using a single- element array at the end of a structure for implementing variable-length types) to the preferred use of C99 flexible array member. Additional code cleanups were done near areas affected by this change. Signed-off-by: Bruce Allan <[email protected]> Tested-by: Andrew Bowers <[email protected]> Signed-off-by: Tony Nguyen <[email protected]>
2020-05-22Merge git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpf-nextDavid S. Miller1-337/+41
Daniel Borkmann says: ==================== pull-request: bpf-next 2020-05-23 The following pull-request contains BPF updates for your *net-next* tree. We've added 50 non-merge commits during the last 8 day(s) which contain a total of 109 files changed, 2776 insertions(+), 2887 deletions(-). The main changes are: 1) Add a new AF_XDP buffer allocation API to the core in order to help lowering the bar for drivers adopting AF_XDP support. i40e, ice, ixgbe as well as mlx5 have been moved over to the new API and also gained a small improvement in performance, from Björn Töpel and Magnus Karlsson. 2) Add getpeername()/getsockname() attach types for BPF sock_addr programs in order to allow for e.g. reverse translation of load-balancer backend to service address/port tuple from a connected peer, from Daniel Borkmann. 3) Improve the BPF verifier is_branch_taken() logic to evaluate pointers being non-NULL, e.g. if after an initial test another non-NULL test on that pointer follows in a given path, then it can be pruned right away, from John Fastabend. 4) Larger rework of BPF sockmap selftests to make output easier to understand and to reduce overall runtime as well as adding new BPF kTLS selftests that run in combination with sockmap, also from John Fastabend. 5) Batch of misc updates to BPF selftests including fixing up test_align to match verifier output again and moving it under test_progs, allowing bpf_iter selftest to compile on machines with older vmlinux.h, and updating config options for lirc and v6 segment routing helpers, from Stanislav Fomichev, Andrii Nakryiko and Alan Maguire. 6) Conversion of BPF tracing samples outdated internal BPF loader to use libbpf API instead, from Daniel T. Lee. 7) Follow-up to BPF kernel test infrastructure in order to fix a flake in the XDP selftests, from Jesper Dangaard Brouer. 8) Minor improvements to libbpf's internal hashmap implementation, from Ian Rogers. ==================== Signed-off-by: David S. Miller <[email protected]>
2020-05-21ice: Rename build_ctob to ice_build_ctobTony Nguyen1-2/+2
To make the function easier to identify as being part of the ice driver, prepend ice to the function name. Signed-off-by: Tony Nguyen <[email protected]> Tested-by: Andrew Bowers <[email protected]> Signed-off-by: Jeff Kirsher <[email protected]>
2020-05-21ice, xsk: Migrate to new MEM_TYPE_XSK_BUFF_POOLBjörn Töpel1-336/+40
Remove MEM_TYPE_ZERO_COPY in favor of the new MEM_TYPE_XSK_BUFF_POOL APIs. v4->v5: Fixed "warning: Excess function parameter 'alloc' description in 'ice_alloc_rx_bufs_zc'" and "warning: Excess function parameter 'xdp' description in 'ice_construct_skb_zc'". (Jakub) Signed-off-by: Maciej Fijalkowski <[email protected]> Signed-off-by: Björn Töpel <[email protected]> Signed-off-by: Alexei Starovoitov <[email protected]> Cc: [email protected] Link: https://lore.kernel.org/bpf/[email protected]
2020-05-21xsk: Move driver interface to xdp_sock_drv.hMagnus Karlsson1-1/+1
Move the AF_XDP zero-copy driver interface to its own include file called xdp_sock_drv.h. This, hopefully, will make it more clear for NIC driver implementors to know what functions to use for zero-copy support. v4->v5: Fix -Wmissing-prototypes by include header file. (Jakub) Signed-off-by: Magnus Karlsson <[email protected]> Signed-off-by: Alexei Starovoitov <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2020-05-14xdp: For Intel AF_XDP drivers add XDP frame_szJesper Dangaard Brouer1-0/+2
Intel drivers implement native AF_XDP zerocopy in separate C-files, that have its own invocation of bpf_prog_run_xdp(). The setup of xdp_buff is also handled in separately from normal code path. This patch update XDP frame_sz for AF_XDP zerocopy drivers i40e, ice and ixgbe, as the code changes needed are very similar. Introduce a helper function xsk_umem_xdp_frame_sz() for calculating frame size. Signed-off-by: Jesper Dangaard Brouer <[email protected]> Signed-off-by: Alexei Starovoitov <[email protected]> Acked-by: Björn Töpel <[email protected]> Cc: [email protected] Cc: Magnus Karlsson <[email protected]> Link: https://lore.kernel.org/bpf/158945347511.97035.8536753731329475655.stgit@firesoul
2020-02-19ice: add backslash-n to stringsJesse Brandeburg1-3/+3
There were several strings found without line feeds, fix them by adding a line feed, as is typical. Without this lotsofmessagescanbejumbledtogether. This patch has known checkpatch warnings from long lines for the NL_* messages, because checkpatch doesn't know how to ignore them. Signed-off-by: Jesse Brandeburg <[email protected]> Tested-by: Andrew Bowers <[email protected]> Signed-off-by: Jeff Kirsher <[email protected]>
2020-02-19ice: Support XDP UMEM wake up mechanismKrzysztof Kazimierczak1-0/+18
Add support for a new AF_XDP feature that has already been introduced in upstreamed Intel NIC drivers. If a user space application signals that it might sleep using the new bind flag XDP_USE_NEED_WAKEUP, the driver will then set this flag if it has no more buffers on the NIC Rx ring and yield to the application. For Tx, it will set the flag if it has no outstanding Tx completion interrupts and return to the application. Signed-off-by: Krzysztof Kazimierczak <[email protected]> Tested-by: Andrew Bowers <[email protected]> Signed-off-by: Jeff Kirsher <[email protected]>
2020-02-15ice: use true/false for bool typesBruce Allan1-1/+1
Subject says it all. Signed-off-by: Bruce Allan <[email protected]> Tested-by: Andrew Bowers <[email protected]> Signed-off-by: Jeff Kirsher <[email protected]>
2020-02-15ice: use proper format for function pointer as a function parameterBruce Allan1-1/+1
Compiling with gcc-9.2.1 with W=1 points out warnings about the improper function parameter list. Fix it. Signed-off-by: Bruce Allan <[email protected]> Tested-by: Andrew Bowers <[email protected]> Signed-off-by: Jeff Kirsher <[email protected]>
2020-02-15ice: replace "fallthrough" comments with fallthrough reserved wordBruce Allan1-2/+2
"fallthrough" comments are used in switch case statements to explicitly indicate the code is intended to fall through to the following statement. Different variants of "fallthough" are acceptable, e.g. "fall through", "fallthrough", "Fall-through". The GCC compiler has an optional warning (-Wimplicit-fallthrough[=n]) to warn when such a comment is not present; the default version of which is enabled when compiling the Linux kernel. There have been recent discussions in kernel mailing lists regarding replacing non-standardized "fallthrough" comments with the pseudo-reserved word 'fallthrough' which will be defined as __attribute__ ((fallthrough)) for versions of gcc that support it (i.e. gcc 7 and newer) or as a nop for versions that do not. Replace "fallthrough" comments with fallthrough reserved word. Signed-off-by: Bruce Allan <[email protected]> Signed-off-by: Tony Nguyen <[email protected]> Tested-by: Andrew Bowers <[email protected]> Signed-off-by: Jeff Kirsher <[email protected]>
2020-02-15ice: Add support to enable/disable all Rx queues before waitingBrett Creeley1-2/+2
Currently when we enable/disable all Rx queues we do the following sequence for each Rx queue and then move to the next queue. 1. Enable/Disable the Rx queue via register write. 2. Read the configuration register to determine if the Rx queue was enabled/disabled successfully. In some cases enabling/disabling queue 0 fails because of step 2 above. Fix this by doing step 1 for all of the Rx queues and then step 2 for all of the Rx queues. Also, there are cases where we enable/disable a single queue (i.e. SR-IOV and XDP) so add a new function that does step 1 and 2 above with a read flush in between. This change also required a single Rx queue to be enabled/disabled with and without waiting for the change to propagate through hardware. Fix this by adding a boolean wait flag to the necessary functions. Also, add the keywords "one" and "all" to distinguish between enabling/disabling a single Rx queue and all Rx queues respectively. Signed-off-by: Brett Creeley <[email protected]> Tested-by: Andrew Bowers <[email protected]> Signed-off-by: Jeff Kirsher <[email protected]>
2020-02-12ice: Make print statements more compactAnirudh Venkataramanan1-2/+2
Formatting strings in print function calls (like dev_info, dev_err, etc.) can exceed 80 columns without making checkpatch unhappy. So remove newlines where applicable and make print statements more compact. Signed-off-by: Anirudh Venkataramanan <[email protected]> Signed-off-by: Tony Nguyen <[email protected]> Tested-by: Andrew Bowers <[email protected]> Signed-off-by: Jeff Kirsher <[email protected]>
2020-01-17ice: remove redundant assignment to variable xmit_doneColin Ian King1-1/+1
The variable xmit_done is being initialized with a value that is never read and it is being updated later with a new value. The initialization is redundant and can be removed. Addresses-Coverity: ("Unused value") Signed-off-by: Colin Ian King <[email protected]> Tested-by: Andrew Bowers <[email protected]> Signed-off-by: Jeff Kirsher <[email protected]>
2020-01-03ice: Add a boundary check in ice_xsk_umem()Krzysztof Kazimierczak1-1/+2
In ice_xsk_umem(), variable qid which is later used as an array index, is not validated for a possible boundary exceedance. Because of that, a calling function might receive an invalid address, which causes general protection fault when dereferenced. To address this, add a boundary check to see if qid is greater than the size of a UMEM array. Also, don't let user change vsi->num_xsk_umems just by trying to setup a second UMEM if its value is already set up (i.e. UMEM region has already been allocated for this VSI). While at it, make sure that ring->zca.free pointer is always zeroed out if there is no UMEM on a specified ring. Signed-off-by: Krzysztof Kazimierczak <[email protected]> Tested-by: Andrew Bowers <[email protected]> Signed-off-by: Jeff Kirsher <[email protected]>
2019-12-20xsk: ixgbe: i40e: ice: mlx5: Xsk_umem_discard_addr to xsk_umem_release_addrMagnus Karlsson1-2/+2
Change the name of xsk_umem_discard_addr to xsk_umem_release_addr to better reflect the new naming of the AF_XDP queue manipulation functions. As this functions is used by drivers implementing support for AF_XDP zero-copy, it requires a name change to these drivers. The function xsk_umem_release_addr_rq has also changed name in the same fashion. Signed-off-by: Magnus Karlsson <[email protected]> Signed-off-by: Alexei Starovoitov <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2019-11-22ice: Add ice_pf_to_dev(pf) macroBrett Creeley1-2/+2
We use &pf->dev->pdev all over the code. Add a simple macro to do this for us. When multiple de-references like this are being done add a local struct device variable. Signed-off-by: Brett Creeley <[email protected]> Signed-off-by: Tony Nguyen <[email protected]> Tested-by: Andrew Bowers <[email protected]> Signed-off-by: Jeff Kirsher <[email protected]>
2019-11-04ice: Add support for AF_XDPKrzysztof Kazimierczak1-0/+1181
Add zero copy AF_XDP support. This patch adds zero copy support for Tx and Rx; code for zero copy is added to ice_xsk.h and ice_xsk.c. For Tx, implement ndo_xsk_wakeup. As with other drivers, reuse existing XDP Tx queues for this task, since XDP_REDIRECT guarantees mutual exclusion between different NAPI contexts based on CPU ID. In turn, a netdev can XDP_REDIRECT to another netdev with a different NAPI context, since the operation is bound to a specific core and each core has its own hardware ring. For Rx, allocate frames as MEM_TYPE_ZERO_COPY on queues that AF_XDP is enabled. Signed-off-by: Krzysztof Kazimierczak <[email protected]> Co-developed-by: Maciej Fijalkowski <[email protected]> Signed-off-by: Maciej Fijalkowski <[email protected]> Signed-off-by: Tony Nguyen <[email protected]> Signed-off-by: Jeff Kirsher <[email protected]>