aboutsummaryrefslogtreecommitdiff
path: root/tools/testing/selftests/net/lib/py
AgeCommit message (Collapse)AuthorFilesLines
2024-08-12selftests: drv-net: rss_ctx: test dumping RSS contextsJakub Kicinski1-0/+6
Add a test for dumping RSS contexts. Make sure indir table and key are sane when contexts are created with various combination of inputs. Test the dump filtering by ifname and start-context. Reviewed-by: Petr Machata <petrm@nvidia.com> Reviewed-by: Edward Cree <ecree.xilinx@gmail.com> Signed-off-by: Jakub Kicinski <kuba@kernel.org> Signed-off-by: David S. Miller <davem@davemloft.net>
2024-08-02selftests: net: ksft: print more of the stack for checksJakub Kicinski1-2/+12
Print more stack frames and the failing line when check fails. This helps when tests use helpers to do the checks. Before: # At ./ksft/drivers/net/hw/rss_ctx.py line 92: # Check failed 1037698 >= 396893.0 traffic on other queues:[344612, 462380, 233020, 449174, 342298] not ok 8 rss_ctx.test_rss_context_queue_reconfigure After: # Check| At ./ksft/drivers/net/hw/rss_ctx.py, line 387, in test_rss_context_queue_reconfigure: # Check| test_rss_queue_reconfigure(cfg, main_ctx=False) # Check| At ./ksft/drivers/net/hw/rss_ctx.py, line 230, in test_rss_queue_reconfigure: # Check| _send_traffic_check(cfg, port, ctx_ref, { 'target': (0, 3), # Check| At ./ksft/drivers/net/hw/rss_ctx.py, line 92, in _send_traffic_check: # Check| ksft_lt(sum(cnts[i] for i in params['noise']), directed / 2, # Check failed 1045235 >= 405823.5 traffic on other queues (context 1)':[460068, 351995, 565970, 351579, 127270] not ok 8 rss_ctx.test_rss_context_queue_reconfigure Link: https://patch.msgid.link/20240801232317.545577-1-kuba@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2024-08-02selftests: net: ksft: support marking tests as disruptiveStanislav Fomichev1-0/+40
Add new @ksft_disruptive decorator to mark the tests that might be disruptive to the system. Depending on how well the previous test works in the CI we might want to disable disruptive tests by default and only let the developers run them manually. KSFT framework runs disruptive tests by default. DISRUPTIVE=False environment (or config file) can be used to disable these tests. ksft_setup should be called by the test cases that want to use new decorator (ksft_setup is only called via NetDrvEnv/NetDrvEpEnv for now). In the future we can add similar decorators to, for example, avoid running slow tests all the time. And/or have some option to run only 'fast' tests for some sort of smoke test scenario. $ DISRUPTIVE=False ./stats.py KTAP version 1 1..5 ok 1 stats.check_pause ok 2 stats.check_fec ok 3 stats.pkt_byte_sum ok 4 stats.qstat_by_ifindex ok 5 stats.check_down # SKIP marked as disruptive # Totals: pass:4 fail:0 xfail:0 xpass:0 skip:1 error:0 v3: - parse yes and properly treat non-zero nums as true (Petr) v2: - convert from cli argument to env variable (Jakub) Signed-off-by: Stanislav Fomichev <sdf@fomichev.me> Link: https://patch.msgid.link/20240802000309.2368-2-sdf@fomichev.me Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2024-07-08selftests: net: ksft: interrupt cleanly on KeyboardInterruptJakub Kicinski1-1/+8
It's very useful to be able to interrupt the tests during development. Detect KeyboardInterrupt, run the cleanups and exit. Link: https://patch.msgid.link/20240705015222.675840-1-kuba@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2024-06-28selftests: drv-net: add ability to schedule cleanup with defer()Jakub Kicinski2-0/+55
This implements what I was describing in [1]. When writing a test author can schedule cleanup / undo actions right after the creation completes, eg: cmd("touch /tmp/file") defer(cmd, "rm /tmp/file") defer() takes the function name as first argument, and the rest are arguments for that function. defer()red functions are called in inverse order after test exits. It's also possible to capture them and execute earlier (in which case they get automatically de-queued). undo = defer(cmd, "rm /tmp/file") # ... some unsafe code ... undo.exec() As a nice safety all exceptions from defer()ed calls are captured, printed, and ignored (they do make the test fail, however). This addresses the common problem of exceptions in cleanup paths often being unhandled, leading to potential leaks. There is a global action queue, flushed by ksft_run(). We could support function level defers too, I guess, but there's no immediate need.. Link: https://lore.kernel.org/all/877cedb2ki.fsf@nvidia.com/ # [1] Reviewed-by: Przemek Kitszel <przemyslaw.kitszel@intel.com> Reviewed-by: Petr Machata <petrm@nvidia.com> Link: https://patch.msgid.link/20240627185502.3069139-3-kuba@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2024-06-28selftests: net: ksft: avoid continue when handling resultsJakub Kicinski1-15/+15
Exception handlers print the result and use continue to skip the non-exception result printing. This makes inserting common post-test code hard. Refactor to avoid the continues and have only one ktap_result() call. Reviewed-by: Petr Machata <petrm@nvidia.com> Link: https://patch.msgid.link/20240627185502.3069139-2-kuba@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2024-06-26selftests: drv-net: rss_ctx: add tests for RSS configuration and contextsJakub Kicinski2-2/+11
Add tests focusing on indirection table configuration and creating extra RSS contexts in drivers which support it. $ export NETIF=eth0 REMOTE_... $ ./drivers/net/hw/rss_ctx.py KTAP version 1 1..8 ok 1 rss_ctx.test_rss_key_indir ok 2 rss_ctx.test_rss_context ok 3 rss_ctx.test_rss_context4 # Increasing queue count 44 -> 66 # Failed to create context 32, trying to test what we got ok 4 rss_ctx.test_rss_context32 # SKIP Tested only 31 contexts, wanted 32 ok 5 rss_ctx.test_rss_context_overlap ok 6 rss_ctx.test_rss_context_overlap2 # .. sprays traffic like a headless chicken .. not ok 7 rss_ctx.test_rss_context_out_of_order ok 8 rss_ctx.test_rss_context4_create_with_cfg # Totals: pass:6 fail:1 xfail:0 xpass:0 skip:1 error:0 Note that rss_ctx.test_rss_context_out_of_order fails with the device I tested with, but it seems to be a device / driver bug. Reviewed-by: Petr Machata <petrm@nvidia.com> Reviewed-by: Willem de Bruijn <willemb@google.com> Link: https://patch.msgid.link/20240626012456.2326192-5-kuba@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2024-06-26selftests: drv-net: add helper to wait for HW stats to syncJakub Kicinski1-0/+4
Some devices DMA stats to the host periodically. Add a helper which can wait for that to happen, based on frequency reported by the driver in ethtool. Reviewed-by: Petr Machata <petrm@nvidia.com> Reviewed-by: Willem de Bruijn <willemb@google.com> Link: https://patch.msgid.link/20240626012456.2326192-3-kuba@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2024-06-26selftests: drv-net: try to check if port is in useJakub Kicinski1-2/+13
We use random ports for communication. As Willem predicted this leads to occasional failures. Try to check if port is already in use by opening a socket and binding to that port. Reviewed-by: Przemek Kitszel <przemyslaw.kitszel@intel.com> Reviewed-by: Petr Machata <petrm@nvidia.com> Reviewed-by: Willem de Bruijn <willemb@google.com> Link: https://patch.msgid.link/20240626012456.2326192-2-kuba@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2024-05-08net: selftest: add test for netdev netlink queue-get APIDavid Wei2-6/+6
Add a selftest for netdev generic netlink. For now there is only a single test that exercises the `queue-get` API. The test works with netdevsim by default or with a real device by setting NETIF. Add a timeout param to cmd() since ethtool -L can take a long time on real devices. Signed-off-by: David Wei <dw@davidwei.uk> Link: https://lore.kernel.org/r/20240507163228.2066817-3-dw@davidwei.uk Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2024-05-02selftests: net: py: check process exit code in bkg() and background cmd()Jakub Kicinski1-2/+6
We're a bit too loose with error checking for background processes. cmd() completely ignores the fail argument passed to the constructor if background is True. Default to checking for errors if process is not terminated explicitly. Caller can override with True / False. For bkg() the processing step is called magically by __exit__ so record the value passed in the constructor. Reported-by: Willem de Bruijn <willemb@google.com> Tested-by: Willem de Bruijn <willemb@google.com> Link: https://lore.kernel.org/r/20240502025325.1924923-1-kuba@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2024-04-30selftests: drv-net-hw: add test for memory allocation failures with page poolJakub Kicinski1-0/+4
Bugs in memory allocation failure paths are quite common. Add a test exercising those paths based on qstat and page pool failure hook. Running on bnxt: # ./drivers/net/hw/pp_alloc_fail.py KTAP version 1 1..1 # ethtool -G change retval: success ok 1 pp_alloc_fail.test_pp_alloc # Totals: pass:1 fail:0 xfail:0 xpass:0 skip:0 error:0 I initially wrote this test to validate commit be43b7489a3c ("net/mlx5e: RX, Fix page_pool allocation failure recovery for striding rq") but mlx5 still doesn't have qstat. So I run it on bnxt, and while bnxt survives I found the problem fixed in commit 730117730709 ("eth: bnxt: fix counting packets discarded due to OOM and netpoll"). Reviewed-by: Willem de Bruijn <willemb@google.com> Link: https://lore.kernel.org/r/20240429144426.743476-7-kuba@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2024-04-30selftests: net: py: avoid all ports < 10kJakub Kicinski1-1/+1
When picking TCP ports to use, avoid all below 10k. This should lower the chance of collision or running afoul whatever random policies may be on the host. Reviewed-by: Willem de Bruijn <willemb@google.com> Link: https://lore.kernel.org/r/20240429144426.743476-5-kuba@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2024-04-30selftests: net: py: extract tool logicJakub Kicinski1-3/+9
The main use of the ip() wrapper over cmd() is that it can parse JSON. cmd("ip -j link show") will return stdout as a string, and test has to call json.loads(). With ip("link show", json=True) the return value will be already parsed. More tools (ethtool, bpftool etc.) support the --json switch. To avoid having to wrap all of them individually create a tool() helper. Switch from -j to --json (for ethtool). While at it consume the netns attribute at the ip() level. Reviewed-by: Willem de Bruijn <willemb@google.com> Link: https://lore.kernel.org/r/20240429144426.743476-4-kuba@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2024-04-23selftests: drv-net: add a TCP ping test case (and useful helpers)Jakub Kicinski1-0/+43
More complex tests often have to spawn a background process, like a server which will respond to requests or tcpdump. Add support for creating such processes using the with keyword: with bkg("my-daemon", ..): # my-daemon is alive in this block My initial thought was to add this support to cmd() directly but it runs the command in the constructor, so by the time we __enter__ it's too late to make sure we used "background=True". Second useful helper transplanted from net_helper.sh is wait_port_listen(). The test itself uses socat, which insists on v6 addresses being wrapped in [], it's not the only command which requires this format, so add the wrapped address to env. The hope is to save test code from checking if address is v6. Reviewed-by: Willem de Bruijn <willemb@google.com> Link: https://lore.kernel.org/r/20240420025237.3309296-7-kuba@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2024-04-23selftests: net: support matching cases by name prefixJakub Kicinski1-1/+12
While writing tests with a lot more cases I got tired of having to jump back and forth to add the name of the test to the ksft_run() list. Most unittest frameworks do some name matching, e.g. assume that functions with names starting with test_ are test cases. Support similar flow in ksft_run(). Let the author list the desired prefixes. globals() need to be passed explicitly, IDK how to work around that. Reviewed-by: Willem de Bruijn <willemb@google.com> Link: https://lore.kernel.org/r/20240420025237.3309296-6-kuba@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2024-04-23selftests: drv-net: construct environment for running tests which require an ↵Jakub Kicinski2-0/+32
endpoint Nothing surprising here, hopefully. Wrap the variables from the environment into a class or spawn a netdevsim based env and pass it to the tests. Reviewed-by: Willem de Bruijn <willemb@google.com> Link: https://lore.kernel.org/r/20240420025237.3309296-4-kuba@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2024-04-23selftests: drv-net: define endpoint structuresJakub Kicinski1-8/+9
Define the remote endpoint "model". To execute most meaningful device driver tests we need to be able to communicate with a remote system, and have it send traffic to the device under test. Various test environments will have different requirements. 0) "Local" netdevsim-based testing can simply use net namespaces. netdevsim supports connecting two devices now, to form a veth-like construct. 1) Similarly on hosts with multiple NICs, the NICs may be connected together with a loopback cable or internal device loopback. One interface may be placed into separate netns, and tests would proceed much like in the netdevsim case. Note that the loopback config or the moving of one interface into a netns is not expected to be part of selftest code. 2) Some systems may need to communicate with the remote endpoint via SSH. 3) Last but not least environment may have its own custom communication method. Fundamentally we only need two operations: - run a command remotely - deploy a binary (if some tool we need is built as part of kselftests) Wrap these two in a class. Use dynamic loading to load the Remote class. This will allow very easy definition of other communication methods without bothering upstream code base. Stick to the "simple" / "no unnecessary abstractions" model for referring to the remote endpoints. The host / remote object are passed as an argument to the usual cmd() or ip() invocation. For example: ip("link show", json=True, host=remote) Reviewed-by: Willem de Bruijn <willemb@google.com> Link: https://lore.kernel.org/r/20240420025237.3309296-2-kuba@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2024-04-23selftests: drv-net: test dumping qstats per deviceJakub Kicinski1-0/+18
Add a test for dumping qstats device by device. ksft framework grows a ksft_raises() helper, to be used under with, which should be familiar to unittest users. Link: https://lore.kernel.org/r/20240420023543.3300306-5-kuba@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2024-04-18selftests: net: set the exit code correctly in Python testsJakub Kicinski1-0/+10
Test cases need to exit with non-zero status if they failed, we currently don't do that: # KTAP version 1 # 1..3 # # At /root/ksft-net-drv/drivers/net/./ping.py line 18: # # Check failed 1 != 2 # not ok 1 ping.test_v4 # ok 2 ping.test_v6 # ok 3 ping.test_tcp # # Totals: pass:2 fail:1 xfail:0 xpass:0 skip:0 error:0 ok 1 selftests: drivers/net: ping.py ^^^^ It's a bit tempting to make the exit part of ksft_run(), but that only works well for very trivial setups. We can revisit this later, if people forget to call ksft_exit(). Link: https://lore.kernel.org/r/20240417231146.2435572-3-kuba@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2024-04-18selftests: net: fix counting totals when some checks failJakub Kicinski1-1/+4
Totals currently only pay attention to exceptions, if check fails (say ksft_eq()) the test case will be counted as pass: # At /ksft/drivers/net/./ping.py line 18: # Check failed 1 != 2 not ok 1 ping.test_v4 ok 2 ping.test_v6 ok 3 ping.test_tcp # Totals: pass:3 fail:0 xfail:0 xpass:0 skip:0 error:0 ^^^^^^^^^^^^^ Pay attention to the result. Fixes: b86761ff6374 ("selftests: net: add scaffolding for Netlink tests in Python") Link: https://lore.kernel.org/r/20240417231146.2435572-2-kuba@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2024-04-16selftests: drv-net: add stdout to the command failed exceptionJakub Kicinski1-1/+2
ping prints all the info to stdout. To make debug easier capture stdout in the Exception raised when command unexpectedly fails. Reviewed-by: Willem de Bruijn <willemb@google.com> Link: https://lore.kernel.org/r/20240416004556.1618804-2-kuba@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2024-04-15selftests: net: exercise page pool reporting via netlinkJakub Kicinski2-0/+13
Add a Python test for the basic ops. # ./net/nl_netdev.py KTAP version 1 1..3 ok 1 nl_netdev.empty_check ok 2 nl_netdev.lo_check ok 3 nl_netdev.page_pool_check # Totals: pass:3 fail:0 xfail:0 xpass:0 skip:0 error:0 Reviewed-by: Petr Machata <petrm@nvidia.com> Link: https://lore.kernel.org/r/20240412141436.828666-7-kuba@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2024-04-15selftests: net: support use of NetdevSimDev under "with" in pythonJakub Kicinski1-1/+14
Using "with" on an entire driver test env is supported already, but it's also useful to use "with" on an individual nsim. Reviewed-by: Petr Machata <petrm@nvidia.com> Link: https://lore.kernel.org/r/20240412141436.828666-6-kuba@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2024-04-15selftests: net: print full exception on failureJakub Kicinski1-1/+3
Instead of a summary line print the full exception. This makes debugging Python tests much easier. Reviewed-by: Petr Machata <petrm@nvidia.com> Link: https://lore.kernel.org/r/20240412141436.828666-5-kuba@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2024-04-15selftests: net: print report check location in python testsJakub Kicinski1-11/+14
Developing Python tests is a bit annoying because when test fails we only print the fail message and no info about which exact check led to it. Print the location (the first line of this example is new): # At /root/ksft-net-drv/./net/nl_netdev.py line 38: # Check failed 0 != 10 not ok 3 nl_netdev.page_pool_check Reviewed-by: Petr Machata <petrm@nvidia.com> Link: https://lore.kernel.org/r/20240412141436.828666-4-kuba@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2024-04-10selftests: net: reuse common code in bpf_offloadJakub Kicinski1-2/+7
net/lib/py/nsim.py already contains the most useful parts of the netdevsim wrapper classes. Reuse them. Acked-by: Stanislav Fomichev <sdf@google.com> Acked-by: Martin KaFai Lau <martin.lau@kernel.org> Link: https://lore.kernel.org/r/20240409031549.3531084-5-kuba@kernel.org Signed-off-by: Jakub Kicinski <kuba@kernel.org>
2024-04-08selftests: drivers: add scaffolding for Netlink tests in PythonJakub Kicinski2-0/+116
Add drivers/net as a target for mixed-use tests. The setup is expected to work similarly to the forwarding tests. Since we only need one interface (unlike forwarding tests) read the target device name from NETIF. If not present we'll try to run the test against netdevsim. Reviewed-by: Petr Machata <petrm@nvidia.com> Signed-off-by: Jakub Kicinski <kuba@kernel.org> Signed-off-by: David S. Miller <davem@davemloft.net>
2024-04-08selftests: net: add scaffolding for Netlink tests in PythonJakub Kicinski5-0/+207
Add glue code for accessing the YNL library which lives under tools/net and YAML spec files from under Documentation/. Automatically figure out if tests are run in tree or not. Since we'll want to use this library both from net and drivers/net test targets make the library a target as well, and automatically include it when net or drivers/net are included. Making net/lib a target ensures that we end up with only one copy of it, and saves us some path guessing. Add a tiny bit of formatting support to be able to output KTAP from the start. Reviewed-by: Petr Machata <petrm@nvidia.com> Signed-off-by: Jakub Kicinski <kuba@kernel.org> Signed-off-by: David S. Miller <davem@davemloft.net>
Powered by cgit, Gruvboxed by Blaster4385.