aboutsummaryrefslogtreecommitdiff
path: root/tools
AgeCommit message (Collapse)AuthorFilesLines
2021-07-30tools: bpftool: Complete metrics list in "bpftool prog profile" docQuentin Monnet2-2/+4
Profiling programs with bpftool was extended some time ago to support two new metrics, namely itlb_misses and dtlb_misses (misses for the instruction/data translation lookaside buffer). Update the manual page and bash completion accordingly. Fixes: 450d060e8f75 ("bpftool: Add {i,d}tlb_misses support for bpftool profile") Signed-off-by: Quentin Monnet <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2021-07-30tools: bpftool: Document and add bash completion for -L, -B optionsQuentin Monnet8-10/+96
The -L|--use-loader option for using loader programs when loading, or when generating a skeleton, did not have any documentation or bash completion. Same thing goes for -B|--base-btf, used to pass a path to a base BTF object for split BTF such as BTF for kernel modules. This patch documents and adds bash completion for those options. Fixes: 75fa1777694c ("tools/bpftool: Add bpftool support for split BTF") Fixes: d510296d331a ("bpftool: Use syscall/loader program in "prog load" and "gen skeleton" command.") Signed-off-by: Quentin Monnet <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2021-07-30selftests/bpf: Update bpftool's consistency script for checking optionsQuentin Monnet1-11/+111
Update the script responsible for checking that the different types used at various places in bpftool are synchronised, and extend it to check the consistency of options between the help messages in the source code and the manual pages. Signed-off-by: Quentin Monnet <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2021-07-30tools: bpftool: Update and synchronise option list in doc and help msgQuentin Monnet25-28/+45
All bpftool commands support the options for JSON output and debug from libbpf. In addition, some commands support additional options corresponding to specific use cases. The list of options described in the man pages for the different commands are not always accurate. The messages for interactive help are mostly limited to HELP_SPEC_OPTIONS, and are even less representative of the actual set of options supported for the commands. Let's update the lists: - HELP_SPEC_OPTIONS is modified to contain the "default" options (JSON and debug), and to be extensible (no ending curly bracket). - All commands use HELP_SPEC_OPTIONS in their help message, and then complete the list with their specific options. - The lists of options in the man pages are updated. - The formatting of the list for bpftool.rst is adjusted to match formatting for the other man pages. This is for consistency, and also because it will be helpful in a future patch to automatically check that the files are synchronised. Signed-off-by: Quentin Monnet <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2021-07-30tools: bpftool: Complete and synchronise attach or map typesQuentin Monnet4-5/+12
Update bpftool's list of attach type names to tell it about the latest attach types, or the "ringbuf" map. Also update the documentation, help messages, and bash completion when relevant. These missing items were reported by the newly added Python script used to help maintain consistency in bpftool. Signed-off-by: Quentin Monnet <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2021-07-30selftests/bpf: Check consistency between bpftool source, doc, completionQuentin Monnet1-0/+486
Whenever the eBPF subsystem gains new elements, such as new program or map types, it is necessary to update bpftool if we want it able to handle the new items. In addition to the main arrays containing the names of these elements in the source code, there are also multiple locations to update: - The help message in the do_help() functions in bpftool's source code. - The RST documentation files. - The bash completion file. This has led to omissions multiple times in the past. This patch attempts to address this issue by adding consistency checks for all these different locations. It also verifies that the bpf_prog_type, bpf_map_type and bpf_attach_type enums from the UAPI BPF header have all their members present in bpftool. The script requires no argument to run, it reads and parses the different files to check, and prints the mismatches, if any. It currently reports a number of missing elements, which will be fixed in a later patch: $ ./test_bpftool_synctypes.py Comparing [...]/linux/tools/bpf/bpftool/map.c (map_type_name) and [...]/linux/tools/bpf/bpftool/bash-completion/bpftool (BPFTOOL_MAP_CREATE_TYPES): {'ringbuf'} Comparing BPF header (enum bpf_attach_type) and [...]/linux/tools/bpf/bpftool/common.c (attach_type_name): {'BPF_TRACE_ITER', 'BPF_XDP_DEVMAP', 'BPF_XDP', 'BPF_SK_REUSEPORT_SELECT', 'BPF_XDP_CPUMAP', 'BPF_SK_REUSEPORT_SELECT_OR_MIGRATE'} Comparing [...]/linux/tools/bpf/bpftool/prog.c (attach_type_strings) and [...]/linux/tools/bpf/bpftool/prog.c (do_help() ATTACH_TYPE): {'skb_verdict'} Comparing [...]/linux/tools/bpf/bpftool/prog.c (attach_type_strings) and [...]/linux/tools/bpf/bpftool/Documentation/bpftool-prog.rst (ATTACH_TYPE): {'skb_verdict'} Comparing [...]/linux/tools/bpf/bpftool/prog.c (attach_type_strings) and [...]/linux/tools/bpf/bpftool/bash-completion/bpftool (BPFTOOL_PROG_ATTACH_TYPES): {'skb_verdict'} Note that the script does NOT check for consistency between the list of program types that bpftool claims it accepts and the actual list of keywords that can be used. This is because bpftool does not "see" them, they are ELF section names parsed by libbpf. It is not hard to parse the section_defs[] array in libbpf, but some section names are associated with program types that bpftool cannot load at the moment. For example, some programs require a BTF target and an attach target that bpftool cannot handle. The script may be extended to parse the array and check only relevant values in the future. The script is not added to the selftests' Makefile, because doing so would require all patches with BPF UAPI change to also update bpftool. Instead it is to be added to the CI. Signed-off-by: Quentin Monnet <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2021-07-30tools: bpftool: Slightly ease bash completion updatesQuentin Monnet1-25/+29
Bash completion for bpftool gets two minor improvements in this patch. Move the detection of attach types for "bpftool cgroup attach" outside of the "case/esac" bloc, where we cannot reuse our variable holding the list of supported attach types as a pattern list. After the change, we have only one list of cgroup attach types to update when new types are added, instead of the former two lists. Also rename the variables holding lists of names for program types, map types, and attach types, to make them more unique. This can make it slightly easier to point people to the relevant variables to update, but the main objective here is to help run a script to check that bash completion is up-to-date with bpftool's source code. Signed-off-by: Quentin Monnet <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2021-07-30Revert "perf map: Fix dso->nsinfo refcounting"Arnaldo Carvalho de Melo1-2/+0
This makes 'perf top' abort in some cases, and the right fix will involve surgery that is too much to do at this stage, so revert for now and fix it in the next merge window. This reverts commit 2d6b74baa7147251c30a46c4996e8cc224aa2dc5. Cc: Riccardo Mancini <[email protected]> Cc: Ian Rogers <[email protected]> Cc: Jiri Olsa <[email protected]> Cc: Krister Johansen <[email protected]> Cc: Mark Rutland <[email protected]> Cc: Namhyung Kim <[email protected]> Cc: Peter Zijlstra <[email protected]> Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>
2021-07-30libbpf: Add btf__load_vmlinux_btf/btf__load_module_btfHengqi Chen4-6/+20
Add two new APIs: btf__load_vmlinux_btf and btf__load_module_btf. btf__load_vmlinux_btf is just an alias to the existing API named libbpf_find_kernel_btf, rename to be more precisely and consistent with existing BTF APIs. btf__load_module_btf can be used to load module BTF, add it for completeness. These two APIs are useful for implementing tracing tools and introspection tools. This is part of the effort towards libbpf 1.0 ([0]). [0] Closes: https://github.com/libbpf/libbpf/issues/280 Signed-off-by: Hengqi Chen <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2021-07-29tools: bpftool: Support dumping split BTF by idQuentin Monnet1-1/+1
Split BTF objects are typically BTF objects for kernel modules, which are incrementally built on top of kernel BTF instead of redefining all kernel symbols they need. We can use bpftool with its -B command-line option to dump split BTF objects. It works well when the handle provided for the BTF object to dump is a "path" to the BTF object, typically under /sys/kernel/btf, because bpftool internally calls btf__parse_split() which can take a "base_btf" pointer and resolve the BTF reconstruction (although in that case, the "-B" option is unnecessary because bpftool performs autodetection). However, it did not work so far when passing the BTF object through its id, because bpftool would call btf__get_from_id() which did not provide a way to pass a "base_btf" pointer. In other words, the following works: # bpftool btf dump file /sys/kernel/btf/i2c_smbus -B /sys/kernel/btf/vmlinux But this was not possible: # bpftool btf dump id 6 -B /sys/kernel/btf/vmlinux The libbpf API has recently changed, and btf__get_from_id() has been deprecated in favour of btf__load_from_kernel_by_id() and its version with support for split BTF, btf__load_from_kernel_by_id_split(). Let's update bpftool to make it able to dump the BTF object in the second case as well. Signed-off-by: Quentin Monnet <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Acked-by: John Fastabend <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2021-07-29libbpf: Add split BTF support for btf__load_from_kernel_by_id()Quentin Monnet3-2/+9
Add a new API function btf__load_from_kernel_by_id_split(), which takes a pointer to a base BTF object in order to support split BTF objects when retrieving BTF information from the kernel. Reference: https://github.com/libbpf/libbpf/issues/314 Signed-off-by: Quentin Monnet <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Acked-by: John Fastabend <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2021-07-29tools: Replace btf__get_from_id() with btf__load_from_kernel_by_id()Quentin Monnet7-29/+42
Replace the calls to function btf__get_from_id(), which we plan to deprecate before the library reaches v1.0, with calls to btf__load_from_kernel_by_id() in tools/ (bpftool, perf, selftests). Update the surrounding code accordingly (instead of passing a pointer to the btf struct, get it as a return value from the function). Signed-off-by: Quentin Monnet <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Acked-by: John Fastabend <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2021-07-29tools: Free BTF objects at various locationsQuentin Monnet4-4/+9
Make sure to call btf__free() (and not simply free(), which does not free all pointers stored in the struct) on pointers to struct btf objects retrieved at various locations. These were found while updating the calls to btf__get_from_id(). Fixes: 999d82cbc044 ("tools/bpf: enhance test_btf file testing to test func info") Fixes: 254471e57a86 ("tools/bpf: bpftool: add support for func types") Fixes: 7b612e291a5a ("perf tools: Synthesize PERF_RECORD_* for loaded BPF programs") Fixes: d56354dc4909 ("perf tools: Save bpf_prog_info and BTF of new BPF programs") Fixes: 47c09d6a9f67 ("bpftool: Introduce "prog profile" command") Fixes: fa853c4b839e ("perf stat: Enable counting events for BPF programs") Signed-off-by: Quentin Monnet <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2021-07-29libbpf: Rename btf__get_from_id() as btf__load_from_kernel_by_id()Quentin Monnet4-11/+24
Rename function btf__get_from_id() as btf__load_from_kernel_by_id() to better indicate what the function does. Change the new function so that, instead of requiring a pointer to the pointer to update and returning with an error code, it takes a single argument (the id of the BTF object) and returns the corresponding pointer. This is more in line with the existing constructors. The other tools calling the (soon-to-be) deprecated btf__get_from_id() function will be updated in a future commit. References: - https://github.com/libbpf/libbpf/issues/278 - https://github.com/libbpf/libbpf/wiki/Libbpf:-the-road-to-v1.0#btfh-apis Signed-off-by: Quentin Monnet <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Acked-by: John Fastabend <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2021-07-29libbpf: Rename btf__load() as btf__load_into_kernel()Quentin Monnet4-2/+5
As part of the effort to move towards a v1.0 for libbpf, rename btf__load() function, used to "upload" BTF information into the kernel, as btf__load_into_kernel(). This new name better reflects what the function does. References: - https://github.com/libbpf/libbpf/issues/278 - https://github.com/libbpf/libbpf/wiki/Libbpf:-the-road-to-v1.0#btfh-apis Signed-off-by: Quentin Monnet <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Acked-by: John Fastabend <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2021-07-29libbpf: Return non-null error on failures in libbpf_find_prog_btf_id()Quentin Monnet1-1/+3
Variable "err" is initialised to -EINVAL so that this error code is returned when something goes wrong in libbpf_find_prog_btf_id(). However, a recent change in the function made use of the variable in such a way that it is set to 0 if retrieving linear information on the program is successful, and this 0 value remains if we error out on failures at later stages. Let's fix this by setting err to -EINVAL later in the function. Fixes: e9fc3ce99b34 ("libbpf: Streamline error reporting for high-level APIs") Signed-off-by: Quentin Monnet <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2021-07-29tools/resolve_btfids: Emit warnings and patch zero id for missing symbolsHengqi Chen1-6/+7
Kernel functions referenced by .BTF_ids may be changed from global to static and get inlined or get renamed/removed, and thus disappears from BTF. This causes kernel build failure when resolve_btfids do id patch for symbols in .BTF_ids in vmlinux. Update resolve_btfids to emit warning messages and patch zero id for missing symbols instead of aborting kernel build process. Suggested-by: Andrii Nakryiko <[email protected]> Signed-off-by: Hengqi Chen <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Acked-by: Yonghong Song <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2021-07-29Merge tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvmLinus Torvalds6-2/+435
Pull kvm fixes from Paolo Bonzini: "ARM: - Fix MTE shared page detection - Enable selftest's use of PMU registers when asked to s390: - restore 5.13 debugfs names x86: - fix sizes for vcpu-id indexed arrays - fixes for AMD virtualized LAPIC (AVIC) - other small bugfixes Generic: - access tracking performance test - dirty_log_perf_test command line parsing fix - Fix selftest use of obsolete pthread_yield() in favour of sched_yield() - use cpu_relax when halt polling - fixed missing KVM_CLEAR_DIRTY_LOG compat ioctl" * tag 'for-linus' of git://git.kernel.org/pub/scm/virt/kvm/kvm: KVM: add missing compat KVM_CLEAR_DIRTY_LOG KVM: use cpu_relax when halt polling KVM: SVM: use vmcb01 in svm_refresh_apicv_exec_ctrl KVM: SVM: tweak warning about enabled AVIC on nested entry KVM: SVM: svm_set_vintr don't warn if AVIC is active but is about to be deactivated KVM: s390: restore old debugfs names KVM: SVM: delay svm_vcpu_init_msrpm after svm->vmcb is initialized KVM: selftests: Introduce access_tracking_perf_test KVM: selftests: Fix missing break in dirty_log_perf_test arg parsing x86/kvm: fix vcpu-id indexed array sizes KVM: x86: Check the right feature bit for MSR_KVM_ASYNC_PF_ACK access docs: virt: kvm: api.rst: replace some characters KVM: Documentation: Fix KVM_CAP_ENFORCE_PV_FEATURE_CPUID name KVM: nSVM: Swap the parameter order for svm_copy_vmrun_state()/svm_copy_vmloadsave_state() KVM: nSVM: Rename nested_svm_vmloadsave() to svm_copy_vmloadsave_state() KVM: arm64: selftests: get-reg-list: actually enable pmu regs in pmu sublist KVM: selftests: change pthread_yield to sched_yield KVM: arm64: Fix detection of shared VMAs on guest fault
2021-07-29Merge git://git.kernel.org/pub/scm/linux/kernel/git/bpf/bpfDavid S. Miller1-0/+229
Daniel Borkmann says: ==================== pull-request: bpf 2021-07-29 The following pull-request contains BPF updates for your *net* tree. We've added 9 non-merge commits during the last 14 day(s) which contain a total of 20 files changed, 446 insertions(+), 138 deletions(-). The main changes are: 1) Fix UBSAN out-of-bounds splat for showing XDP link fdinfo, from Lorenz Bauer. 2) Fix insufficient Spectre v4 mitigation in BPF runtime, from Daniel Borkmann, Piotr Krysiuk and Benedict Schlueter. 3) Batch of fixes for BPF sockmap found under stress testing, from John Fastabend. ==================== Signed-off-by: David S. Miller <[email protected]>
2021-07-28tc-testing: Add control-plane selftest for skbmod SKBMOD_F_ECN optionPeilin Ye1-0/+24
Recently we added a new option, SKBMOD_F_ECN, to tc-skbmod(8). Add a control-plane selftest for it. Depends on kernel patch "net/sched: act_skbmod: Add SKBMOD_F_ECN option support", as well as iproute2 patch "tc/skbmod: Introduce SKBMOD_F_ECN option". Reviewed-by: Cong Wang <[email protected]> Signed-off-by: Peilin Ye <[email protected]> Signed-off-by: David S. Miller <[email protected]>
2021-07-27bpf: Increase supported cgroup storage value sizeStanislav Fomichev3-15/+35
Current max cgroup storage value size is 4k (PAGE_SIZE). The other local storages accept up to 64k (BPF_LOCAL_STORAGE_MAX_VALUE_SIZE). Let's align max cgroup value size with the other storages. For percpu, the max is 32k (PCPU_MIN_UNIT_SIZE) because percpu allocator is not happy about larger values. netcnt test is extended to exercise those maximum values (non-percpu max size is close to, but not real max). v4: * remove inner union (Andrii Nakryiko) * keep net_cnt on the stack (Andrii Nakryiko) v3: * refine SIZEOF_BPF_LOCAL_STORAGE_ELEM comment (Yonghong Song) * anonymous struct in percpu_net_cnt & net_cnt (Yonghong Song) * reorder free (Yonghong Song) v2: * cap max_value_size instead of BUILD_BUG_ON (Martin KaFai Lau) Signed-off-by: Stanislav Fomichev <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Acked-by: Martin KaFai Lau <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2021-07-27libbpf: Fix race when pinning maps in parallelMartynas Pumputis1-1/+14
When loading in parallel multiple programs which use the same to-be pinned map, it is possible that two instances of the loader will call bpf_object__create_maps() at the same time. If the map doesn't exist when both instances call bpf_object__reuse_map(), then one of the instances will fail with EEXIST when calling bpf_map__pin(). Fix the race by retrying reusing a map if bpf_map__pin() returns EEXIST. The fix is similar to the one in iproute2: e4c4685fd6e4 ("bpf: Fix race condition with map pinning"). Before retrying the pinning, we don't do any special cleaning of an internal map state. The closer code inspection revealed that it's not required: - bpf_object__create_map(): map->inner_map is destroyed after a successful call, map->fd is closed if pinning fails. - bpf_object__populate_internal_map(): created map elements is destroyed upon close(map->fd). - init_map_slots(): slots are freed after their initialization. Signed-off-by: Martynas Pumputis <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2021-07-27libbpf: Fix comment typoJason Wang1-3/+3
Remove the repeated word 'the' in line 48. Signed-off-by: Jason Wang <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Acked-by: Yonghong Song <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2021-07-27KVM: selftests: Introduce access_tracking_perf_testDavid Matlack3-0/+431
This test measures the performance effects of KVM's access tracking. Access tracking is driven by the MMU notifiers test_young, clear_young, and clear_flush_young. These notifiers do not have a direct userspace API, however the clear_young notifier can be triggered by marking a pages as idle in /sys/kernel/mm/page_idle/bitmap. This test leverages that mechanism to enable access tracking on guest memory. To measure performance this test runs a VM with a configurable number of vCPUs that each touch every page in disjoint regions of memory. Performance is measured in the time it takes all vCPUs to finish touching their predefined region. Example invocation: $ ./access_tracking_perf_test -v 8 Testing guest mode: PA-bits:ANY, VA-bits:48, 4K pages guest physical test memory offset: 0xffdfffff000 Populating memory : 1.337752570s Writing to populated memory : 0.010177640s Reading from populated memory : 0.009548239s Mark memory idle : 23.973131748s Writing to idle memory : 0.063584496s Mark memory idle : 24.924652964s Reading from idle memory : 0.062042814s Breaking down the results: * "Populating memory": The time it takes for all vCPUs to perform the first write to every page in their region. * "Writing to populated memory" / "Reading from populated memory": The time it takes for all vCPUs to write and read to every page in their region after it has been populated. This serves as a control for the later results. * "Mark memory idle": The time it takes for every vCPU to mark every page in their region as idle through page_idle. * "Writing to idle memory" / "Reading from idle memory": The time it takes for all vCPUs to write and read to every page in their region after it has been marked idle. This test should be portable across architectures but it is only enabled for x86_64 since that's all I have tested. Reviewed-by: Ben Gardon <[email protected]> Signed-off-by: David Matlack <[email protected]> Message-Id: <[email protected]> Signed-off-by: Paolo Bonzini <[email protected]>
2021-07-27KVM: selftests: Fix missing break in dirty_log_perf_test arg parsingDavid Matlack1-0/+1
There is a missing break statement which causes a fallthrough to the next statement where optarg will be null and a segmentation fault will be generated. Fixes: 9e965bb75aae ("KVM: selftests: Add backing src parameter to dirty_log_perf_test") Reviewed-by: Ben Gardon <[email protected]> Signed-off-by: David Matlack <[email protected]> Message-Id: <[email protected]> Signed-off-by: Paolo Bonzini <[email protected]>
2021-07-27torture: Make kvm-test-1-run-qemu.sh check for reboot loopsPaul E. McKenney1-1/+5
It turns out that certain types of early boot bugs can result in reboot loops, even within a guest OS running under qemu/KVM. This commit therefore upgrades the kvm-test-1-run-qemu.sh script's hang-detection heuristics to detect such situations and to terminate the run when they occur. Signed-off-by: Paul E. McKenney <[email protected]>
2021-07-27torture: Add timestamps to kvm-test-1-run-qemu.sh outputPaul E. McKenney1-5/+5
The kvm-test-1-run-qemu.sh script logs the torture-test start time and also when it starts getting impatient for the test to finish. However, it does not timestamp these log messages, which can make debugging needlessly challenging. This commit therefore adds timestamps to these messages. Signed-off-by: Paul E. McKenney <[email protected]>
2021-07-27torture: Don't use "test" command's "-a" argumentPaul E. McKenney1-5/+5
There was a time long ago when the "test" command's documentation claimed that the "-a" and "-o" arguments did something useful. But this documentation now suggests letting the shell execute these boolean operators, so this commit applies that suggestion to kvm-test-1-run-qemu.sh. Signed-off-by: Paul E. McKenney <[email protected]>
2021-07-27torture: Make kvm-test-1-run-batch.sh select per-scenario affinity masksPaul E. McKenney4-2/+26
This commit causes kvm-test-1-run-batch.sh to use the new kvm-assign-cpus.sh and kvm-get-cpus-script.sh scripts to create a TORTURE_AFFINITY environment variable containing either an empty string (for no affinity) or a list of CPUs to pin the scenario's vCPUs to. The additional change to kvm-test-1-run.sh places the per-scenario number-of-CPUs information where it can easily be found. If there is some reason why affinity cannot be supplied, this commit prints and logs the reason via changes to kvm-again.sh. Finally, this commit updates the kvm-remote.sh script to copy the qemu-affinity output files back to the host system. Signed-off-by: Paul E. McKenney <[email protected]>
2021-07-27torture: Consistently name "qemu*" test output filesPaul E. McKenney3-10/+10
There is "qemu-affinity", "qemu-cmd", "qemu-retval", but also "qemu_pid". This is hard to remember, not so good for bash tab completion, and just plain inconsistent. This commit therefore renames the "qemu_pid" file to "qemu-pid". A couple of the scripts must deal with old runs, and thus must handle both "qemu_pid" and "qemu-pid", but new runs will produce "qemu-pid". Signed-off-by: Paul E. McKenney <[email protected]>
2021-07-27torture: Use numeric taskset argument in jitter.shPaul E. McKenney1-7/+3
The jitter.sh script has some entertaining awk code to generate a hex mask from a randomly selected CPU number, which is handed to the "taskset" command. Except that this command has a "-c" parameter to take a comma/dash-separated list of CPU numbers. This commit therefore saves a few lines of awk by switching to a single-number CPU list. Signed-off-by: Paul E. McKenney <[email protected]>
2021-07-27rcutorture: Upgrade two-CPU scenarios to four CPUsPaul E. McKenney3-3/+3
There is no way to place the vCPUs in a two-CPU rcutorture scenario to get variable memory latency. This commit therefore upgrades the current two-CPU rcutorture scenarios to four CPUs. Signed-off-by: Paul E. McKenney <[email protected]>
2021-07-27torture: Make kvm-test-1-run-qemu.sh apply affinityPaul E. McKenney1-3/+10
This commit causes the kvm-test-1-run-qemu.sh script to check the TORTURE_AFFINITY environment variable and to add "taskset" commands to the qemu-cmd file. The first "taskset" command is applied only if the TORTURE_AFFINITY environment variable is a non-empty string, and this command pins the current scenario's guest OS to the specified CPUs. The second "taskset" command reports the guest OS's affinity in a new "qemu-affinity" file. Signed-off-by: Paul E. McKenney <[email protected]>
2021-07-27torture: Don't redirect qemu-cmd comment linesPaul E. McKenney1-1/+1
Currently, kvm-test-1-run-qemu.sh applies redirection to each and every line of each qemu-cmd script. Only the first line (the only one that is not a bash comment) needs to be redirected. Although redirecting the comments is currently harmless, just adding to the comment, it is an accident waiting to happen. This commit therefore adjusts the "sed" command to redirect only the qemu-system* command itself. Signed-off-by: Paul E. McKenney <[email protected]>
2021-07-27torture: Make kvm.sh select per-scenario affinity masksPaul E. McKenney4-2/+207
This commit causes kvm.sh to use the new kvm-assign-cpus.sh and kvm-get-cpus-script.sh scripts to create a TORTURE_AFFINITY environment variable containing either an empty string (for no affinity) or a list of CPUs to pin the scenario's vCPUs to. A later commit will make use of this information to actually pin the vCPUs. Signed-off-by: Paul E. McKenney <[email protected]>
2021-07-27perf pmu: Fix alias matchingJohn Garry1-9/+24
Commit c47a5599eda324ba ("perf tools: Fix pattern matching for same substring in different PMU type"), may have fixed some alias matching, but has broken some others. Firstly it cannot handle the simple scenario of PMU name in form pmu_name{digits} - it can only handle pmu_name_{digits}. Secondly it cannot handle more complex matching in the case where we have multiple tokens. In this scenario, the code failed to realise that we may examine multiple substrings in the PMU name. Fix in two ways: - Change perf_pmu__valid_suffix() to accept a PMU name without '_' in the suffix - Only pay attention to perf_pmu__valid_suffix() for the final token Also add const qualifiers as necessary to avoid casting. Fixes: c47a5599eda324ba ("perf tools: Fix pattern matching for same substring in different PMU type") Signed-off-by: John Garry <[email protected]> Tested-by: Jin Yao <[email protected]> Cc: Alexander Shishkin <[email protected]> Cc: Ian Rogers <[email protected]> Cc: Jiri Olsa <[email protected]> Cc: Kajol Jain <[email protected]> Cc: Mark Rutland <[email protected]> Cc: Namhyung Kim <[email protected]> Cc: Peter Zijlstra <[email protected]> Link: http://lore.kernel.org/lkml/[email protected] Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>
2021-07-27perf cs-etm: Split --dump-raw-trace by AUX recordsJames Clark1-2/+18
Currently --dump-raw-trace skips queueing and splitting buffers because of an early exit condition in cs_etm__process_auxtrace_info(). Once that is removed we can print the split data by using the queues and searching for split buffers with the same reference as the one that is currently being processed. This keeps the same behaviour of dumping in file order when an AUXTRACE event appears, rather than moving trace dump to where AUX records are in the file. There will be a newline and size printout for each fragment. For example this buffer is comprised of two AUX records, but was printed as one: 0 0 0x8098 [0x30]: PERF_RECORD_AUXTRACE size: 0xa0 offset: 0 ref: 0x491a4dfc52fc0e6e idx: 0 t . ... CoreSight ETM Trace data: size 160 bytes Idx:0; ID:10; I_ASYNC : Alignment Synchronisation. Idx:12; ID:10; I_TRACE_INFO : Trace Info.; INFO=0x0 { CC.0 } Idx:17; ID:10; I_ADDR_L_64IS0 : Address, Long, 64 bit, IS0.; Addr=0x0000000000000000; Idx:80; ID:10; I_ASYNC : Alignment Synchronisation. Idx:92; ID:10; I_TRACE_INFO : Trace Info.; INFO=0x0 { CC.0 } Idx:97; ID:10; I_ADDR_L_64IS0 : Address, Long, 64 bit, IS0.; Addr=0xFFFFDE2AD3FD76D4; But is now printed as two fragments: 0 0 0x8098 [0x30]: PERF_RECORD_AUXTRACE size: 0xa0 offset: 0 ref: 0x491a4dfc52fc0e6e idx: 0 t . ... CoreSight ETM Trace data: size 80 bytes Idx:0; ID:10; I_ASYNC : Alignment Synchronisation. Idx:12; ID:10; I_TRACE_INFO : Trace Info.; INFO=0x0 { CC.0 } Idx:17; ID:10; I_ADDR_L_64IS0 : Address, Long, 64 bit, IS0.; Addr=0x0000000000000000; . ... CoreSight ETM Trace data: size 80 bytes Idx:80; ID:10; I_ASYNC : Alignment Synchronisation. Idx:92; ID:10; I_TRACE_INFO : Trace Info.; INFO=0x0 { CC.0 } Idx:97; ID:10; I_ADDR_L_64IS0 : Address, Long, 64 bit, IS0.; Addr=0xFFFFDE2AD3FD76D4; Decoding errors that appeared in problematic files are now not present, for example: Idx:808; ID:1c; I_BAD_SEQUENCE : Invalid Sequence in packet.[I_ASYNC] ... PKTP_ETMV4I_0016 : 0x0014 (OCSD_ERR_INVALID_PCKT_HDR) [Invalid packet header]; TrcIdx=822 Signed-off-by: James Clark <[email protected]> Reviewed-by: Mathieu Poirier <[email protected]> Tested-by: Leo Yan <[email protected]> Cc: Al Grant <[email protected]> Cc: Alexander Shishkin <[email protected]> Cc: Anshuman Khandual <[email protected]> Cc: Branislav Rankov <[email protected]> Cc: Denis Nikitin <[email protected]> Cc: Jiri Olsa <[email protected]> Cc: John Garry <[email protected]> Cc: Mark Rutland <[email protected]> Cc: Mike Leach <[email protected]> Cc: Namhyung Kim <[email protected]> Cc: Suzuki Poulouse <[email protected]> Cc: Will Deacon <[email protected]> Cc: [email protected] Cc: [email protected] Link: http://lore.kernel.org/lkml/[email protected] Signed-off-by: Arnaldo Carvalho de Melo <[email protected]>
2021-07-27Merge 5.14-rc3 into driver-core-nextGreg Kroah-Hartman4-9/+269
We need the driver-core fixes in here as well. Signed-off-by: Greg Kroah-Hartman <[email protected]>
2021-07-26selftests/bpf: Use ping6 only if available in tc_redirectJussi Maki1-6/+17
In the tc_redirect test only use ping6 if it's available and otherwise fall back to using "ping -6". Signed-off-by: Jussi Maki <[email protected]>
2021-07-26libbpf: Split CO-RE logic into relo_core.c.Alexei Starovoitov5-1297/+1319
Move CO-RE logic into separate file. The internal interface between libbpf and CO-RE is through bpf_core_apply_relo_insn() function and few structs defined in relo_core.h. Signed-off-by: Alexei Starovoitov <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2021-07-26libbpf: Move CO-RE types into relo_core.h.Alexei Starovoitov3-93/+102
In order to make a clean split of CO-RE logic move its types into independent header file. Signed-off-by: Alexei Starovoitov <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2021-07-26libbpf: Split bpf_core_apply_relo() into bpf_program independent helper.Alexei Starovoitov1-46/+71
bpf_core_apply_relo() doesn't need to know bpf_program internals and hashmap details. Signed-off-by: Alexei Starovoitov <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2021-07-26libbpf: Cleanup the layering between CORE and bpf_program.Alexei Starovoitov1-36/+38
CO-RE processing functions don't need to know 'struct bpf_program' details. Cleanup the layering to eventually be able to move CO-RE logic into a separate file. Signed-off-by: Alexei Starovoitov <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2021-07-26tests: add move_mount(MOVE_MOUNT_SET_GROUP) selftestPavel Tikhomirov5-0/+385
Add a simple selftest for a move_mount(MOVE_MOUNT_SET_GROUP). This tests that one can copy sharing from one mount from nested mntns with nested userns owner to another mount from other nested mntns with other nested userns owner while in their parent userns. TAP version 13 1..1 # Starting 1 tests from 2 test cases. # RUN move_mount_set_group.complex_sharing_copying ... # OK move_mount_set_group.complex_sharing_copying ok 1 move_mount_set_group.complex_sharing_copying # PASSED: 1 / 1 tests passed. # Totals: pass:1 fail:0 xfail:0 xpass:0 skip:0 error:0 Link: https://lore.kernel.org/r/[email protected] Cc: Shuah Khan <[email protected]> Cc: Eric W. Biederman <[email protected]> Cc: Alexander Viro <[email protected]> Cc: Christian Brauner <[email protected]> Cc: Mattias Nissler <[email protected]> Cc: Aleksa Sarai <[email protected]> Cc: Andrei Vagin <[email protected]> Cc: [email protected] Cc: [email protected] Cc: lkml <[email protected]> Signed-off-by: Pavel Tikhomirov <[email protected]> Signed-off-by: Christian Brauner <[email protected]>
2021-07-26Merge 5.14-rc3 into char-misc-nextGreg Kroah-Hartman4-9/+269
We need the char-misc fixes from 5.14-rc3 into here as well. Signed-off-by: Greg Kroah-Hartman <[email protected]>
2021-07-23selftest: use mmap instead of posix_memalign to allocate memoryPeter Collingbourne1-2/+4
This test passes pointers obtained from anon_allocate_area to the userfaultfd and mremap APIs. This causes a problem if the system allocator returns tagged pointers because with the tagged address ABI the kernel rejects tagged addresses passed to these APIs, which would end up causing the test to fail. To make this test compatible with such system allocators, stop using the system allocator to allocate memory in anon_allocate_area, and instead just use mmap. Link: https://lkml.kernel.org/r/[email protected] Link: https://linux-review.googlesource.com/id/Icac91064fcd923f77a83e8e133f8631c5b8fc241 Fixes: c47174fc362a ("userfaultfd: selftest") Co-developed-by: Lokesh Gidra <[email protected]> Signed-off-by: Lokesh Gidra <[email protected]> Signed-off-by: Peter Collingbourne <[email protected]> Reviewed-by: Catalin Marinas <[email protected]> Cc: Vincenzo Frascino <[email protected]> Cc: Dave Martin <[email protected]> Cc: Will Deacon <[email protected]> Cc: Andrea Arcangeli <[email protected]> Cc: Alistair Delva <[email protected]> Cc: William McVicker <[email protected]> Cc: Evgenii Stepanov <[email protected]> Cc: Mitch Phillips <[email protected]> Cc: Andrey Konovalov <[email protected]> Cc: <[email protected]> [5.4] Signed-off-by: Andrew Morton <[email protected]> Signed-off-by: Linus Torvalds <[email protected]>
2021-07-23selftests/bpf: Document vmtest.sh dependenciesEvgeniy Litvinenko1-0/+7
Add a list of vmtest script dependencies to make it easier for new contributors to get going. Signed-off-by: Evgeniy Litvinenko <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2021-07-23libbpf: Add bpf_map__pin_path functionEvgeniy Litvinenko4-0/+16
Add bpf_map__pin_path, so that the inconsistently named bpf_map__get_pin_path can be deprecated later. This is part of the effort towards libbpf v1.0: https://github.com/libbpf/libbpf/issues/307 Also, add a selftest for the new function. Signed-off-by: Evgeniy Litvinenko <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2021-07-23bpf: selftest: Test batching and bpf_(get|set)sockopt in bpf tcp iterMartin KaFai Lau5-9/+384
This patch adds tests for the batching and bpf_(get|set)sockopt in bpf tcp iter. It first creates: a) 1 non SO_REUSEPORT listener in lhash2. b) 256 passive and active fds connected to the listener in (a). c) 256 SO_REUSEPORT listeners in one of the lhash2 bucket. The test sets all listeners and connections to bpf_cubic before running the bpf iter. The bpf iter then calls setsockopt(TCP_CONGESTION) to switch each listener and connection from bpf_cubic to bpf_dctcp. The bpf iter has a random_retry mode such that it can return EAGAIN to the usespace in the middle of a batch. Signed-off-by: Martin KaFai Lau <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Reviewed-by: Eric Dumazet <[email protected]> Acked-by: Kuniyuki Iwashima <[email protected]> Acked-by: Yonghong Song <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2021-07-23Merge git://git.kernel.org/pub/scm/linux/kernel/git/netdev/netDavid S. Miller54-126/+774
Conflicts are simple overlapping changes. Signed-off-by: David S. Miller <[email protected]>