aboutsummaryrefslogtreecommitdiff
AgeCommit message (Collapse)AuthorFilesLines
2023-03-03selftests/bpf: Tweak cgroup kfunc test.Alexei Starovoitov1-1/+11
Adjust cgroup kfunc test to dereference RCU protected cgroup pointer as PTR_TRUSTED and pass into KF_TRUSTED_ARGS kfunc. Signed-off-by: Alexei Starovoitov <[email protected]> Signed-off-by: Daniel Borkmann <[email protected]> Acked-by: David Vernet <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2023-03-03selftests/bpf: Add a test case for kptr_rcu.Alexei Starovoitov1-0/+12
Tweak existing map_kptr test to check kptr_rcu. Signed-off-by: Alexei Starovoitov <[email protected]> Signed-off-by: Daniel Borkmann <[email protected]> Acked-by: David Vernet <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2023-03-03bpf: Introduce kptr_rcu.Alexei Starovoitov9-23/+65
The life time of certain kernel structures like 'struct cgroup' is protected by RCU. Hence it's safe to dereference them directly from __kptr tagged pointers in bpf maps. The resulting pointer is MEM_RCU and can be passed to kfuncs that expect KF_RCU. Derefrence of other kptr-s returns PTR_UNTRUSTED. For example: struct map_value { struct cgroup __kptr *cgrp; }; SEC("tp_btf/cgroup_mkdir") int BPF_PROG(test_cgrp_get_ancestors, struct cgroup *cgrp_arg, const char *path) { struct cgroup *cg, *cg2; cg = bpf_cgroup_acquire(cgrp_arg); // cg is PTR_TRUSTED and ref_obj_id > 0 bpf_kptr_xchg(&v->cgrp, cg); cg2 = v->cgrp; // This is new feature introduced by this patch. // cg2 is PTR_MAYBE_NULL | MEM_RCU. // When cg2 != NULL, it's a valid cgroup, but its percpu_ref could be zero if (cg2) bpf_cgroup_ancestor(cg2, level); // safe to do. } Signed-off-by: Alexei Starovoitov <[email protected]> Signed-off-by: Daniel Borkmann <[email protected]> Acked-by: Tejun Heo <[email protected]> Acked-by: David Vernet <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2023-03-03bpf: Mark cgroups and dfl_cgrp fields as trusted.Alexei Starovoitov1-0/+6
bpf programs sometimes do: bpf_cgrp_storage_get(&map, task->cgroups->dfl_cgrp, ...); It is safe to do, because cgroups->dfl_cgrp pointer is set diring init and never changes. The task->cgroups is also never NULL. It is also set during init and will change when task switches cgroups. For any trusted task pointer dereference of cgroups and dfl_cgrp should yield trusted pointers. The verifier wasn't aware of this. Hence in gcc compiled kernels task->cgroups dereference was producing PTR_TO_BTF_ID without modifiers while in clang compiled kernels the verifier recognizes __rcu tag in cgroups field and produces PTR_TO_BTF_ID | MEM_RCU | MAYBE_NULL. Tag cgroups and dfl_cgrp as trusted to equalize clang and gcc behavior. When GCC supports btf_type_tag such tagging will done directly in the type. Signed-off-by: Alexei Starovoitov <[email protected]> Signed-off-by: Daniel Borkmann <[email protected]> Acked-by: David Vernet <[email protected]> Acked-by: Tejun Heo <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2023-03-03bpf: Rename __kptr_ref -> __kptr and __kptr -> __kptr_untrusted.Alexei Starovoitov14-30/+30
__kptr meant to store PTR_UNTRUSTED kernel pointers inside bpf maps. The concept felt useful, but didn't get much traction, since bpf_rdonly_cast() was added soon after and bpf programs received a simpler way to access PTR_UNTRUSTED kernel pointers without going through restrictive __kptr usage. Rename __kptr_ref -> __kptr and __kptr -> __kptr_untrusted to indicate its intended usage. The main goal of __kptr_untrusted was to read/write such pointers directly while bpf_kptr_xchg was a mechanism to access refcnted kernel pointers. The next patch will allow RCU protected __kptr access with direct read. At that point __kptr_untrusted will be deprecated. Signed-off-by: Alexei Starovoitov <[email protected]> Signed-off-by: Daniel Borkmann <[email protected]> Acked-by: David Vernet <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2023-03-02selftests/bpf: Add absolute timer testTero Kristo2-0/+48
Add test for the absolute BPF timer under the existing timer tests. This will run the timer two times with 1us expiration time, and then re-arm the timer at ~35s in the future. At the end, it is verified that the absolute timer expired exactly two times. Signed-off-by: Tero Kristo <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Alexei Starovoitov <[email protected]>
2023-03-02bpf: Add support for absolute value BPF timersTero Kristo3-2/+39
Add a new flag BPF_F_TIMER_ABS that can be passed to bpf_timer_start() to start an absolute value timer instead of the default relative value. This makes the timer expire at an exact point in time, instead of a time with latencies induced by both the BPF and timer subsystems. Suggested-by: Artem Bityutskiy <[email protected]> Signed-off-by: Tero Kristo <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Alexei Starovoitov <[email protected]>
2023-03-02selftests/bpf: Add -Wuninitialized flag to bpf prog flagsDave Marchevsky6-11/+14
Per C99 standard [0], Section 6.7.8, Paragraph 10: If an object that has automatic storage duration is not initialized explicitly, its value is indeterminate. And in the same document, in appendix "J.2 Undefined behavior": The behavior is undefined in the following circumstances: [...] The value of an object with automatic storage duration is used while it is indeterminate (6.2.4, 6.7.8, 6.8). This means that use of an uninitialized stack variable is undefined behavior, and therefore that clang can choose to do a variety of scary things, such as not generating bytecode for "bunch of useful code" in the below example: void some_func() { int i; if (!i) return; // bunch of useful code } To add insult to injury, if some_func above is a helper function for some BPF program, clang can choose to not generate an "exit" insn, causing verifier to fail with "last insn is not an exit or jmp". Going from that verification failure to the root cause of uninitialized use is certain to be frustrating. This patch adds -Wuninitialized to the cflags for selftest BPF progs and fixes up existing instances of uninitialized use. [0]: https://www.open-std.org/jtc1/sc22/WG14/www/docs/n1256.pdf Signed-off-by: Dave Marchevsky <[email protected]> Cc: David Vernet <[email protected]> Cc: Tejun Heo <[email protected]> Acked-by: David Vernet <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Alexei Starovoitov <[email protected]>
2023-03-02bpf: Make bpf_get_current_[ancestor_]cgroup_id() available for all program typesTejun Heo4-14/+4
These helpers are safe to call from any context and there's no reason to restrict access to them. Remove them from bpf_trace and filter lists and add to bpf_base_func_proto() under perfmon_capable(). v2: After consulting with Andrii, relocated in bpf_base_func_proto() so that they require bpf_capable() but not perfomon_capable() as it doesn't read from or affect others on the system. Signed-off-by: Tejun Heo <[email protected]> Cc: Andrii Nakryiko <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Alexei Starovoitov <[email protected]>
2023-03-02bpf, docs: Fix final bpf docs build failureDavid Vernet1-3/+4
maps.rst in the BPF documentation links to the /userspace-api/ebpf/syscall document (Documentation/userspace-api/ebpf/syscall.rst). For some reason, if you try to reference the document with :doc:, the docs build emits the following warning: ./Documentation/bpf/maps.rst:13: WARNING: \ unknown document: '/userspace-api/ebpf/syscall' It appears that other places in the docs tree also don't support using :doc:. Elsewhere in the BPF documentation, we just reference the kernel docs page directly. Let's do that here to clean up the last remaining noise in the docs build. Signed-off-by: David Vernet <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Alexei Starovoitov <[email protected]>
2023-03-02bpf, docs: Fix link to netdev-FAQ targetDavid Vernet1-7/+7
The BPF devel Q&A documentation page makes frequent reference to the netdev-QA page via the netdev-FAQ rst link. This link is currently broken, as is evidenced by the build output when making BPF docs: ./Documentation/bpf/bpf_devel_QA.rst:150: WARNING: undefined label: 'netdev-faq' ./Documentation/bpf/bpf_devel_QA.rst:206: WARNING: undefined label: 'netdev-faq' ./Documentation/bpf/bpf_devel_QA.rst:231: WARNING: undefined label: 'netdev-faq' ./Documentation/bpf/bpf_devel_QA.rst:396: WARNING: undefined label: 'netdev-faq' ./Documentation/bpf/bpf_devel_QA.rst:412: WARNING: undefined label: 'netdev-faq' Fix the links to point to the actual netdev-faq page. Signed-off-by: David Vernet <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Alexei Starovoitov <[email protected]>
2023-03-01bpf: Fix bpf_dynptr_slice{_rdwr} to return NULL instead of 0Joanne Koong1-4/+4
Change bpf_dynptr_slice and bpf_dynptr_slice_rdwr to return NULL instead of 0, in accordance with the codebase guidelines. Fixes: 66e3a13e7c2c ("bpf: Add bpf_dynptr_slice and bpf_dynptr_slice_rdwr") Reported-by: kernel test robot <[email protected]> Signed-off-by: Joanne Koong <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2023-03-01Merge branch 'Make uprobe attachment APK aware'Andrii Nakryiko4-28/+495
Daniel Müller says: ==================== On Android, APKs (android packages; zip packages with somewhat prescriptive contents) are first class citizens in the system: the shared objects contained in them don't exist in unpacked form on the file system. Rather, they are mmaped directly from within the archive and the archive is also what the kernel is aware of. For users that complicates the process of attaching a uprobe to a function contained in a shared object in one such APK: they'd have to find the byte offset of said function from the beginning of the archive. That is cumbersome to do manually and can be fragile, because various changes could invalidate said offset. That is why for uprobes inside ELF files (not inside an APK), commit d112c9ce249b ("libbpf: Support function name-based attach uprobes") added support for attaching to symbols by name. On Android, that mechanism currently does not work, because this logic is not APK aware. This patch set introduces first class support for attaching uprobes to functions inside ELF objects contained in APKs via function names. We add support for recognizing the following syntax for a binary path: <archive>!/<binary-in-archive> (e.g., /system/app/test-app.apk!/lib/arm64-v8a/libc++.so) This syntax is common in the Android eco system and used by tools such as simpleperf. It is also what is being proposed for bcc [0]. If the user provides such a binary path, we find <binary-in-archive> (lib/arm64-v8a/libc++.so in the example) inside of <archive> (/system/app/test-app.apk). We perform the regular ELF offset search inside the binary and add that to the offset within the archive itself, to retrieve the offset at which to attach the uprobe. [0] https://github.com/iovisor/bcc/pull/4440 Changelog --------- v3->v4: - use ERR_PTR instead of libbpf_err_ptr() in zip_archive_open() - eliminated err variable from elf_find_func_offset_from_archive() v2->v3: - adjusted zip_archive_open() to report errno - fixed provided libbpf_strlcpy() buffer size argument - adjusted find_cd() to handle errors better - use fewer local variables in get_entry_at_offset() v1->v2: - removed unaligned_* types - switched to using __u32 and __u16 - switched to using errno constants instead of hard-coded negative values - added another pr_debug() message - shortened central_directory_* to cd_* - inlined cd_file_header_at_offset() function - bunch of syntactical changes ==================== Signed-off-by: Andrii Nakryiko <[email protected]>
2023-03-01libbpf: Add support for attaching uprobes to shared objects in APKsDaniel Müller1-7/+84
This change adds support for attaching uprobes to shared objects located in APKs, which is relevant for Android systems where various libraries may reside in APKs. To make that happen, we extend the syntax for the "binary path" argument to attach to with that supported by various Android tools: <archive>!/<binary-in-archive> For example: /system/app/test-app/test-app.apk!/lib/arm64-v8a/libc++_shared.so APKs need to be specified via full path, i.e., we do not attempt to resolve mere file names by searching system directories. We cannot currently test this functionality end-to-end in an automated fashion, because it relies on an Android system being present, but there is no support for that in CI. I have tested the functionality manually, by creating a libbpf program containing a uretprobe, attaching it to a function inside a shared object inside an APK, and verifying the sanity of the returned values. Signed-off-by: Daniel Müller <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2023-03-01libbpf: Introduce elf_find_func_offset_from_file() functionDaniel Müller1-21/+36
This change splits the elf_find_func_offset() function in two: elf_find_func_offset(), which now accepts an already opened Elf object instead of a path to a file that is to be opened, as well as elf_find_func_offset_from_file(), which opens a binary based on a path and then invokes elf_find_func_offset() on the Elf object. Having this split in responsibilities will allow us to call elf_find_func_offset() from other code paths on Elf objects that did not necessarily come from a file on disk. Signed-off-by: Daniel Müller <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2023-03-01libbpf: Implement basic zip archive parsing supportDaniel Müller3-1/+376
This change implements support for reading zip archives, including opening an archive, finding an entry based on its path and name in it, and closing it. The code was copied from https://github.com/iovisor/bcc/pull/4440, which implements similar functionality for bcc. The author confirmed that he is fine with this usage and the corresponding relicensing. I adjusted it to adhere to libbpf coding standards. Signed-off-by: Daniel Müller <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Acked-by: Michał Gregorczyk <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2023-03-01bpf, docs: Fix __uninit kfunc doc sectionDavid Vernet1-1/+1
In commit d96d937d7c5c ("bpf: Add __uninit kfunc annotation"), the __uninit kfunc annotation was documented in kfuncs.rst. You have to fully underline a section in rst, or the build will issue a warning that the title underline is too short: ./Documentation/bpf/kfuncs.rst:104: WARNING: Title underline too short. 2.2.2 __uninit Annotation -------------------- This patch fixes that title underline. Fixes: d96d937d7c5c ("bpf: Add __uninit kfunc annotation") Signed-off-by: David Vernet <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Alexei Starovoitov <[email protected]>
2023-03-01bpf: Fix doxygen comments for dynptr slice kfuncsDavid Vernet1-16/+14
In commit 66e3a13e7c2c ("bpf: Add bpf_dynptr_slice and bpf_dynptr_slice_rdwr"), the bpf_dynptr_slice() and bpf_dynptr_slice_rdwr() kfuncs were added to BPF. These kfuncs included doxygen headers, but unfortunately those headers are not properly formatted according to [0], and causes the following warnings during the docs build: ./kernel/bpf/helpers.c:2225: warning: \ Excess function parameter 'returns' description in 'bpf_dynptr_slice' ./kernel/bpf/helpers.c:2303: warning: \ Excess function parameter 'returns' description in 'bpf_dynptr_slice_rdwr' ... This patch fixes those doxygen comments. [0]: https://docs.kernel.org/doc-guide/kernel-doc.html#function-documentation Fixes: 66e3a13e7c2c ("bpf: Add bpf_dynptr_slice and bpf_dynptr_slice_rdwr") Signed-off-by: David Vernet <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Alexei Starovoitov <[email protected]>
2023-03-01Merge branch 'selftests/bpf: support custom per-test flags and multiple ↵Andrii Nakryiko3-9/+84
expected messages' Eduard Zingerman says: ==================== This patch allows to specify program flags and multiple verifier log messages for the test_loader kind of tests. For example: tools/testing/selftets/bpf/progs/foobar.c: SEC("tc") __success __log_level(7) __msg("first message") __msg("next message") __flag(BPF_F_ANY_ALIGNMENT) int buz(struct __sk_buff *skb) { ... } It was developed by Andrii Nakryiko ([1]), I reused it in a "test_verifier tests migration to inline assembly" patch series ([2]), but the series is currently stuck on my side. Andrii asked to spin this particular patch separately ([3]). [1] https://lore.kernel.org/bpf/CAEf4BzZH0ZxorCi7nPDbRqSK9f+410RooNwNJGwfw8=0a5i1nw@mail.gmail.com/ [2] https://lore.kernel.org/bpf/[email protected]/ [3] https://lore.kernel.org/bpf/[email protected]/T/#m52e806c5a679a2aa8f484d011be7ec105939127a ==================== Signed-off-by: Andrii Nakryiko <[email protected]>
2023-03-01selftests/bpf: Support custom per-test flags and multiple expected messagesAndrii Nakryiko3-9/+84
Extend __flag attribute by allowing to specify one of the following: * BPF_F_STRICT_ALIGNMENT * BPF_F_ANY_ALIGNMENT * BPF_F_TEST_RND_HI32 * BPF_F_TEST_STATE_FREQ * BPF_F_SLEEPABLE * BPF_F_XDP_HAS_FRAGS * Some numeric value Extend __msg attribute by allowing to specify multiple exepcted messages. All messages are expected to be present in the verifier log in the order of application. Signed-off-by: Andrii Nakryiko <[email protected]> Signed-off-by: Eduard Zingerman <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Link: https://lore.kernel.org/bpf/[email protected] [ Eduard: added commit message, formatting, comments ]
2023-03-01Merge branch 'libbpf: fix several issues reported by static analysers'Andrii Nakryiko4-16/+3
Viktor Malik says: ==================== Fixing several issues reported by Coverity and Clang Static Analyzer (scan-build) for libbpf, mostly removing unnecessary symbols and assignments. No functional changes should be introduced. ==================== Signed-off-by: Andrii Nakryiko <[email protected]>
2023-03-01libbpf: Cleanup linker_append_elf_relosViktor Malik1-9/+2
Clang Static Analyser (scan-build) reports some unused symbols and dead assignments in the linker_append_elf_relos function. Clean these up. Signed-off-by: Viktor Malik <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Link: https://lore.kernel.org/bpf/c5c8fe9f411b69afada8399d23bb048ef2a70535.1677658777.git.vmalik@redhat.com
2023-03-01libbpf: Remove several dead assignmentsViktor Malik3-6/+0
Clang Static Analyzer (scan-build) reports several dead assignments in libbpf where the assigned value is unconditionally overridden by another value before it is read. Remove these assignments. Signed-off-by: Viktor Malik <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Link: https://lore.kernel.org/bpf/5503d18966583e55158471ebbb2f67374b11bf5e.1677658777.git.vmalik@redhat.com
2023-03-01libbpf: Remove unnecessary ternary operatorViktor Malik1-1/+1
Coverity reports that the first check of 'err' in bpf_object__init_maps is always false as 'err' is initialized to 0 at that point. Remove the unnecessary ternary operator. Signed-off-by: Viktor Malik <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Link: https://lore.kernel.org/bpf/78a3702f2ea9f32a84faaae9b674c56269d330a7.1677658777.git.vmalik@redhat.com
2023-03-01selftests/bpf: Set __BITS_PER_LONG if target is bpf for LoongArchTiezhu Yang1-1/+2
If target is bpf, there is no __loongarch__ definition, __BITS_PER_LONG defaults to 32, __NR_nanosleep is not defined: #if defined(__ARCH_WANT_TIME32_SYSCALLS) || __BITS_PER_LONG != 32 #define __NR_nanosleep 101 __SC_3264(__NR_nanosleep, sys_nanosleep_time32, sys_nanosleep) #endif Work around this problem, by explicitly setting __BITS_PER_LONG to __loongarch_grlen which is defined by compiler as 64 for LA64. This is similar with commit 36e70b9b06bf ("selftests, bpf: Fix broken riscv build"). Signed-off-by: Tiezhu Yang <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2023-03-01Merge branch 'Add support for kptrs in more BPF maps'Alexei Starovoitov8-96/+553
Kumar Kartikeya Dwivedi says: ==================== This set adds support for kptrs in percpu hashmaps, percpu LRU hashmaps, and local storage maps (covering sk, cgrp, task, inode). Tests are expanded to test more existing maps at runtime and also test the code path for the local storage maps (which is shared by all implementations). A question for reviewers is what the position of the BPF runtime should be on dealing with reference cycles that can be created by BPF programs at runtime using this additional support. For instance, one can store the kptr of the task in its own task local storage, creating a cycle which prevents destruction of task local storage. Cycles can be formed using arbitrarily long kptr ownership chains. Therefore, just preventing storage of such kptrs in some maps is not a sufficient solution, and is more likely to hurt usability. There is precedence in existing runtimes which promise memory safety, like Rust, where reference cycles and memory leaks are permitted. However, traditionally the safety guarantees of BPF have been stronger. Thus, more discussion and thought is invited on this topic to ensure we cover all usage aspects. Changelog: ---------- v2 -> v3 v2: https://lore.kernel.org/bpf/[email protected]/ * Fix a use-after-free bug in local storage patch * Fix selftest for aarch64 (don't use fentry/fmod_ret) * Wait for RCU Tasks Trace GP along with RCU GP in selftest v1 -> v2 v1: https://lore.kernel.org/bpf/[email protected] * Simplify selftests, fix a couple of bugs ==================== Signed-off-by: Alexei Starovoitov <[email protected]>
2023-03-01selftests/bpf: Add more tests for kptrs in mapsKumar Kartikeya Dwivedi3-65/+451
Firstly, ensure programs successfully load when using all of the supported maps. Then, extend existing tests to test more cases at runtime. We are currently testing both the synchronous freeing of items and asynchronous destruction when map is freed, but the code needs to be adjusted a bit to be able to also accomodate support for percpu maps. We now do a delete on the item (and update for array maps which has a similar effect for kptrs) to perform a synchronous free of the kptr, and test destruction both for the synchronous and asynchronous deletion. Next time the program runs, it should observe the refcount as 1 since all existing references should have been released by then. By running the program after both possible paths freeing kptrs, we establish that they correctly release resources. Next, we augment the existing test to also test the same code path shared by all local storage maps using a task local storage map. Signed-off-by: Kumar Kartikeya Dwivedi <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Alexei Starovoitov <[email protected]>
2023-03-01bpf: Support kptrs in local storage mapsKumar Kartikeya Dwivedi4-9/+63
Enable support for kptrs in local storage maps by wiring up the freeing of these kptrs from map value. Freeing of bpf_local_storage_map is only delayed in case there are special fields, therefore bpf_selem_free_* path can also only dereference smap safely in that case. This is recorded using a bool utilizing a hole in bpF_local_storage_elem. It could have been tagged in the pointer value smap using the lowest bit (since alignment > 1), but since there was already a hole I went with the simpler option. Only the map structure freeing is delayed using RCU barriers, as the buckets aren't used when selem is being freed, so they can be freed once all readers of the bucket lists can no longer access it. Cc: Martin KaFai Lau <[email protected]> Cc: KP Singh <[email protected]> Cc: Paul E. McKenney <[email protected]> Signed-off-by: Kumar Kartikeya Dwivedi <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Alexei Starovoitov <[email protected]>
2023-03-01bpf: Support kptrs in percpu hashmap and percpu LRU hashmapKumar Kartikeya Dwivedi2-22/+39
Enable support for kptrs in percpu BPF hashmap and percpu BPF LRU hashmap by wiring up the freeing of these kptrs from percpu map elements. Signed-off-by: Kumar Kartikeya Dwivedi <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Alexei Starovoitov <[email protected]>
2023-03-01Merge branch 'Add skb + xdp dynptrs'Alexei Starovoitov25-186/+3322
Joanne Koong says: ==================== This patchset is the 2nd in the dynptr series. The 1st can be found here [0]. This patchset adds skb and xdp type dynptrs, which have two main benefits for packet parsing: * allowing operations on sizes that are not statically known at compile-time (eg variable-sized accesses). * more ergonomic and less brittle iteration through data (eg does not need manual if checking for being within bounds of data_end) When comparing the differences in runtime for packet parsing without dynptrs vs. with dynptrs, there is no noticeable difference. Patch 9 contains more details as well as examples of how to use skb and xdp dynptrs. [0] https://lore.kernel.org/bpf/[email protected]/ --- Changelog: v12 = https://lore.kernel.org/bpf/[email protected]/ v12 -> v13: * Fix missing { } for case statement v11 = https://lore.kernel.org/bpf/[email protected]/ v11 -> v12: * Change constant mem size checking to use "__szk" kfunc annotation for slices * Use autoloading for success selftests v10 = https://lore.kernel.org/bpf/[email protected]/ v10 -> v11: * Reject bpf_dynptr_slice_rdwr() for non-writable progs at load time instead of runtime * Add additional patch (__uninit kfunc annotation) * Expand on documentation * Add bpf_dynptr_write() calls for persisting writes in tests v9 = https://lore.kernel.org/bpf/[email protected]/ v9 -> v10: * Add bpf_dynptr_slice and bpf_dynptr_slice_rdwr interface * Add some more tests * Split up patchset into more parts to make it easier to review v8 = https://lore.kernel.org/bpf/[email protected]/ v8 -> v9: * Fix dynptr_get_type() to check non-stack dynptrs v7 = https://lore.kernel.org/bpf/[email protected]/ v7 -> v8: * Change helpers to kfuncs * Add 2 new patches (1/5 and 2/5) v6 = https://lore.kernel.org/bpf/[email protected]/ v6 -> v7 * Change bpf_dynptr_data() to return read-only data slices if the skb prog is read-only (Martin) * Add test "skb_invalid_write" to test that writes to rd-only data slices are rejected v5 = https://lore.kernel.org/bpf/[email protected]/ v5 -> v6 * Address kernel test robot errors by static inlining v4 = https://lore.kernel.org/bpf/[email protected]/ v4 -> v5 * Address kernel test robot errors for configs w/out CONFIG_NET set * For data slices, return PTR_TO_MEM instead of PTR_TO_PACKET (Kumar) * Split selftests into subtests (Andrii) * Remove insn patching. Use rdonly and rdwr protos for dynptr skb construction (Andrii) * bpf_dynptr_data() returns NULL for rd-only dynptrs. There will be a separate bpf_dynptr_data_rdonly() added later (Andrii and Kumar) v3 = https://lore.kernel.org/bpf/[email protected]/ v3 -> v4 * Forgot to commit --amend the kernel test robot error fixups v2 = https://lore.kernel.org/bpf/[email protected]/ v2 -> v3 * Fix kernel test robot build test errors v1 = https://lore.kernel.org/bpf/[email protected]/ v1 -> v2 * Return data slices to rd-only skb dynptrs (Martin) * bpf_dynptr_write allows writes to frags for skb dynptrs, but always invalidates associated data slices (Martin) * Use switch casing instead of ifs (Andrii) * Use 0xFD for experimental kind number in the selftest (Zvi) * Put selftest conversions w/ dynptrs into new files (Alexei) * Add new selftest "test_cls_redirect_dynptr.c" ==================== Signed-off-by: Alexei Starovoitov <[email protected]>
2023-03-01selftests/bpf: tests for using dynptrs to parse skb and xdp buffersJoanne Koong15-23/+2522
Test skb and xdp dynptr functionality in the following ways: 1) progs/test_cls_redirect_dynptr.c * Rewrite "progs/test_cls_redirect.c" test to use dynptrs to parse skb data * This is a great example of how dynptrs can be used to simplify a lot of the parsing logic for non-statically known values. When measuring the user + system time between the original version vs. using dynptrs, and averaging the time for 10 runs (using "time ./test_progs -t cls_redirect"): original version: 0.092 sec with dynptrs: 0.078 sec 2) progs/test_xdp_dynptr.c * Rewrite "progs/test_xdp.c" test to use dynptrs to parse xdp data When measuring the user + system time between the original version vs. using dynptrs, and averaging the time for 10 runs (using "time ./test_progs -t xdp_attach"): original version: 0.118 sec with dynptrs: 0.094 sec 3) progs/test_l4lb_noinline_dynptr.c * Rewrite "progs/test_l4lb_noinline.c" test to use dynptrs to parse skb data When measuring the user + system time between the original version vs. using dynptrs, and averaging the time for 10 runs (using "time ./test_progs -t l4lb_all"): original version: 0.062 sec with dynptrs: 0.081 sec For number of processed verifier instructions: original version: 6268 insns with dynptrs: 2588 insns 4) progs/test_parse_tcp_hdr_opt_dynptr.c * Add sample code for parsing tcp hdr opt lookup using dynptrs. This logic is lifted from a real-world use case of packet parsing in katran [0], a layer 4 load balancer. The original version "progs/test_parse_tcp_hdr_opt.c" (not using dynptrs) is included here as well, for comparison. When measuring the user + system time between the original version vs. using dynptrs, and averaging the time for 10 runs (using "time ./test_progs -t parse_tcp_hdr_opt"): original version: 0.031 sec with dynptrs: 0.045 sec 5) progs/dynptr_success.c * Add test case "test_skb_readonly" for testing attempts at writes on a prog type with read-only skb ctx. * Add "test_dynptr_skb_data" for testing that bpf_dynptr_data isn't supported for skb progs. 6) progs/dynptr_fail.c * Add test cases "skb_invalid_data_slice{1,2,3,4}" and "xdp_invalid_data_slice{1,2}" for testing that helpers that modify the underlying packet buffer automatically invalidate the associated data slice. * Add test cases "skb_invalid_ctx" and "xdp_invalid_ctx" for testing that prog types that do not support bpf_dynptr_from_skb/xdp don't have access to the API. * Add test case "dynptr_slice_var_len{1,2}" for testing that variable-sized len can't be passed in to bpf_dynptr_slice * Add test case "skb_invalid_slice_write" for testing that writes to a read-only data slice are rejected by the verifier. * Add test case "data_slice_out_of_bounds_skb" for testing that writes to an area outside the slice are rejected. * Add test case "invalid_slice_rdwr_rdonly" for testing that prog types that don't allow writes to packet data don't accept any calls to bpf_dynptr_slice_rdwr. [0] https://github.com/facebookincubator/katran/blob/main/katran/lib/bpf/pckt_parsing.h Signed-off-by: Joanne Koong <[email protected]> Acked-by: Andrii Nakryiko <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Alexei Starovoitov <[email protected]>
2023-03-01bpf: Add bpf_dynptr_slice and bpf_dynptr_slice_rdwrJoanne Koong6-7/+288
Two new kfuncs are added, bpf_dynptr_slice and bpf_dynptr_slice_rdwr. The user must pass in a buffer to store the contents of the data slice if a direct pointer to the data cannot be obtained. For skb and xdp type dynptrs, these two APIs are the only way to obtain a data slice. However, for other types of dynptrs, there is no difference between bpf_dynptr_slice(_rdwr) and bpf_dynptr_data. For skb type dynptrs, the data is copied into the user provided buffer if any of the data is not in the linear portion of the skb. For xdp type dynptrs, the data is copied into the user provided buffer if the data is between xdp frags. If the skb is cloned and a call to bpf_dynptr_data_rdwr is made, then the skb will be uncloned (see bpf_unclone_prologue()). Please note that any bpf_dynptr_write() automatically invalidates any prior data slices of the skb dynptr. This is because the skb may be cloned or may need to pull its paged buffer into the head. As such, any bpf_dynptr_write() will automatically have its prior data slices invalidated, even if the write is to data in the skb head of an uncloned skb. Please note as well that any other helper calls that change the underlying packet buffer (eg bpf_skb_pull_data()) invalidates any data slices of the skb dynptr as well, for the same reasons. Signed-off-by: Joanne Koong <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Alexei Starovoitov <[email protected]>
2023-03-01bpf: Add xdp dynptrsJoanne Koong7-6/+76
Add xdp dynptrs, which are dynptrs whose underlying pointer points to a xdp_buff. The dynptr acts on xdp data. xdp dynptrs have two main benefits. One is that they allow operations on sizes that are not statically known at compile-time (eg variable-sized accesses). Another is that parsing the packet data through dynptrs (instead of through direct access of xdp->data and xdp->data_end) can be more ergonomic and less brittle (eg does not need manual if checking for being within bounds of data_end). For reads and writes on the dynptr, this includes reading/writing from/to and across fragments. Data slices through the bpf_dynptr_data API are not supported; instead bpf_dynptr_slice() and bpf_dynptr_slice_rdwr() should be used. For examples of how xdp dynptrs can be used, please see the attached selftests. Signed-off-by: Joanne Koong <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Alexei Starovoitov <[email protected]>
2023-03-01bpf: Add skb dynptrsJoanne Koong8-19/+261
Add skb dynptrs, which are dynptrs whose underlying pointer points to a skb. The dynptr acts on skb data. skb dynptrs have two main benefits. One is that they allow operations on sizes that are not statically known at compile-time (eg variable-sized accesses). Another is that parsing the packet data through dynptrs (instead of through direct access of skb->data and skb->data_end) can be more ergonomic and less brittle (eg does not need manual if checking for being within bounds of data_end). For bpf prog types that don't support writes on skb data, the dynptr is read-only (bpf_dynptr_write() will return an error) For reads and writes through the bpf_dynptr_read() and bpf_dynptr_write() interfaces, reading and writing from/to data in the head as well as from/to non-linear paged buffers is supported. Data slices through the bpf_dynptr_data API are not supported; instead bpf_dynptr_slice() and bpf_dynptr_slice_rdwr() (added in subsequent commit) should be used. For examples of how skb dynptrs can be used, please see the attached selftests. Signed-off-by: Joanne Koong <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Alexei Starovoitov <[email protected]>
2023-03-01bpf: Add __uninit kfunc annotationJoanne Koong2-2/+33
This patch adds __uninit as a kfunc annotation. This will be useful for scenarios such as for example in dynptrs, indicating whether the dynptr should be checked by the verifier as an initialized or an uninitialized dynptr. Without this annotation, the alternative would be needing to hard-code in the verifier the specific kfunc to indicate that arg should be treated as an uninitialized arg. Signed-off-by: Joanne Koong <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Alexei Starovoitov <[email protected]>
2023-03-01bpf: Refactor verifier dynptr into get_dynptr_arg_regJoanne Koong1-30/+50
This commit refactors the logic for determining which register in a function is the dynptr into "get_dynptr_arg_reg". This will be used in the future when the dynptr reg for BPF_FUNC_dynptr_write will need to be obtained in order to support writes for skb dynptrs. Signed-off-by: Joanne Koong <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Alexei Starovoitov <[email protected]>
2023-03-01bpf: Define no-ops for externally called bpf dynptr functionsJoanne Koong1-30/+45
Some bpf dynptr functions will be called from places where if CONFIG_BPF_SYSCALL is not set, then the dynptr function is undefined. For example, when skb type dynptrs are added in the next commit, dynptr functions are called from net/core/filter.c This patch defines no-op implementations of these dynptr functions so that they do not break compilation by being an undefined reference. Signed-off-by: Joanne Koong <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Alexei Starovoitov <[email protected]>
2023-03-01bpf: Allow initializing dynptrs in kfuncsJoanne Koong1-45/+22
This change allows kfuncs to take in an uninitialized dynptr as a parameter. Before this change, only helper functions could successfully use uninitialized dynptrs. This change moves the memory access check (including stack state growing and slot marking) into process_dynptr_func(), which both helpers and kfuncs call into. Signed-off-by: Joanne Koong <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Alexei Starovoitov <[email protected]>
2023-03-01bpf: Refactor process_dynptr_funcJoanne Koong2-34/+31
This change cleans up process_dynptr_func's flow to be more intuitive and updates some comments with more context. Signed-off-by: Joanne Koong <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Alexei Starovoitov <[email protected]>
2023-03-01bpf: Support "sk_buff" and "xdp_buff" as valid kfunc arg typesJoanne Koong1-0/+4
The bpf mirror of the in-kernel sk_buff and xdp_buff data structures are __sk_buff and xdp_md. Currently, when we pass in the program ctx to a kfunc where the program ctx is a skb or xdp buffer, we reject the program if the in-kernel definition is sk_buff/xdp_buff instead of __sk_buff/xdp_md. This change allows "sk_buff <--> __sk_buff" and "xdp_buff <--> xdp_md" to be recognized as valid matches. The user program may pass in their program ctx as a __sk_buff or xdp_md, and the in-kernel definition of the kfunc may define this arg as a sk_buff or xdp_buff. Signed-off-by: Joanne Koong <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Alexei Starovoitov <[email protected]>
2023-02-28bpf, docs: Document BPF insn encoding in term of stored bytesJose E. Marchesi1-22/+24
[Changes from V4: - s/regs:16/regs:8 in figure.] [Changes from V3: - Back to src_reg and dst_reg, since they denote register numbers as opposed to the values stored in these registers.] [Changes from V2: - Use src and dst consistently in the document. - Use a more graphical depiction of the 128-bit instruction. - Remove `Where:' fragment. - Clarify that unused bits are reserved and shall be zeroed.] [Changes from V1: - Use rst literal blocks for figures. - Avoid using | in the basic instruction/pseudo instruction figure. - Rebased to today's bpf-next master branch.] This patch modifies instruction-set.rst so it documents the encoding of BPF instructions in terms of how the bytes are stored (be it in an ELF file or as bytes in a memory buffer to be loaded into the kernel or some other BPF consumer) as opposed to how the instruction looks like once loaded. This is hopefully easier to understand by implementors looking to generate and/or consume bytes conforming BPF instructions. The patch also clarifies that the unused bytes in a pseudo-instruction shall be cleared with zeros. Signed-off-by: Jose E. Marchesi <[email protected]> Acked-by: Yonghong Song <[email protected]> Acked-by: David Vernet <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Alexei Starovoitov <[email protected]>
2023-02-28bpf: Fix bpf_cgroup_from_id() doxygen headerDavid Vernet1-2/+1
In commit 332ea1f697be ("bpf: Add bpf_cgroup_from_id() kfunc"), a new bpf_cgroup_from_id() kfunc was added which allows a BPF program to lookup and acquire a reference to a cgroup from a cgroup id. The commit's doxygen comment seems to have copy-pasted fields, which causes BPF kfunc helper documentation to fail to render: <snip>/helpers.c:2114: warning: Excess function parameter 'cgrp'... <snip>/helpers.c:2114: warning: Excess function parameter 'level'... <snip> <snip>/helpers.c:2114: warning: Excess function parameter 'level'... This patch fixes the doxygen header. Fixes: 332ea1f697be ("bpf: Add bpf_cgroup_from_id() kfunc") Signed-off-by: David Vernet <[email protected]> Acked-by: Yonghong Song <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Alexei Starovoitov <[email protected]>
2023-02-28bpf, mips: Implement R4000 workarounds for JITJiaxun Yang2-3/+4
For R4000 erratas around multiplication and division instructions, as our use of those instructions are always followed by mflo/mfhi instructions, the only issue we need care is "MIPS R4000PC/SC Errata, Processor Revision 2.2 and 3.0" Errata 28: "A double-word or a variable shift may give an incorrect result if executed while an integer multiplication is in progress." We just emit a mfhi $0 to ensure the operation is completed after every multiplication instruction according to workaround suggestion in the document. Signed-off-by: Jiaxun Yang <[email protected]> Signed-off-by: Daniel Borkmann <[email protected]> Reviewed-by: Philippe Mathieu-Daudé <[email protected]> Acked-by: Johan Almbladh <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2023-02-28bpf, mips: Implement DADDI workarounds for JITJiaxun Yang2-1/+4
For DADDI errata we just workaround by disable immediate operation for BPF_ADD / BPF_SUB to avoid generation of DADDIU. All other use cases in JIT won't cause overflow thus they are all safe. Signed-off-by: Jiaxun Yang <[email protected]> Signed-off-by: Daniel Borkmann <[email protected]> Reviewed-by: Philippe Mathieu-Daudé <[email protected]> Acked-by: Johan Almbladh <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2023-02-27libbpf: Fix bpf_xdp_query() in old kernelsYonghong Song1-1/+7
Commit 04d58f1b26a4("libbpf: add API to get XDP/XSK supported features") added feature_flags to struct bpf_xdp_query_opts. If a user uses bpf_xdp_query_opts with feature_flags member, the bpf_xdp_query() will check whether 'netdev' family exists or not in the kernel. If it does not exist, the bpf_xdp_query() will return -ENOENT. But 'netdev' family does not exist in old kernels as it is introduced in the same patch set as Commit 04d58f1b26a4. So old kernel with newer libbpf won't work properly with bpf_xdp_query() api call. To fix this issue, if the return value of libbpf_netlink_resolve_genl_family_id() is -ENOENT, bpf_xdp_query() will just return 0, skipping the rest of xdp feature query. This preserves backward compatibility. Fixes: 04d58f1b26a4 ("libbpf: add API to get XDP/XSK supported features") Signed-off-by: Yonghong Song <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2023-02-27Merge tag 'net-6.3-rc1' of ↵Linus Torvalds72-318/+546
git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net Pull networking fixes from Jakub Kicinski: "Including fixes from wireless and netfilter. The notable fixes here are the EEE fix which restores boot for many embedded platforms (real and QEMU); WiFi warning suppression and the ICE Kconfig cleanup. Current release - regressions: - phy: multiple fixes for EEE rework - wifi: wext: warn about usage only once - wifi: ath11k: allow system suspend to survive ath11k Current release - new code bugs: - mlx5: Fix memory leak in IPsec RoCE creation - ibmvnic: assign XPS map to correct queue index Previous releases - regressions: - netfilter: ip6t_rpfilter: Fix regression with VRF interfaces - netfilter: ctnetlink: make event listener tracking global - nf_tables: allow to fetch set elements when table has an owner - mlx5: - fix skb leak while fifo resync and push - fix possible ptp queue fifo use-after-free Previous releases - always broken: - sched: fix action bind logic - ptp: vclock: use mutex to fix "sleep on atomic" bug if driver also uses a mutex - netfilter: conntrack: fix rmmod double-free race - netfilter: xt_length: use skb len to match in length_mt6, avoid issues with BIG TCP Misc: - ice: remove unnecessary CONFIG_ICE_GNSS - mlx5e: remove hairpin write debugfs files - sched: act_api: move TCA_EXT_WARN_MSG to the correct hierarchy" * tag 'net-6.3-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/netdev/net: (53 commits) tcp: tcp_check_req() can be called from process context net: phy: c45: fix network interface initialization failures on xtensa, arm:cubieboard xen-netback: remove unused variables pending_idx and index net/sched: act_api: move TCA_EXT_WARN_MSG to the correct hierarchy net: dsa: ocelot_ext: remove unnecessary phylink.h include net: mscc: ocelot: fix duplicate driver name error net: dsa: felix: fix internal MDIO controller resource length net: dsa: seville: ignore mscc-miim read errors from Lynx PCS net/sched: act_sample: fix action bind logic net/sched: act_mpls: fix action bind logic net/sched: act_pedit: fix action bind logic wifi: wext: warn about usage only once wifi: mt76: usb: fix use-after-free in mt76u_free_rx_queue qede: avoid uninitialized entries in coal_entry array nfc: fix memory leak of se_io context in nfc_genl_se_io ice: remove unnecessary CONFIG_ICE_GNSS net/sched: cls_api: Move call to tcf_exts_miss_cookie_base_destroy() ibmvnic: Assign XPS map to correct queue index docs: net: fix inaccuracies in msg_zerocopy.rst tools: net: add __pycache__ to gitignore ...
2023-02-27libbpf: Document bpf_{btf,link,map,prog}_get_info_by_fd()Ilya Leoshkevich1-5/+64
Replace the short informal description with the proper doc comments. Suggested-by: Andrii Nakryiko <[email protected]> Signed-off-by: Ilya Leoshkevich <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2023-02-27tcp: tcp_check_req() can be called from process contextEric Dumazet1-2/+5
This is a follow up of commit 0a375c822497 ("tcp: tcp_rtx_synack() can be called from process context"). Frederick Lawler reported another "__this_cpu_add() in preemptible" warning caused by the same reason. In my former patch I took care of tcp_rtx_synack() but forgot that tcp_check_req() also contained some SNMP updates. Note that some parts of tcp_check_req() always run in BH context, I added a comment to clarify this. Fixes: 8336886f786f ("tcp: TCP Fast Open Server - support TFO listeners") Link: https://lore.kernel.org/netdev/[email protected]/T/ Signed-off-by: Eric Dumazet <[email protected]> Reported-by: Frederick Lawler <[email protected]> Tested-by: Frederick Lawler <[email protected]> Link: https://lore.kernel.org/r/[email protected] Signed-off-by: Jakub Kicinski <[email protected]>
2023-02-27libbpf: Fix arm syscall regs spec in bpf_tracing.hPuranjay Mohan1-0/+1
The syscall register definitions for ARM in bpf_tracing.h doesn't define the fifth parameter for the syscalls. Because of this some KPROBES based selftests fail to compile for ARM architecture. Define the fifth parameter that is passed in the R5 register (uregs[4]). Fixes: 3a95c42d65d5 ("libbpf: Define arm syscall regs spec in bpf_tracing.h") Signed-off-by: Puranjay Mohan <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
2023-02-27selftests/bpf: Fix compilation errors: Assign a value to a constantRong Tao1-1/+1
Commit bc292ab00f6c("mm: introduce vma->vm_flags wrapper functions") turns the vm_flags into a const variable. Added bpf_find_vma test in commit f108662b27c9("selftests/bpf: Add tests for bpf_find_vma") to assign values to variables that declare const in find_vma_fail1.c programs, which is an error to the compiler and does not test BPF verifiers. It is better to replace 'const vm_flags_t vm_flags' with 'unsigned long vm_start' for testing. $ make -C tools/testing/selftests/bpf/ -j8 ... progs/find_vma_fail1.c:16:16: error: cannot assign to non-static data member 'vm_flags' with const-qualified type 'const vm_flags_t' (aka 'const unsigned long') vma->vm_flags |= 0x55; ~~~~~~~~~~~~~ ^ ../tools/testing/selftests/bpf/tools/include/vmlinux.h:1898:20: note: non-static data member 'vm_flags' declared const here const vm_flags_t vm_flags; ~~~~~~~~~~~`~~~~~~^~~~~~~~ Signed-off-by: Rong Tao <[email protected]> Signed-off-by: Andrii Nakryiko <[email protected]> Link: https://lore.kernel.org/bpf/[email protected]
Powered by cgit, Gruvboxed by Blaster4385.