| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
The include/asm-generic/atomic-instrumented.h checksum got out
of sync, so regenerate it. (No change to actual code.)
Also make scripts/atomic/gen-atomics.sh executable, to make
it easier to use.
The auto-generated atomic header signatures are now fine:
thule:~/tip> scripts/atomic/check-atomics.sh
thule:~/tip>
Signed-off-by: Ingo Molnar <[email protected]>
Cc: [email protected]
Cc: Peter Zijlstra <[email protected]>
Cc: Linus Torvalds <[email protected]>
Cc: Andrew Morton <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: Paul E. McKenney <[email protected]>
Cc: Will Deacon <[email protected]>
Signed-off-by: Ingo Molnar <[email protected]>
|
|
Conflicts:
include/asm-generic/atomic-instrumented.h
kernel/kprobes.c
Use the upstream atomic-instrumented.h checksum, and pick
the kprobes version of kernel/kprobes.c, which effectively
reverts this upstream workaround:
645f224e7ba2: ("kprobes: Tell lockdep about kprobe nesting")
Since the new code *should* be fine without nesting.
Knock on wood ...
Signed-off-by: Ingo Molnar <[email protected]>
|
|
Only x86 provides try_cmpxchg() outside of the atomic_t interfaces,
provide generic fallbacks to create this interface from the widely
available cmpxchg() function.
Signed-off-by: Peter Zijlstra (Intel) <[email protected]>
Signed-off-by: Masami Hiramatsu <[email protected]>
Signed-off-by: Ingo Molnar <[email protected]>
Acked-by: Will Deacon <[email protected]>
Link: https://lore.kernel.org/r/159870621515.1229682.15506193091065001742.stgit@devnote2
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/paulmck/linux-rcu into locking/core
Pull KCSAN updates for v5.10 from Paul E. McKenney:
- Improve kernel messages.
- Be more permissive with bitops races under KCSAN_ASSUME_PLAIN_WRITES_ATOMIC=y.
- Optimize debugfs stat counters.
- Introduce the instrument_*read_write() annotations, to provide a
finer description of certain ops - using KCSAN's compound instrumentation.
Use them for atomic RNW and bitops, where appropriate.
Doing this might find new races.
(Depends on the compiler having tsan-compound-read-before-write=1 support.)
- Support atomic built-ins, which will help certain architectures, such as s390.
- Misc enhancements and smaller fixes.
Signed-off-by: Ingo Molnar <[email protected]>
|
|
The sha1sum of include/linux/atomic-arch-fallback.h isn't checked by
check-atomics.sh. It's not clear why it's skipped so let's check it too.
Signed-off-by: Paul Bolle <[email protected]>
Signed-off-by: Peter Zijlstra (Intel) <[email protected]>
Reviewed-by: Mark Rutland <[email protected]>
Link: https://lkml.kernel.org/r/[email protected]
|
|
Use instrument_atomic_read_write() for atomic RMW ops.
Cc: Will Deacon <[email protected]>
Cc: Boqun Feng <[email protected]>
Cc: Arnd Bergmann <[email protected]>
Cc: <[email protected]>
Acked-by: Peter Zijlstra (Intel) <[email protected]>
Signed-off-by: Marco Elver <[email protected]>
Signed-off-by: Paul E. McKenney <[email protected]>
|
|
Architectures with instrumented (KASAN/KCSAN) atomic operations
natively provide arch_atomic_ variants that are not instrumented.
It turns out that some generic code also requires arch_atomic_ in
order to avoid instrumentation, so provide the arch_atomic_ interface
as a direct map into the regular atomic_ interface for
non-instrumented architectures.
Signed-off-by: Peter Zijlstra (Intel) <[email protected]>
Signed-off-by: Paul E. McKenney <[email protected]>
|
|
Merge the state of the locking kcsan branch before the read/write_once()
and the atomics modifications got merged.
Squash the fallout of the rebase on top of the read/write once and atomic
fallback work into the merge. The history of the original branch is
preserved in tag locking-kcsan-2020-06-02.
Signed-off-by: Thomas Gleixner <[email protected]>
|
|
Currently instrumentation of atomic primitives is done at the architecture
level, while composites or fallbacks are provided at the generic level.
The result is that there are no uninstrumented variants of the
fallbacks. Since there is now need of such variants to isolate text poke
from any form of instrumentation invert this ordering.
Doing this means moving the instrumentation into the generic code as
well as having (for now) two variants of the fallbacks.
Notes:
- the various *cond_read* primitives are not proper fallbacks
and got moved into linux/atomic.c. No arch_ variants are
generated because the base primitives smp_cond_load*()
are instrumented.
- once all architectures are moved over to arch_atomic_ one of the
fallback variants can be removed and some 2300 lines reclaimed.
- atomic_{read,set}*() are no longer double-instrumented
Reported-by: Thomas Gleixner <[email protected]>
Signed-off-by: Peter Zijlstra (Intel) <[email protected]>
Signed-off-by: Thomas Gleixner <[email protected]>
Acked-by: Mark Rutland <[email protected]>
Link: https://lkml.kernel.org/r/[email protected]
|
|
Use __always_inline for atomic fallback wrappers. When building for size
(CC_OPTIMIZE_FOR_SIZE), some compilers appear to be less inclined to
inline even relatively small static inline functions that are assumed to
be inlinable such as atomic ops. This can cause problems, for example in
UACCESS regions.
While the fallback wrappers aren't pure wrappers, they are trivial
nonetheless, and the function they wrap should determine the final
inlining policy.
For x86 tinyconfig we observe:
- vmlinux baseline: 1315988
- vmlinux with patch: 1315928 (-60 bytes)
[ tglx: Cherry-picked from KCSAN ]
Suggested-by: Mark Rutland <[email protected]>
Signed-off-by: Marco Elver <[email protected]>
Acked-by: Mark Rutland <[email protected]>
Signed-off-by: Paul E. McKenney <[email protected]>
Signed-off-by: Thomas Gleixner <[email protected]>
|
|
This switches atomic-instrumented.h to use the generic instrumentation
wrappers provided by instrumented.h.
No functional change intended.
Suggested-by: Arnd Bergmann <[email protected]>
Signed-off-by: Marco Elver <[email protected]>
Acked-by: Arnd Bergmann <[email protected]>
Signed-off-by: Paul E. McKenney <[email protected]>
Signed-off-by: Ingo Molnar <[email protected]>
|
|
Use __always_inline for atomic fallback wrappers. When building for size
(CC_OPTIMIZE_FOR_SIZE), some compilers appear to be less inclined to
inline even relatively small static inline functions that are assumed to
be inlinable such as atomic ops. This can cause problems, for example in
UACCESS regions.
While the fallback wrappers aren't pure wrappers, they are trivial
nonetheless, and the function they wrap should determine the final
inlining policy.
For x86 tinyconfig we observe:
- vmlinux baseline: 1315988
- vmlinux with patch: 1315928 (-60 bytes)
Suggested-by: Mark Rutland <[email protected]>
Signed-off-by: Marco Elver <[email protected]>
Acked-by: Mark Rutland <[email protected]>
Signed-off-by: Paul E. McKenney <[email protected]>
|
|
Prefer __always_inline for atomic wrappers. When building for size
(CC_OPTIMIZE_FOR_SIZE), some compilers appear to be less inclined to
inline even relatively small static inline functions that are assumed to
be inlinable such as atomic ops. This can cause problems, for example in
UACCESS regions.
By using __always_inline, we let the real implementation and not the
wrapper determine the final inlining preference.
For x86 tinyconfig we observe:
- vmlinux baseline: 1316204
- vmlinux with patch: 1315988 (-216 bytes)
This came up when addressing UACCESS warnings with CC_OPTIMIZE_FOR_SIZE
in the KCSAN runtime:
http://lkml.kernel.org/r/[email protected]
Reported-by: Randy Dunlap <[email protected]>
Signed-off-by: Marco Elver <[email protected]>
Acked-by: Mark Rutland <[email protected]>
Signed-off-by: Paul E. McKenney <[email protected]>
|
|
This adds KCSAN instrumentation to atomic-instrumented.h.
Signed-off-by: Marco Elver <[email protected]>
Reviewed-by: Mark Rutland <[email protected]>
Acked-by: Paul E. McKenney <[email protected]>
Signed-off-by: Paul E. McKenney <[email protected]>
|
|
POSIX says the -n option must be a positive decimal integer. Not all
implementations of head(1) support negative numbers meaning offset from
the end of the file.
Instead, the sed expression '$d' has the same effect of removing the
last line of the file.
Signed-off-by: Michael Forney <[email protected]>
Signed-off-by: Peter Zijlstra (Intel) <[email protected]>
Acked-by: Will Deacon <[email protected]>
Cc: Boqun Feng <[email protected]>
Cc: Linus Torvalds <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Link: https://lkml.kernel.org/r/[email protected]
Signed-off-by: Ingo Molnar <[email protected]>
|
|
patch(1) doesn't set the x bit on files. So if someone downloads and
applies patch-4.21.xz, their kernel won't build. Fix that by executing
/bin/sh.
Signed-off-by: Andrew Morton <[email protected]>
Acked-by: Mark Rutland <[email protected]>
Cc: Boqun Feng <[email protected]>
Cc: Linus Torvalds <[email protected]>
Cc: Peter Zijlstra (Intel) <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: Will Deacon <[email protected]>
Signed-off-by: Ingo Molnar <[email protected]>
|
|
We currently check the atomic headers at build-time to ensure they
haven't been modified directly, and these checks require regenerating
the headers in full. As this takes a few seconds, even when
parallelized, this is too slow to run for every kernel build.
Instead, we can generate a hash of each header as we generate them,
which we can cheaply check at build time (~0.16s for all headers).
This patch does so, updating headers with their hashes using the new
gen-atomics.sh script. As some users apparently build the kernel wihout
coreutils, lacking sha1sum, the checks are skipped in this case.
Presumably, most developers have a working coreutils installation.
Signed-off-by: Mark Rutland <[email protected]>
Acked-by: Will Deacon <[email protected]>
Cc: Andrew Morton <[email protected]>
Cc: Boqun Feng <[email protected]>
Cc: Linus Torvalds <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Signed-off-by: Ingo Molnar <[email protected]>
|
|
Some distibutions and build systems doesn't include 'fold' from
coreutils default.
.../scripts/atomic/atomic-tbl.sh: line 183: fold: command not found
Rework to use 'grep' instead of 'fold' to use a dependency that is
already used a lot in the kernel.
[Mark: rework commit message]
Suggested-by: Will Deacon <[email protected]>
Reported-by: Naresh Kamboju <[email protected]>
Signed-off-by: Anders Roxell <[email protected]>
Signed-off-by: Mark Rutland <[email protected]>
Acked-by: Will Deacon <[email protected]>
Cc: Linus Torvalds <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Thomas Gleixner <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Signed-off-by: Ingo Molnar <[email protected]>
|
|
Mark all these scripts executable.
Cc: Mark Rutland <[email protected]>
Cc: Peter Zijlstra (Intel) <[email protected]>
Cc: Will Deacon <[email protected]>
Cc: [email protected]
Cc: Catalin Marinas <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Signed-off-by: Ingo Molnar <[email protected]>
|
|
Now that all the generated atomic headers are in place, it would be good
to ensure that:
a) the headers are up-to-date when scripting changes.
b) developers don't directly modify the generated headers.
To ensure both of these properties, let's add a Kbuild step to check
that the generated headers are up-to-date.
Signed-off-by: Mark Rutland <[email protected]>
Signed-off-by: Peter Zijlstra (Intel) <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: Will Deacon <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: Boqun Feng <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Ingo Molnar <[email protected]>
|
|
To minimize repetition, to allow for future rework, and to ensure
regularity of the various atomic APIs, we'd like to automatically
generate (the bulk of) a number of headers related to atomics.
This patch adds the infrastructure to do so, leaving actual conversion
of headers to subsequent patches. This infrastructure consists of:
* atomics.tbl - a table describing the functions in the atomics API,
with names, prototypes, and metadata describing the variants that
exist (e.g fetch/return, acquire/release/relaxed). Note that the
return type is dependent on the particular variant.
* atomic-tbl.sh - a library of routines useful for dealing with
atomics.tbl (e.g. querying which variants exist, or generating
argument/parameter lists for a given function variant).
* gen-atomic-fallback.sh - a script which generates a header of
fallbacks, covering cases where architecture omit certain functions
(e.g. omitting relaxed variants).
* gen-atomic-long.sh - a script which generates wrappers providing the
atomic_long API atomic of the relevant atomic or atomic64 API,
ensuring the APIs are consistent.
* gen-atomic-instrumented.sh - a script which generates atomic* wrappers
atop of arch_atomic* functions, with automatically generated KASAN
instrumentation.
* fallbacks/* - a set of fallback implementations for atomics, which
should be used when no implementation of a given atomic is provided.
These are used by gen-atomic-fallback.sh to generate fallbacks, and
these are also used by other scripts to determine the set of optional
atomics (as required to generate preprocessor guards correctly).
Fallbacks may use the following variables:
${atomic} atomic prefix: atomic/atomic64/atomic_long, which can be
used to derive the atomic type, and to prefix functions
${int} integer type: int/s64/long
${pfx} variant prefix, e.g. fetch_
${name} base function name, e.g. add
${sfx} variant suffix, e.g. _return
${order} order suffix, e.g. _relaxed
${atomicname} full name, e.g. atomic64_fetch_add_relaxed
${ret} return type of the function, e.g. void
${retstmt} a return statement (with a trailing space), unless the
variant returns void
${params} parameter list for the function declaration, e.g.
"int i, atomic_t *v"
${args} argument list for invoking the function, e.g. "i, v"
... for clarity, ${ret}, ${retstmt}, ${params}, and ${args} are
open-coded for fallbacks where these do not vary, or are critical to
understanding the logic of the fallback.
The MAINTAINERS entry for the atomic infrastructure is updated to cover
the new scripts.
There should be no functional change as a result of this patch.
Signed-off-by: Mark Rutland <[email protected]>
Signed-off-by: Peter Zijlstra (Intel) <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: Will Deacon <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: Boqun Feng <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Ingo Molnar <[email protected]>
|