Age | Commit message (Collapse) | Author | Files | Lines |
|
Vitezslav reported a case where the
"Timeout during microcode update!"
panic would hit. After a deeper look, it turned out that his .config had
CONFIG_HOTPLUG_CPU disabled which practically made save_mc_for_early() a
no-op.
When that happened, the discovered microcode patch wasn't saved into the
cache and the late loading path wouldn't find any.
This, then, lead to early exit from __reload_late() and thus CPUs waiting
until the timeout is reached, leading to the panic.
In hindsight, that function should have been written so it does not return
before the post-synchronization. Oh well, I know better now...
Fixes: bb8c13d61a62 ("x86/microcode: Fix CPU synchronization routine")
Reported-by: Vitezslav Samel <[email protected]>
Signed-off-by: Borislav Petkov <[email protected]>
Signed-off-by: Thomas Gleixner <[email protected]>
Tested-by: Vitezslav Samel <[email protected]>
Tested-by: Ashok Raj <[email protected]>
Cc: [email protected]
Link: http://lkml.kernel.org/r/[email protected]
Link: https://lkml.kernel.org/r/[email protected]
|
|
save_mc_for_early() was a no-op on !CONFIG_HOTPLUG_CPU but the
generic_load_microcode() path saves the microcode patches it has found into
the cache of patches which is used for late loading too. Regardless of
whether CPU hotplug is used or not.
Make the saving unconditional so that late loading can find the proper
patch.
Reported-by: Vitezslav Samel <[email protected]>
Signed-off-by: Borislav Petkov <[email protected]>
Signed-off-by: Thomas Gleixner <[email protected]>
Tested-by: Vitezslav Samel <[email protected]>
Tested-by: Ashok Raj <[email protected]>
Cc: [email protected]
Link: http://lkml.kernel.org/r/[email protected]
Link: https://lkml.kernel.org/r/[email protected]
|
|
The current code extracts the physical address for UE errors and then
hooks it up into memory failure infrastructure. On successful
extraction of physical address it wrongly sets "handled = 1" which
means this UE error has been recovered. Since MCE handler gets return
value as handled = 1, it assumes that error has been recovered and
goes back to same NIP. This causes MCE interrupt again and again in a
loop leading to hard lockup.
Also, initialize phys_addr to ULONG_MAX so that we don't end up
queuing undesired page to hwpoison.
Without this patch we see:
Severe Machine check interrupt [Recovered]
NIP: [000000001002588c] PID: 7109 Comm: find
Initiator: CPU
Error type: UE [Load/Store]
Effective address: 00007fffd2755940
Physical address: 000020181a080000
...
Severe Machine check interrupt [Recovered]
NIP: [000000001002588c] PID: 7109 Comm: find
Initiator: CPU
Error type: UE [Load/Store]
Effective address: 00007fffd2755940
Physical address: 000020181a080000
Severe Machine check interrupt [Recovered]
NIP: [000000001002588c] PID: 7109 Comm: find
Initiator: CPU
Error type: UE [Load/Store]
Effective address: 00007fffd2755940
Physical address: 000020181a080000
Memory failure: 0x20181a08: recovery action for dirty LRU page: Recovered
Memory failure: 0x20181a08: already hardware poisoned
Memory failure: 0x20181a08: already hardware poisoned
Memory failure: 0x20181a08: already hardware poisoned
Memory failure: 0x20181a08: already hardware poisoned
Memory failure: 0x20181a08: already hardware poisoned
Memory failure: 0x20181a08: already hardware poisoned
...
Watchdog CPU:38 Hard LOCKUP
After this patch we see:
Severe Machine check interrupt [Not recovered]
NIP: [00007fffaae585f4] PID: 7168 Comm: find
Initiator: CPU
Error type: UE [Load/Store]
Effective address: 00007fffaafe28ac
Physical address: 00002017c0bd0000
find[7168]: unhandled signal 7 at 00007fffaae585f4 nip 00007fffaae585f4 lr 00007fffaae585e0 code 4
Memory failure: 0x2017c0bd: recovery action for dirty LRU page: Recovered
Fixes: 01eaac2b0591 ("powerpc/mce: Hookup ierror (instruction) UE errors")
Fixes: ba41e1e1ccb9 ("powerpc/mce: Hookup derror (load/store) UE errors")
Cc: [email protected] # v4.15+
Signed-off-by: Mahesh Salgaonkar <[email protected]>
Signed-off-by: Balbir Singh <[email protected]>
Reviewed-by: Balbir Singh <[email protected]>
Signed-off-by: Michael Ellerman <[email protected]>
|
|
address range
The NPU has a limited number of address translation shootdown (ATSD)
registers and the GPU has limited bandwidth to process ATSDs. This can
result in contention of ATSD registers leading to soft lockups on some
threads, particularly when invalidating a large address range in
pnv_npu2_mn_invalidate_range().
At some threshold it becomes more efficient to flush the entire GPU
TLB for the given MM context (PID) than individually flushing each
address in the range. This patch will result in ranges greater than
2MB being converted from 32+ ATSDs into a single ATSD which will flush
the TLB for the given PID on each GPU.
Fixes: 1ab66d1fbada ("powerpc/powernv: Introduce address translation services for Nvlink2")
Cc: [email protected] # v4.12+
Signed-off-by: Alistair Popple <[email protected]>
Acked-by: Balbir Singh <[email protected]>
Tested-by: Balbir Singh <[email protected]>
Signed-off-by: Michael Ellerman <[email protected]>
|
|
parameters
There is a single npu context per set of callback parameters. Callers
should be prevented from overwriting existing callback values so
instead return an error if different parameters are passed.
Fixes: 1ab66d1fbada ("powerpc/powernv: Introduce address translation services for Nvlink2")
Cc: [email protected] # v4.12+
Signed-off-by: Alistair Popple <[email protected]>
Reviewed-by: Mark Hairgrove <[email protected]>
Tested-by: Mark Hairgrove <[email protected]>
Reviewed-by: Balbir Singh <[email protected]>
Signed-off-by: Michael Ellerman <[email protected]>
|
|
The pnv_npu2_init_context() and pnv_npu2_destroy_context() functions
are used to allocate/free contexts to allow address translation and
shootdown by the NPU on a particular GPU. Context initialisation is
implicitly safe as it is protected by the requirement mmap_sem be held
in write mode, however pnv_npu2_destroy_context() does not require
mmap_sem to be held and it is not safe to call with a concurrent
initialisation for a different GPU.
It was assumed the driver would ensure destruction was not called
concurrently with initialisation. However the driver may be simplified
by allowing concurrent initialisation and destruction for different
GPUs. As npu context creation/destruction is not a performance
critical path and the critical section is not large a single spinlock
is used for simplicity.
Fixes: 1ab66d1fbada ("powerpc/powernv: Introduce address translation services for Nvlink2")
Cc: [email protected] # v4.12+
Signed-off-by: Alistair Popple <[email protected]>
Reviewed-by: Mark Hairgrove <[email protected]>
Tested-by: Mark Hairgrove <[email protected]>
Reviewed-by: Balbir Singh <[email protected]>
Signed-off-by: Michael Ellerman <[email protected]>
|
|
Don't do this via custom code, instead now that we have support in the
arch hotplug/hotunplug code, rely on those routines to do the right
thing.
The existing flush doesn't work because it uses ppc64_caches.l1d.size
instead of ppc64_caches.l1d.line_size.
Fixes: 9d5171a8f248 ("powerpc/powernv: Enable removal of memory for in memory tracing")
Signed-off-by: Balbir Singh <[email protected]>
Reviewed-by: Rashmica Gupta <[email protected]>
Signed-off-by: Michael Ellerman <[email protected]>
|
|
This patch adds support for flushing potentially dirty cache lines
when memory is hot-plugged/hot-un-plugged. The support is currently
limited to 64 bit systems.
The bug was exposed when mappings for a device were actually
hot-unplugged and plugged in back later. A similar issue was observed
during the development of memtrace, but memtrace does it's own
flushing of region via a custom routine.
These patches do a flush both on hotplug/unplug to clear any stale
data in the cache w.r.t mappings, there is a small race window where a
clean cache line may be created again just prior to tearing down the
mapping.
The patches were tested by disabling the flush routines in memtrace
and doing I/O on the trace file. The system immediately
checkstops (quite reliablly if prior to the hot-unplug of the memtrace
region, we memset the regions we are about to hot unplug). After these
patches no custom flushing is needed in the memtrace code.
Fixes: 9d5171a8f248 ("powerpc/powernv: Enable removal of memory for in memory tracing")
Cc: [email protected] # v4.14+
Signed-off-by: Balbir Singh <[email protected]>
Acked-by: Reza Arbab <[email protected]>
Reviewed-by: Rashmica Gupta <[email protected]>
Signed-off-by: Michael Ellerman <[email protected]>
|
|
We're missing a sentinel entry in kpti_safe_list. Thus is_midr_in_range_list()
can walk past the end of kpti_safe_list. Depending on the contents of memory,
this could erroneously match a CPU's MIDR, cause a data abort, or other bad
outcomes.
Add the sentinel entry to avoid this.
Fixes: be5b299830c63ed7 ("arm64: capabilities: Add support for checks based on a list of MIDRs")
Signed-off-by: Mark Rutland <[email protected]>
Reported-by: Jan Kiszka <[email protected]>
Tested-by: Jan Kiszka <[email protected]>
Reviewed-by: Suzuki K Poulose <[email protected]>
Cc: Catalin Marinas <[email protected]>
Cc: Suzuki K Poulose <[email protected]>
Cc: Will Deacon <[email protected]>
Signed-off-by: Will Deacon <[email protected]>
|
|
GPL2.0 is not a valid SPDX identiier. Replace it with GPL-2.0.
Fixes: 4a362601baa6 ("x86/jailhouse: Add infrastructure for running in non-root cell")
Signed-off-by: Thomas Gleixner <[email protected]>
Acked-by: Jan Kiszka <[email protected]>
Cc: Kate Stewart <[email protected]>
Cc: Jonathan Corbet <[email protected]>
Cc: Greg Kroah-Hartman <[email protected]>
Cc: Philippe Ombredanne <[email protected]>
Link: https://lkml.kernel.org/r/[email protected]
|
|
The main linker script vmlinux.lds.S for the kernel image merges
the expoline code patch tables into two section ".nospec_call_table"
and ".nospec_return_table". This is *not* done for the modules,
there the sections retain their original names as generated by gcc:
".s390_indirect_call", ".s390_return_mem" and ".s390_return_reg".
The module_finalize code has to check for the compiler generated
section names, otherwise no code patching is done. This slows down
the module code in case of "spectre_v2=off".
Cc: [email protected] # 4.16
Fixes: f19fbd5ed6 ("s390: introduce execute-trampolines for branches")
Signed-off-by: Martin Schwidefsky <[email protected]>
|
|
In a multi-threaded program any thread can call execve(). If this
is not done by the thread group leader, the de_thread() function
replaces the pid of the task that calls execve() with the pid of
thread group leader. If the task reaches user space again without
going over __switch_to() the sampling tag is still set to the old
pid.
Define the arch_setup_new_exec function to verify the task pid
and udpate the tag with LPP if it has changed.
Signed-off-by: Martin Schwidefsky <[email protected]>
|
|
Change the IBM z13/z14 counter names to be in sync with all other models.
Cc: [email protected] # v4.12+
Fixes: 3593eb944c ("s390/cpum_cf: add hardware counter support for IBM z14")
Fixes: 3fc7acebae ("s390/cpum_cf: add IBM z13 counter event names")
Signed-off-by: André Wild <[email protected]>
Signed-off-by: Hendrik Brueckner <[email protected]>
Signed-off-by: Martin Schwidefsky <[email protected]>
|
|
Implement s390 specific arch_uretprobe_is_alive() to avoid SIGSEGVs
observed with uretprobes in combination with setjmp/longjmp.
See commit 2dea1d9c38e4 ("powerpc/uprobes: Implement
arch_uretprobe_is_alive()") for more details.
With this implemented all test cases referenced in the above commit
pass.
Reported-by: Ziqian SUN <[email protected]>
Cc: <[email protected]> # v4.3+
Signed-off-by: Heiko Carstens <[email protected]>
Signed-off-by: Martin Schwidefsky <[email protected]>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull x86 fixes from Thomas Gleixner:
"A small set of fixes for x86:
- Prevent X2APIC ID 0xFFFFFFFF from being treated as valid, which
causes the possible CPU count to be wrong.
- Prevent 32bit truncation in calc_hpet_ref() which causes the TSC
calibration to fail
- Fix the page table setup for temporary text mappings in the resume
code which causes resume failures
- Make the page table dump code handle HIGHPTE correctly instead of
oopsing
- Support for topologies where NUMA nodes share an LLC to prevent a
invalid topology warning and further malfunction on such systems.
- Remove the now unused pci-nommu code
- Remove stale function declarations"
* 'x86-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
x86/power/64: Fix page-table setup for temporary text mapping
x86/mm: Prevent kernel Oops in PTDUMP code with HIGHPTE=y
x86,sched: Allow topologies where NUMA nodes share an LLC
x86/processor: Remove two unused function declarations
x86/acpi: Prevent X2APIC id 0xffffffff from being accounted
x86/tsc: Prevent 32bit truncation in calc_hpet_ref()
x86: Remove pci-nommu.c
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull perf fixes from Thomas Gleixner:
"A larger set of updates for perf.
Kernel:
- Handle the SBOX uncore monitoring correctly on Broadwell CPUs which
do not have SBOX.
- Store context switch out type in PERF_RECORD_SWITCH[_CPU_WIDE]. The
percentage of preempting and non-preempting context switches help
understanding the nature of workloads (CPU or IO bound) that are
running on a machine. This adds the kernel facility and userspace
changes needed to show this information in 'perf script' and 'perf
report -D' (Alexey Budankov)
- Remove a WARN_ON() in the trace/kprobes code which is pointless
because the return error code is already telling the caller what's
wrong.
- Revert a fugly workaround for clang BPF targets.
- Fix sample_max_stack maximum check and do not proceed when an error
has been detect, return them to avoid misidentifying errors (Jiri
Olsa)
- Add SPDX idenitifiers and get rid of GPL boilderplate.
Tools:
- Synchronize kernel ABI headers, v4.17-rc1 (Ingo Molnar)
- Support MAP_FIXED_NOREPLACE, noticed when updating the
tools/include/ copies (Arnaldo Carvalho de Melo)
- Add '\n' at the end of parse-options error messages (Ravi Bangoria)
- Add s390 support for detailed/verbose PMU event description (Thomas
Richter)
- perf annotate fixes and improvements:
* Allow showing offsets in more than just jump targets, use the
new 'O' hotkey in the TUI, config ~/.perfconfig
annotate.offset_level for it and for --stdio2 (Arnaldo Carvalho
de Melo)
* Use the resolved variable names from objdump disassembled lines
to make them more compact, just like was already done for some
instructions, like "mov", this eventually will be done more
generally, but lets now add some more to the existing mechanism
(Arnaldo Carvalho de Melo)
- perf record fixes:
* Change warning for missing topology sysfs entry to debug, as not
all architectures have those files, s390 being one of those
(Thomas Richter)
* Remove old error messages about things that unlikely to be the
root cause in modern systems (Andi Kleen)
- perf sched fixes:
* Fix -g/--call-graph documentation (Takuya Yamamoto)
- perf stat:
* Enable 1ms interval for printing event counters values in
(Alexey Budankov)
- perf test fixes:
* Run dwarf unwind on arm32 (Kim Phillips)
* Remove unused ptrace.h include from LLVM test, sidesteping older
clang's lack of support for some asm constructs (Arnaldo
Carvalho de Melo)
* Fixup BPF test using epoll_pwait syscall function probe, to cope
with the syscall routines renames performed in this development
cycle (Arnaldo Carvalho de Melo)
- perf version fixes:
* Do not print info about HAVE_LIBAUDIT_SUPPORT in 'perf version
--build-options' when HAVE_SYSCALL_TABLE_SUPPORT is true, as
libaudit won't be used in that case, print info about
syscall_table support instead (Jin Yao)
- Build system fixes:
* Use HAVE_..._SUPPORT used consistently (Jin Yao)
* Restore READ_ONCE() C++ compatibility in tools/include (Mark
Rutland)
* Give hints about package names needed to build jvmti (Arnaldo
Carvalho de Melo)"
* 'perf-urgent-for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip: (37 commits)
perf/x86/intel/uncore: Fix SBOX support for Broadwell CPUs
perf/x86/intel/uncore: Revert "Remove SBOX support for Broadwell server"
coresight: Move to SPDX identifier
perf test BPF: Fixup BPF test using epoll_pwait syscall function probe
perf tests mmap: Show which tracepoint is failing
perf tools: Add '\n' at the end of parse-options error messages
perf record: Remove suggestion to enable APIC
perf record: Remove misleading error suggestion
perf hists browser: Clarify top/report browser help
perf mem: Allow all record/report options
perf trace: Support MAP_FIXED_NOREPLACE
perf: Remove superfluous allocation error check
perf: Fix sample_max_stack maximum check
perf: Return proper values for user stack errors
perf list: Add s390 support for detailed/verbose PMU event description
perf script: Extend misc field decoding with switch out event type
perf report: Extend raw dump (-D) out with switch out event type
perf/core: Store context switch out type in PERF_RECORD_SWITCH[_CPU_WIDE]
tools/headers: Synchronize kernel ABI headers, v4.17-rc1
trace_kprobe: Remove warning message "Could not insert probe at..."
...
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux
Pull arm64 fixes from Catalin Marinas:
- kasan: avoid pfn_to_nid() before the page array is initialised
- Fix typo causing the "upgrade" of known signals to SIGKILL
* tag 'arm64-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/arm64/linux:
arm64: signal: don't force known signals to SIGKILL
arm64: kasan: avoid pfn_to_nid() before page array is initialized
|
|
Chun-Yi reported a kernel warning message below:
WARNING: CPU: 0 PID: 0 at ../mm/early_ioremap.c:182 early_iounmap+0x4f/0x12c()
early_iounmap(ffffffffff200180, 00000118) [0] size not consistent 00000120
The problem is x86 kexec_file_load adds extra alignment to the efi
memmap: in bzImage64_load():
efi_map_sz = efi_get_runtime_map_size();
efi_map_sz = ALIGN(efi_map_sz, 16);
And __efi_memmap_init maps with the size including the alignment bytes
but efi_memmap_unmap use nr_maps * desc_size which does not include the
extra bytes.
The alignment in kexec code is only needed for the kexec buffer internal
use Actually kexec should pass exact size of the efi memmap to 2nd
kernel.
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Dave Young <[email protected]>
Reported-by: joeyli <[email protected]>
Tested-by: Randy Wright <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>
|
|
Commit 95846ecf9dac ("pid: replace pid bitmap implementation with IDR
API") changed last field of /proc/loadavg (last pid allocated) to be off
by one:
# unshare -p -f --mount-proc cat /proc/loadavg
0.00 0.00 0.00 1/60 2 <===
It should be 1 after first fork into pid namespace.
This is formally a regression but given how useless this field is I
don't think anyone is affected.
Bug was found by /proc testsuite!
Link: http://lkml.kernel.org/r/20180413175408.GA27246@avx2
Fixes: 95846ecf9dac508 ("pid: replace pid bitmap implementation with IDR API")
Signed-off-by: Alexey Dobriyan <[email protected]>
Cc: "Eric W. Biederman" <[email protected]>
Cc: Gargi Sharma <[email protected]>
Cc: Oleg Nesterov <[email protected]>
Signed-off-by: Andrew Morton <[email protected]>
Signed-off-by: Linus Torvalds <[email protected]>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs
Pull vfs fixes from Al Viro:
"Assorted fixes.
Some of that is only a matter with fault injection (broken handling of
small allocation failure in various mount-related places), but the
last one is a root-triggerable stack overflow, and combined with
userns it gets really nasty ;-/"
* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/viro/vfs:
Don't leak MNT_INTERNAL away from internal mounts
mm,vmscan: Allow preallocating memory for register_shrinker().
rpc_pipefs: fix double-dput()
orangefs_kill_sb(): deal with allocation failures
jffs2_kill_sb(): deal with failed allocations
hypfs_kill_super(): deal with failed allocations
|
|
Although we've implemented PSCI 0.1, 0.2 and 1.0, we expose either 0.1
or 1.0 to a guest, defaulting to the latest version of the PSCI
implementation that is compatible with the requested version. This is
no different from doing a firmware upgrade on KVM.
But in order to give a chance to hypothetical badly implemented guests
that would have a fit by discovering something other than PSCI 0.2,
let's provide a new API that allows userspace to pick one particular
version of the API.
This is implemented as a new class of "firmware" registers, where
we expose the PSCI version. This allows the PSCI version to be
save/restored as part of a guest migration, and also set to
any supported version if the guest requires it.
Cc: [email protected] #4.16
Reviewed-by: Christoffer Dall <[email protected]>
Signed-off-by: Marc Zyngier <[email protected]>
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/jhogan/mips
Pull MIPS fixes from James Hogan:
- io: Add barriers to read*() & write*()
- dts: Fix boston PCI bus DTC warnings (4.17)
- memset: Several corner case fixes (one 3.10, others longer)
* tag 'mips_fixes_4.17_1' of git://git.kernel.org/pub/scm/linux/kernel/git/jhogan/mips:
MIPS: uaccess: Add micromips clobbers to bzero invocation
MIPS: memset.S: Fix clobber of v1 in last_fixup
MIPS: memset.S: Fix return of __clear_user from Lpartial_fixup
MIPS: memset.S: EVA & fault support for small_memset
MIPS: dts: Boston: Fix PCI bus dtc warnings:
MIPS: io: Add barrier after register read in readX()
MIPS: io: Prevent compiler reordering writeX()
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux
Pull powerpc fixes from Michael Ellerman:
- Fix an off-by-one bug in our alternative asm patching which leads to
incorrectly patched code. This bug lay dormant for nearly 10 years
but we finally hit it due to a recent change.
- Fix lockups when running KVM guests on Power8 due to a missing check
when a thread that's running KVM comes out of idle.
- Fix an out-of-spec behaviour in the XIVE code (P9 interrupt
controller).
- Fix EEH handling of bridge MMIO windows.
- Prevent crashes in our RFI fallback flush handler if firmware didn't
tell us the size of the L1 cache (only seen on simulators).
Thanks to: Benjamin Herrenschmidt, Madhavan Srinivasan, Michael Neuling.
* tag 'powerpc-4.17-3' of git://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux:
powerpc/kvm: Fix lockups when running KVM guests on Power8
powerpc/eeh: Fix enabling bridge MMIO windows
powerpc/xive: Fix trying to "push" an already active pool VP
powerpc/64s: Default l1d_size to 64K in RFI fallback flush
powerpc/lib: Fix off-by-one in alternate feature patching
|
|
git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux
Pull s390 fixes and kexec-file-load from Martin Schwidefsky:
"After the common code kexec patches went in via Andrew we can now push
the architecture parts to implement the kexec-file-load system call.
Plus a few more bug fixes and cleanups, this includes an update to the
default configurations"
* 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/s390/linux:
s390/signal: cleanup uapi struct sigaction
s390: rename default_defconfig to debug_defconfig
s390: remove gcov defconfig
s390: update defconfig
s390: add support for IBM z14 Model ZR1
s390: remove couple of duplicate includes
s390/boot: remove unused COMPILE_VERSION and ccflags-y
s390/nospec: include cpu.h
s390/decompressor: Ignore file vmlinux.bin.full
s390/kexec_file: add generated files to .gitignore
s390/Kconfig: Move kexec config options to "Processor type and features"
s390/kexec_file: Add ELF loader
s390/kexec_file: Add crash support to image loader
s390/kexec_file: Add image loader
s390/kexec_file: Add kexec_file_load system call
s390/kexec_file: Add purgatory
s390/kexec_file: Prepare setup.h for kexec_file_load
s390/smsgiucv: disable SMSG on module unload
s390/sclp: avoid potential usage of uninitialized value
|
|
SBOX on some Broadwell CPUs is broken because it's enabled unconditionally
despite the fact that there are no SBOXes available.
Check the Power Control Unit CAPID4 register to determine the number of
available SBOXes on the particular CPU before trying to enable them. If
there are none, nullify the SBOX descriptor so it isn't tried to be
initialized.
Signed-off-by: Oskar Senft <[email protected]>
Signed-off-by: Thomas Gleixner <[email protected]>
Tested-by: Mark van Dijk <[email protected]>
Reviewed-by: Kan Liang <[email protected]>
Acked-by: Peter Zijlstra <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Link: https://lkml.kernel.org/r/[email protected]
|
|
This reverts commit 3b94a891667c ("perf/x86/intel/uncore: Remove
SBOX support for Broadwell server")
Revert because there exists a proper workaround for Broadwell-EP servers
without SBOX now. Note that BDX-DE does not have a SBOX.
Signed-off-by: Stephane Eranian <[email protected]>
Signed-off-by: Thomas Gleixner <[email protected]>
Reviewed-by: Kan Liang <[email protected]>
Acked-by: Peter Zijlstra <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Link: https://lkml.kernel.org/r/[email protected]
|
|
On a system with 4-level page-tables there is no p4d, so the pud in the pgd
should be mapped. The old code before commit fb43d6cb91ef already did that.
The change from above commit causes an invalid page-table which causes
undefined behavior. In one report it caused triple faults.
Fix it by changing the p4d back to pud.
Fixes: fb43d6cb91ef ('x86/mm: Do not auto-massage page protections')
Reported-by: Borislav Petkov <[email protected]>
Signed-off-by: Joerg Roedel <[email protected]>
Signed-off-by: Thomas Gleixner <[email protected]>
Tested-by: Michal Kubecek <[email protected]>
Tested-by: Borislav Petkov <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: Dave Hansen <[email protected]>
Link: https://lkml.kernel.org/r/[email protected]
|
|
|
|
When running KVM guests on Power8 we can see a lockup where one CPU
stops responding. This often leads to a message such as:
watchdog: CPU 136 detected hard LOCKUP on other CPUs 72
Task dump for CPU 72:
qemu-system-ppc R running task 10560 20917 20908 0x00040004
And then backtraces on other CPUs, such as:
Task dump for CPU 48:
ksmd R running task 10032 1519 2 0x00000804
Call Trace:
...
--- interrupt: 901 at smp_call_function_many+0x3c8/0x460
LR = smp_call_function_many+0x37c/0x460
pmdp_invalidate+0x100/0x1b0
__split_huge_pmd+0x52c/0xdb0
try_to_unmap_one+0x764/0x8b0
rmap_walk_anon+0x15c/0x370
try_to_unmap+0xb4/0x170
split_huge_page_to_list+0x148/0xa30
try_to_merge_one_page+0xc8/0x990
try_to_merge_with_ksm_page+0x74/0xf0
ksm_scan_thread+0x10ec/0x1ac0
kthread+0x160/0x1a0
ret_from_kernel_thread+0x5c/0x78
This is caused by commit 8c1c7fb0b5ec ("powerpc/64s/idle: avoid sync
for KVM state when waking from idle"), which added a check in
pnv_powersave_wakeup() to see if the kvm_hstate.hwthread_state is
already set to KVM_HWTHREAD_IN_KERNEL, and if so to skip the store and
test of kvm_hstate.hwthread_req.
The problem is that the primary does not set KVM_HWTHREAD_IN_KVM when
entering the guest, so it can then come out to cede with
KVM_HWTHREAD_IN_KERNEL set. It can then go idle in kvm_do_nap after
setting hwthread_req to 1, but because hwthread_state is still
KVM_HWTHREAD_IN_KERNEL we will skip the test of hwthread_req when we
wake up from idle and won't go to kvm_start_guest. From there the
thread will return somewhere garbage and crash.
Fix it by skipping the store of hwthread_state, but not the test of
hwthread_req, when coming out of idle. It's OK to skip the sync in
that case because hwthread_req will have been set on the same thread,
so there is no synchronisation required.
Fixes: 8c1c7fb0b5ec ("powerpc/64s/idle: avoid sync for KVM state when waking from idle")
Signed-off-by: Michael Ellerman <[email protected]>
|
|
On boot we save the configuration space of PCIe bridges. We do this so
when we get an EEH event and everything gets reset that we can restore
them.
Unfortunately we save this state before we've enabled the MMIO space
on the bridges. Hence if we have to reset the bridge when we come back
MMIO is not enabled and we end up taking an PE freeze when the driver
starts accessing again.
This patch forces the memory/MMIO and bus mastering on when restoring
bridges on EEH. Ideally we'd do this correctly by saving the
configuration space writes later, but that will have to come later in
a larger EEH rewrite. For now we have this simple fix.
The original bug can be triggered on a boston machine by doing:
echo 0x8000000000000000 > /sys/kernel/debug/powerpc/PCI0001/err_injct_outbound
On boston, this PHB has a PCIe switch on it. Without this patch,
you'll see two EEH events, 1 expected and 1 the failure we are fixing
here. The second EEH event causes the anything under the PHB to
disappear (i.e. the i40e eth).
With this patch, only 1 EEH event occurs and devices properly recover.
Fixes: 652defed4875 ("powerpc/eeh: Check PCIe link after reset")
Cc: [email protected] # v3.11+
Reported-by: Pridhiviraj Paidipeddi <[email protected]>
Signed-off-by: Michael Neuling <[email protected]>
Acked-by: Russell Currey <[email protected]>
Signed-off-by: Michael Ellerman <[email protected]>
|
|
The micromips implementation of bzero additionally clobbers registers t7
& t8. Specify this in the clobbers list when invoking bzero.
Fixes: 26c5e07d1478 ("MIPS: microMIPS: Optimise 'memset' core library function.")
Reported-by: James Hogan <[email protected]>
Signed-off-by: Matt Redfearn <[email protected]>
Cc: Ralf Baechle <[email protected]>
Cc: [email protected]
Cc: <[email protected]> # 3.10+
Patchwork: https://patchwork.linux-mips.org/patch/19110/
Signed-off-by: James Hogan <[email protected]>
|
|
The label .Llast_fixup\@ is jumped to on page fault within the final
byte set loop of memset (on < MIPSR6 architectures). For some reason, in
this fault handler, the v1 register is randomly set to a2 & STORMASK.
This clobbers v1 for the calling function. This can be observed with the
following test code:
static int __init __attribute__((optimize("O0"))) test_clear_user(void)
{
register int t asm("v1");
char *test;
int j, k;
pr_info("\n\n\nTesting clear_user\n");
test = vmalloc(PAGE_SIZE);
for (j = 256; j < 512; j++) {
t = 0xa5a5a5a5;
if ((k = clear_user(test + PAGE_SIZE - 256, j)) != j - 256) {
pr_err("clear_user (%px %d) returned %d\n", test + PAGE_SIZE - 256, j, k);
}
if (t != 0xa5a5a5a5) {
pr_err("v1 was clobbered to 0x%x!\n", t);
}
}
return 0;
}
late_initcall(test_clear_user);
Which demonstrates that v1 is indeed clobbered (MIPS64):
Testing clear_user
v1 was clobbered to 0x1!
v1 was clobbered to 0x2!
v1 was clobbered to 0x3!
v1 was clobbered to 0x4!
v1 was clobbered to 0x5!
v1 was clobbered to 0x6!
v1 was clobbered to 0x7!
Since the number of bytes that could not be set is already contained in
a2, the andi placing a value in v1 is not necessary and actively
harmful in clobbering v1.
Reported-by: James Hogan <[email protected]>
Signed-off-by: Matt Redfearn <[email protected]>
Cc: Ralf Baechle <[email protected]>
Cc: [email protected]
Cc: [email protected]
Patchwork: https://patchwork.linux-mips.org/patch/19109/
Signed-off-by: James Hogan <[email protected]>
|
|
Correct all SATA ahci and phy controller register
addresses and interrupt lines to proper values.
Fixes: 344a2e514182 ("arm64: dts: Add SATA DT nodes for Stingray SoC")
Signed-off-by: Srinath Mannam <[email protected]>
Reviewed-by: Ray Jui <[email protected]>
Reviewed-by: Scott Branden <[email protected]>
Reviewed-by: Andrew Gospodarek <[email protected]>
Signed-off-by: Florian Fainelli <[email protected]>
|
|
The Khadas VIM2 board connects the dwc3 controller to an internal 4-port
USB hub which. Two of these ports are accessible directly soldered to
the board, while the other two are accessible through the 40-pin "GPIO"
header.
Signed-off-by: Martin Blumenstingl <[email protected]>
Signed-off-by: Kevin Hilman <[email protected]>
|
|
The Nexbox A95X provides two USB ports. Enable the SoC's USB controller
on this board to make these USB ports usable.
Signed-off-by: Martin Blumenstingl <[email protected]>
Signed-off-by: Kevin Hilman <[email protected]>
|
|
The LibreTech CC ("Le Potato") board provides four USB connectors.
These are provided by a hub which is connected to the SoC's USB
controller.
Enable the SoC's USB controller to make the USB ports usable. Also turn
on the HDMI_5V regulator when powering on the PHY because (even though
it's not shown in the schematics) HDMI_5V also supplies the USB VBUS.
Signed-off-by: Martin Blumenstingl <[email protected]>
Signed-off-by: Kevin Hilman <[email protected]>
|
|
All S905D (GXL) and S912 (GXM) reference boards (namely these are
P230, P231, Q200 and Q201) provide USB connectors.
This enables the USB controller on these boards to make the USB ports
actually usable.
Signed-off-by: Martin Blumenstingl <[email protected]>
Signed-off-by: Kevin Hilman <[email protected]>
|
|
All boards based on the P212 reference design (the P212 reference board
itself and the Khadas VIM) have USB connectors (in case of the Khadas
VIM the first port is exposed through the USB Type-C connector, the
second port is connected to a 4-port USB hub).
This enables the USB controller on these boards to make the USB ports
actually usable.
Signed-off-by: Martin Blumenstingl <[email protected]>
Signed-off-by: Kevin Hilman <[email protected]>
|
|
The USB configuration on GXM is slightly different than on GXL. The dwc3
controller's internal hub has three USB2 ports (instead of 2 on GXL)
along with a dedicated USB2 PHY for this port. However, it seems that
there are no pins on GXM which would allow connecting the third port to
a physical USB port.
Passing the third PHY is required though, because without it none of the
other USB ports is working (this seems to be a limitation of how the
internal USB hub works, if one PHY is disabled then no USB port works).
Signed-off-by: Martin Blumenstingl <[email protected]>
Signed-off-by: Kevin Hilman <[email protected]>
|
|
This adds USB host support to the Meson GXL SoC. A dwc3 controller is
used for host-mode, while a dwc2 controller (not added in this patch
because I could not get it working) is used for device-mode only.
The dwc3 controller's internal roothub has two USB2 ports enabled but no
USB3 port. Each of the ports is supplied by a separate PHY. The USB pins
are connected to the SoC's USBHOST_A and USBOTG_B pins.
Due to the way the roothub works internally the USB PHYs are left
enabled. When the dwc3 controller is disabled the PHY is never powered on
so it does not draw any extra power. However, when the dwc3 host
controller is enabled then all PHYs also have to be enabled, otherwise
USB devices will not be detected (regardless of whether they are plugged
into an enabled port or not). This means that only the dwc3 controller
has to be enabled on boards with USB support (instead of requiring all
boards to enable the PHYs additionally with the chance of forgetting to
enable one and breaking all other ports with that as well).
This also adds the USB3 PHY which currently only does some basic
initialization. That however is required because without it high-speed
devices (like USB thumb drives) do not work on some devices (probably
because the bootloader does not configure the USB3 PHY registers).
Signed-off-by: Martin Blumenstingl <[email protected]>
Signed-off-by: Kevin Hilman <[email protected]>
|
|
The sleep33xx and sleep43xx files should not depend on a header file
generated in drivers/memory. Remove this dependency and instead allow
both drivers/memory and arch/arm/mach-omap2 to generate all macros
needed in headers local to their own paths.
This fixes an issue where the build fail will when using O= to set a
split object directory and arch/arm/mach-omap2 is built before
drivers/memory with the following error:
.../drivers/memory/emif-asm-offsets.c:1:0: fatal error: can't open
drivers/memory/emif-asm-offsets.s for writing: No such file or directory
compilation terminated.
Fixes: 41d9d44d7258 ("ARM: OMAP2+: pm33xx-core: Add platform code needed for PM")
Reviewed-by: Masahiro Yamada <[email protected]>
Signed-off-by: Dave Gerlach <[email protected]>
Acked-by: Santosh Shilimkar <[email protected]>
Signed-off-by: Tony Lindgren <[email protected]>
|
|
When setting up a CPU, we "push" (activate) a pool VP for it.
However it's an error to do so if it already has an active
pool VP.
This happens when doing soft CPU hotplug on powernv since we
don't tear down the CPU on unplug. The HW flags the error which
gets captured by the diagnostics.
Fix this by making sure to "pull" out any already active pool
first.
Fixes: 243e25112d06 ("powerpc/xive: Native exploitation of the XIVE interrupt controller")
Cc: [email protected] # v4.12+
Signed-off-by: Benjamin Herrenschmidt <[email protected]>
Signed-off-by: Michael Ellerman <[email protected]>
|
|
Since commit:
a7e6f1ca90354a31 ("arm64: signal: Force SIGKILL for unknown signals in force_signal_inject")
... any signal which is not SIGKILL will be upgraded to a SIGKILL be
force_signal_inject(). This includes signals we do expect, such as
SIGILL triggered by do_undefinstr().
Fix the check to use a logical AND rather than a logical OR, permitting
signals whose layout is SIL_FAULT.
Fixes: a7e6f1ca90354a31 ("arm64: signal: Force SIGKILL for unknown signals in force_signal_inject")
Cc: Will Deacon <[email protected]>
Reviewed-by: Dave Martin <[email protected]>
Signed-off-by: Mark Rutland <[email protected]>
Signed-off-by: Catalin Marinas <[email protected]>
|
|
The __clear_user function is defined to return the number of bytes that
could not be cleared. From the underlying memset / bzero implementation
this means setting register a2 to that number on return. Currently if a
page fault is triggered within the memset_partial block, the value
loaded into a2 on return is meaningless.
The label .Lpartial_fixup\@ is jumped to on page fault. In order to work
out how many bytes failed to copy, the exception handler should find how
many bytes left in the partial block (andi a2, STORMASK), add that to
the partial block end address (a2), and subtract the faulting address to
get the remainder. Currently it incorrectly subtracts the partial block
start address (t1), which has additionally been clobbered to generate a
jump target in memset_partial. Fix this by adding the block end address
instead.
This issue was found with the following test code:
int j, k;
for (j = 0; j < 512; j++) {
if ((k = clear_user(NULL, j)) != j) {
pr_err("clear_user (NULL %d) returned %d\n", j, k);
}
}
Which now passes on Creator Ci40 (MIPS32) and Cavium Octeon II (MIPS64).
Suggested-by: James Hogan <[email protected]>
Signed-off-by: Matt Redfearn <[email protected]>
Cc: Ralf Baechle <[email protected]>
Cc: [email protected]
Cc: [email protected]
Patchwork: https://patchwork.linux-mips.org/patch/19108/
Signed-off-by: James Hogan <[email protected]>
|
|
In arm64's kasan_init(), we use pfn_to_nid() to find the NUMA node a
span of memory is in, hoping to allocate shadow from the same NUMA node.
However, at this point, the page array has not been initialized, and
thus this is bogus.
Since commit:
f165b378bbdf6c8a ("mm: uninitialized struct page poisoning sanity")
... accessing fields of the page array results in a boot time Oops(),
highlighting this problem:
[ 0.000000] Unable to handle kernel paging request at virtual address dfff200000000000
[ 0.000000] Mem abort info:
[ 0.000000] ESR = 0x96000004
[ 0.000000] Exception class = DABT (current EL), IL = 32 bits
[ 0.000000] SET = 0, FnV = 0
[ 0.000000] EA = 0, S1PTW = 0
[ 0.000000] Data abort info:
[ 0.000000] ISV = 0, ISS = 0x00000004
[ 0.000000] CM = 0, WnR = 0
[ 0.000000] [dfff200000000000] address between user and kernel address ranges
[ 0.000000] Internal error: Oops: 96000004 [#1] PREEMPT SMP
[ 0.000000] Modules linked in:
[ 0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 4.16.0-07317-gf165b378bbdf #42
[ 0.000000] Hardware name: ARM Juno development board (r1) (DT)
[ 0.000000] pstate: 80000085 (Nzcv daIf -PAN -UAO)
[ 0.000000] pc : __asan_load8+0x8c/0xa8
[ 0.000000] lr : __dump_page+0x3c/0x3b8
[ 0.000000] sp : ffff2000099b7ca0
[ 0.000000] x29: ffff2000099b7ca0 x28: ffff20000a1762c0
[ 0.000000] x27: ffff7e0000000000 x26: ffff2000099dd000
[ 0.000000] x25: ffff200009a3f960 x24: ffff200008f9c38c
[ 0.000000] x23: ffff20000a9d3000 x22: ffff200009735430
[ 0.000000] x21: fffffffffffffffe x20: ffff7e0001e50420
[ 0.000000] x19: ffff7e0001e50400 x18: 0000000000001840
[ 0.000000] x17: ffffffffffff8270 x16: 0000000000001840
[ 0.000000] x15: 0000000000001920 x14: 0000000000000004
[ 0.000000] x13: 0000000000000000 x12: 0000000000000800
[ 0.000000] x11: 1ffff0012d0f89ff x10: ffff10012d0f89ff
[ 0.000000] x9 : 0000000000000000 x8 : ffff8009687c5000
[ 0.000000] x7 : 0000000000000000 x6 : ffff10000f282000
[ 0.000000] x5 : 0000000000000040 x4 : fffffffffffffffe
[ 0.000000] x3 : 0000000000000000 x2 : dfff200000000000
[ 0.000000] x1 : 0000000000000005 x0 : 0000000000000000
[ 0.000000] Process swapper (pid: 0, stack limit = 0x (ptrval))
[ 0.000000] Call trace:
[ 0.000000] __asan_load8+0x8c/0xa8
[ 0.000000] __dump_page+0x3c/0x3b8
[ 0.000000] dump_page+0xc/0x18
[ 0.000000] kasan_init+0x2e8/0x5a8
[ 0.000000] setup_arch+0x294/0x71c
[ 0.000000] start_kernel+0xdc/0x500
[ 0.000000] Code: aa0403e0 9400063c 17ffffee d343fc00 (38e26800)
[ 0.000000] ---[ end trace 67064f0e9c0cc338 ]---
[ 0.000000] Kernel panic - not syncing: Attempted to kill the idle task!
[ 0.000000] ---[ end Kernel panic - not syncing: Attempted to kill the idle task! ]---
Let's fix this by using early_pfn_to_nid(), as other architectures do in
their kasan init code. Note that early_pfn_to_nid acquires the nid from
the memblock array, which we iterate over in kasan_init(), so this
should be fine.
Signed-off-by: Mark Rutland <[email protected]>
Fixes: 39d114ddc6822302 ("arm64: add KASAN support")
Cc: Will Deacon <[email protected]>
Signed-off-by: Catalin Marinas <[email protected]>
|
|
The walk_pte_level() function just uses __va to get the virtual address of
the PTE page, but that breaks when the PTE page is not in the direct
mapping with HIGHPTE=y.
The result is an unhandled kernel paging request at some random address
when accessing the current_kernel or current_user file.
Use the correct API to access PTE pages.
Fixes: fe770bf0310d ('x86: clean up the page table dumper and add 32-bit support')
Signed-off-by: Joerg Roedel <[email protected]>
Signed-off-by: Thomas Gleixner <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Link: https://lkml.kernel.org/r/[email protected]
|
|
Intel's Skylake Server CPUs have a different LLC topology than previous
generations. When in Sub-NUMA-Clustering (SNC) mode, the package is divided
into two "slices", each containing half the cores, half the LLC, and one
memory controller and each slice is enumerated to Linux as a NUMA
node. This is similar to how the cores and LLC were arranged for the
Cluster-On-Die (CoD) feature.
CoD allowed the same cache line to be present in each half of the LLC.
But, with SNC, each line is only ever present in *one* slice. This means
that the portion of the LLC *available* to a CPU depends on the data being
accessed:
Remote socket: entire package LLC is shared
Local socket->local slice: data goes into local slice LLC
Local socket->remote slice: data goes into remote-slice LLC. Slightly
higher latency than local slice LLC.
The biggest implication from this is that a process accessing all
NUMA-local memory only sees half the LLC capacity.
The CPU describes its cache hierarchy with the CPUID instruction. One of
the CPUID leaves enumerates the "logical processors sharing this
cache". This information is used for scheduling decisions so that tasks
move more freely between CPUs sharing the cache.
But, the CPUID for the SNC configuration discussed above enumerates the LLC
as being shared by the entire package. This is not 100% precise because the
entire cache is not usable by all accesses. But, it *is* the way the
hardware enumerates itself, and this is not likely to change.
The userspace visible impact of all the above is that the sysfs info
reports the entire LLC as being available to the entire package. As noted
above, this is not true for local socket accesses. This patch does not
correct the sysfs info. It is the same, pre and post patch.
The current code emits the following warning:
sched: CPU #3's llc-sibling CPU #0 is not on the same node! [node: 1 != 0]. Ignoring dependency.
The warning is coming from the topology_sane() check in smpboot.c because
the topology is not matching the expectations of the model for obvious
reasons.
To fix this, add a vendor and model specific check to never call
topology_sane() for these systems. Also, just like "Cluster-on-Die" disable
the "coregroup" sched_domain_topology_level and use NUMA information from
the SRAT alone.
This is OK at least on the hardware we are immediately concerned about
because the LLC sharing happens at both the slice and at the package level,
which are also NUMA boundaries.
Signed-off-by: Alison Schofield <[email protected]>
Signed-off-by: Thomas Gleixner <[email protected]>
Reviewed-by: Borislav Petkov <[email protected]>
Cc: Prarit Bhargava <[email protected]>
Cc: Tony Luck <[email protected]>
Cc: Peter Zijlstra (Intel) <[email protected]>
Cc: [email protected]
Cc: Dave Hansen <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: David Rientjes <[email protected]>
Cc: Igor Mammedov <[email protected]>
Cc: "H. Peter Anvin" <[email protected]>
Cc: Tim Chen <[email protected]>
Link: https://lkml.kernel.org/r/[email protected]
|
|
While generating a message about guests probing for SVE/LORegions
is a useful debugging tool, considering it an error is slightly
over the top, as this is the only way the guest can find out
about the presence of the feature.
Let's turn these message into kvm_debug so that they can only
be seen if CONFIG_DYNAMIC_DEBUG, and kept quiet otherwise.
Acked-by: Christoffer Dall <[email protected]>
Signed-off-by: Marc Zyngier <[email protected]>
|
|
early_trap_init() and cpu_set_gdt() have been removed, so remove the stale
declarations as well.
Signed-off-by: Dou Liyang <[email protected]>
Signed-off-by: Thomas Gleixner <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Link: https://lkml.kernel.org/r/[email protected]
|
|
RongQing reported that there are some X2APIC id 0xffffffff in his machine's
ACPI MADT table, which makes the number of possible CPU inaccurate.
The reason is that the ACPI X2APIC parser has no sanity check for APIC ID
0xffffffff, which is an invalid id in all APIC types. See "Intel® 64
Architecture x2APIC Specification", Chapter 2.4.1.
Add a sanity check to acpi_parse_x2apic() which ignores the invalid id.
Reported-by: Li RongQing <[email protected]>
Signed-off-by: Dou Liyang <[email protected]>
Signed-off-by: Thomas Gleixner <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Link: https://lkml.kernel.org/r/[email protected]
|