aboutsummaryrefslogtreecommitdiff
path: root/arch/x86/kernel/fpu
AgeCommit message (Collapse)AuthorFilesLines
2015-05-19x86/fpu: Move XCR0 manipulation to the FPU code properIngo Molnar1-0/+12
The suspend code accesses FPU state internals, add a helper for it and isolate it. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Rename 'xstate_features' to 'xfeatures_nr'Ingo Molnar1-6/+8
The name 'xstate_features' does not tell us whether it's a bitmap or any other value. That it's a count of features is only obvious if you read the code that calculates it. Rename it to the more descriptive 'xfeatures_nr' name. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Rename 'pcntxt_mask' to 'xfeatures_mask'Ingo Molnar2-30/+30
So the 'pcntxt_mask' is a misnomer, it's essentially meaningless to anyone who doesn't know what it does exactly. Name it more descriptively as 'xfeatures_mask'. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Print supported xstate features in human readable wayIngo Molnar1-0/+25
Inform the user/admin about which xstate features the kernel supports. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Improve FPU detection kernel messagesIngo Molnar1-12/+14
Standardize the various boot time messages printed during FPU detection: - Use a common 'x86/fpu: ' prefix for consistency and to make it easy to grep boot logs for FPU related messages - Correct speling errors - Add printout for the legacy FPU case as well - Clarify messages Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Remove xsave_init() __init obfuscationIngo Molnar1-9/+11
So this code surprised me - and being surprised when reading FPU code does not help maintainability of an already overly complex subsystem. Remove the obfuscation and just don't use __init annotation for now. Anyone who wants to free these ~600 bytes of xstate_enable_boot_cpu() should implement it cleanly. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Rename fpu-internal.h to fpu/internal.hIngo Molnar3-3/+3
This unifies all the FPU related header files under a unified, hiearchical naming scheme: - asm/fpu/types.h: FPU related data types, needed for 'struct task_struct', widely included in almost all kernel code, and hence kept as small as possible. - asm/fpu/api.h: FPU related 'public' methods exported to other subsystems. - asm/fpu/internal.h: FPU subsystem internal methods - asm/fpu/xsave.h: XSAVE support internal methods (Also standardize the header guard in asm/fpu/internal.h.) Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Rename i387.h to fpu/api.hIngo Molnar1-1/+1
We already have fpu/types.h, move i387.h to fpu/api.h. The file name has become a misnomer anyway: it offers generic FPU APIs, but is not limited to i387 functionality. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Clean up fpu__clear() a bitIngo Molnar1-2/+2
Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Rename fpu__flush_thread() to fpu__clear()Ingo Molnar1-2/+2
The primary purpose of this function is to clear the current task's FPU before an exec(), to not leak information from the previous task, and to allow the new task to start with freshly initialized FPU registers. Rename the function to reflect this primary purpose. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Use 'struct fpu' in fpu__unlazy_stopped()Ingo Molnar1-12/+17
Migrate this function to pure 'struct fpu' usage. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Use 'struct fpu' in fpstate_alloc_init()Ingo Molnar2-8/+7
Migrate this function to pure 'struct fpu' usage. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Use 'struct fpu' in fpu__copy()Ingo Molnar1-8/+5
Migrate this function to pure 'struct fpu' usage. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Use 'struct fpu' in fpu_copy()Ingo Molnar1-7/+4
Migrate this function to pure 'struct fpu' usage. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Use 'struct fpu' in fpu__save()Ingo Molnar1-5/+3
Migrate this function to pure 'struct fpu' usage. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Use 'struct fpu' in __fpu_save()Ingo Molnar1-10/+10
Migrate this function to pure 'struct fpu' usage. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Move __save_fpu() into fpu/core.cIngo Molnar1-0/+12
This helper function is only used in fpu/core.c, move it there. This slightly speeds up compilation. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Use 'struct fpu' in fpu_reset_state()Ingo Molnar2-6/+5
Migrate this function to pure 'struct fpu' usage. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Use 'struct fpu' in restore_fpu_checking()Ingo Molnar1-2/+2
Migrate this function to pure 'struct fpu' usage. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Remove task_disable_lazy_fpu_restore()Ingo Molnar1-3/+2
Replace task_disable_lazy_fpu_restore() with easier to read open-coded uses: we already update the fpu->last_cpu field explicitly in other cases. (This also removes yet another task_struct using FPU method.) Better explain the fpu::last_cpu field in the structure definition. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Remove 'struct task_struct' usage from drop_fpu()Ingo Molnar2-2/+2
Migrate this function to pure 'struct fpu' usage. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Get rid of PF_USED_MATH usage, convert it to fpu->fpstate_activeIngo Molnar2-18/+31
Introduce a simple fpu->fpstate_active flag in the fpu context data structure and use that instead of PF_USED_MATH in task->flags. Testing for this flag byte should be slightly more efficient than testing a bit in a bitmask, but the main advantage is that most FPU functions can now be performed on a 'struct fpu' alone, they don't need access to 'struct task_struct' anymore. There's a slight linecount increase, mostly due to the 'fpu' local variables and due to extra comments. The local variables will go away once we move most of the FPU methods to pure 'struct fpu' parameters. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Document fpu__unlazy_stopped()Ingo Molnar1-4/+21
Explain its usage and also document a TODO item. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Open code PF_USED_MATH usagesIngo Molnar2-11/+13
PF_USED_MATH is used directly, but also in a handful of helper inlines. To ease the elimination of PF_USED_MATH, convert all inline helpers to open-coded PF_USED_MATH usage. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Remove 'struct task_struct' usage from __thread_fpu_begin()Ingo Molnar1-1/+2
Migrate this function to pure 'struct fpu' usage. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Remove 'struct task_struct' usage from __thread_fpu_end()Ingo Molnar1-1/+1
Migrate this function to pure 'struct fpu' usage. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Change fpu_owner_task to fpu_fpregs_owner_ctxIngo Molnar1-5/+4
Track the FPU owner context instead of the owner task: this change, together with other changes, will allow in subsequent patches the elimination of 'struct task_struct' usage in various FPU code: we'll be able to use 'struct fpu' only. There's no change in code size: text data bss dec hex filename 13066467 2545248 1626112 17237827 1070743 vmlinux.before 13066467 2545248 1626112 17237827 1070743 vmlinux.after Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Move 'PER_CPU(fpu_owner_task)' to fpu/core.cIngo Molnar1-0/+5
Move it closer to other per-cpu FPU data structures. This also unifies the 32-bit and 64-bit code. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Eliminate the __thread_has_fpu() wrapperIngo Molnar1-7/+10
Start migrating FPU methods towards using 'struct fpu *fpu' directly. __thread_has_fpu() is just a trivial wrapper around fpu->has_fpu, eliminate it. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Print out whether we are doing lazy/eager FPU context switchesIngo Molnar1-0/+2
Ever since the kernel started defaulting to eager FPU switches on modern Intel CPUs it's not been obvious whether a given system is using the lazy or the eager FPU context switching logic. So generate a boot message about which mode the FPU code is in: x86/fpu: Using 'lazy' FPU context switches. or: x86/fpu: Using 'eager' FPU context switches. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Add debugging check to fpu_copy()Ingo Molnar1-0/+10
Also add a bit of documentation. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Move fpu_copy() to fpu/core.cIngo Molnar1-0/+14
Move fpu_copy() where its only user is. Beyond readability this also speeds up compilation, as fpu-internal.h is included in over a dozen .c files. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Remove __save_init_fpu()Ingo Molnar1-2/+2
__save_init_fpu() is just a trivial wrapper around fpu_save_init(). Remove the extra layer of obfuscation. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Add kernel_fpu_disabled()Ingo Molnar1-1/+17
Instead of open-coded in_kernel_fpu access, Use kernel_fpu_disabled() in interrupted_kernel_fpu_idle(), matching the other kernel_fpu_*() methods. Also add some documentation for in_kernel_fpu. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Add debug check to kernel_fpu_disable()Ingo Molnar1-2/+3
We are not supposed to call kernel_fpu_disable() if we have not previously enabled it. Also use kernel_fpu_disable()/enable() in the __kernel_fpu_begin/end() primitives, instead of writing to in_kernel_fpu directly, so that we get the debugging checks. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Make kernel_fpu_disable/enable() staticIngo Molnar1-4/+4
This allows the compiler to inline them and to eliminate them: arch/x86/kernel/fpu/core.o: text data bss dec hex filename 6741 4 8 6753 1a61 core.o.before 6716 4 8 6728 1a48 core.o.after Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Make task_xstate_cachep staticIngo Molnar1-2/+1
It's now local to fpu/core.c, make it static. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Uninline fpstate_free() and move it next to the allocation functionIngo Molnar1-0/+9
Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Factor out fpu__copy()Ingo Molnar1-0/+18
Introduce fpu__copy() and use it in arch_dup_task_struct(), thus moving another chunk of FPU logic to fpu/core.c. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Move task_xstate_cachep handling to core.cIngo Molnar1-0/+15
This code was historically in process.c, now we have FPU core internals in fpu/core.c instead - move it there. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Simplify the xsave_state*() methodsIngo Molnar1-1/+2
These functions (xsave_state() and xsave_state_booting()) have a 'mask' argument that is always -1. Propagate this into the functions instead and eliminate the extra argument. Does not change the generated code, because these were inlined functions. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Factor out the FPU bug detection code into fpu__init_check_bugs()Ingo Molnar1-0/+63
Move the boot-time FPU bug detection code to the other FPU boot time init code in fpu/init.c. No change in code size: text data bss dec hex filename 13044568 1884440 1130496 16059504 f50c70 vmlinux.before 13044568 1884440 1130496 16059504 f50c70 vmlinux.after Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Rename math_state_restore() to fpu__restore()Ingo Molnar2-4/+4
Move to the new fpu__*() namespace. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Move math_state_restore() to fpu/core.cIngo Molnar1-0/+42
It's another piece of FPU internals that is better off close to the other FPU internals. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Factor out fpu__flush_thread() from flush_thread()Ingo Molnar1-0/+17
flush_thread() open codes a lot of FPU internals - create a separate function for it in fpu/core.c. Turns out that this does not hurt performance: text data bss dec hex filename 11843039 1884440 1130496 14857975 e2b6f7 vmlinux.before 11843039 1884440 1130496 14857975 e2b6f7 vmlinux.after and since this is a slowpath clarity comes first anyway. We can reconsider inlining decisions after the FPU code has been cleaned up. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Move the no_387 handling and FPU detection code into init.cIngo Molnar2-34/+34
Both no_387() and fpu__detect() run at boot time, so they belong into init.c. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Remove unnecessary includes from core.cIngo Molnar1-13/+0
fpu/core.c includes a lot of files for mostly historic reasons. It only needs fpu-internal.h, which already includes all the required headers. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Split out the boot time FPU init code into fpu/init.cIngo Molnar3-89/+94
Move boot time FPU initialization code into init.c, to better isolate it into its own domain. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>
2015-05-19x86/fpu: Move i387.c and xsave.c to arch/x86/kernel/fpu/Ingo Molnar3-0/+1447
Create a new subdirectory for the FPU support code in arch/x86/kernel/fpu/. Rename 'i387.c' to 'core.c' - as this really collects the core FPU support code, nothing i387 specific. We'll better organize this directory in later patches. Reviewed-by: Borislav Petkov <[email protected]> Cc: Andy Lutomirski <[email protected]> Cc: Dave Hansen <[email protected]> Cc: Fenghua Yu <[email protected]> Cc: H. Peter Anvin <[email protected]> Cc: Linus Torvalds <[email protected]> Cc: Oleg Nesterov <[email protected]> Cc: Peter Zijlstra <[email protected]> Cc: Thomas Gleixner <[email protected]> Signed-off-by: Ingo Molnar <[email protected]>