| Age | Commit message (Collapse) | Author | Files | Lines |
|---|
|
Export the present bit of page fault error code, the later patch
will use it
Signed-off-by: Xiao Guangrong <[email protected]>
Signed-off-by: Avi Kivity <[email protected]>
|
|
Use __drop_large_spte to cleanup this function and comment spte_write_protect
Signed-off-by: Xiao Guangrong <[email protected]>
Signed-off-by: Avi Kivity <[email protected]>
|
|
Introduce a common function to abstract spte write-protect to
cleanup the code
Signed-off-by: Xiao Guangrong <[email protected]>
Signed-off-by: Avi Kivity <[email protected]>
|
|
The reture value of __rmap_write_protect is either 1 or 0, use
true/false instead of these
Signed-off-by: Xiao Guangrong <[email protected]>
Signed-off-by: Avi Kivity <[email protected]>
|
|
Our emulation should be complete enough that we can emulate guests
while they are in big real mode, or in a mode transition that is not
virtualizable without unrestricted guest support.
Signed-off-by: Avi Kivity <[email protected]>
|
|
Opcode 0F 00 /3. Encountered during Windows XP secondary processor bringup.
Signed-off-by: Avi Kivity <[email protected]>
|
|
Guest software doesn't actually depend on it, but vmx will refuse us
entry if we don't. Set the bit in both the cached segment and memory,
just to be nice.
Signed-off-by: Avi Kivity <[email protected]>
|
|
Some operations want to modify the descriptor later on, so save the
address for future use.
Signed-off-by: Avi Kivity <[email protected]>
|
|
Opcode 0F 00 /2. Used by isolinux durign the protected mode transition.
Signed-off-by: Avi Kivity <[email protected]>
|
|
Opcodes 0F C8 - 0F CF.
Used by the SeaBIOS cdrom code (though not in big real mode).
Signed-off-by: Avi Kivity <[email protected]>
|
|
If instruction emulation fails, report it properly to userspace.
Signed-off-by: Avi Kivity <[email protected]>
|
|
Process the event, possibly injecting an interrupt, before continuing.
Signed-off-by: Avi Kivity <[email protected]>
|
|
Opcode C8.
Only ENTER with lexical nesting depth 0 is implemented, since others are
very rare. We'll fail emulation if nonzero lexical depth is used so data
is not corrupted.
Signed-off-by: Avi Kivity <[email protected]>
|
|
This allows us to reuse the code without populating ctxt->src and
overriding ctxt->op_bytes.
Signed-off-by: Avi Kivity <[email protected]>
|
|
Commit 2adb5ad9fe1 removed ByteOp from MOVZX/MOVSX, replacing them by
SrcMem8, but neglected to fix the dependency in the emulation code
on ByteOp. This caused the instruction not to have any effect in
some circumstances.
Fix by replacing the check for ByteOp with the equivalent src.op_bytes == 1.
Signed-off-by: Avi Kivity <[email protected]>
|
|
Opcode 9F.
Signed-off-by: Avi Kivity <[email protected]>
|
|
If we return early from an invalid guest state emulation loop, make
sure we return to it later if the guest state is still invalid.
Signed-off-by: Avi Kivity <[email protected]>
|
|
Checking EFLAGS.IF is incorrect as we might be in interrupt shadow. If
that is the case, the main loop will notice that and not inject the interrupt,
causing an endless loop.
Fix by using vmx_interrupt_allowed() to check if we can inject an interrupt
instead.
Signed-off-by: Avi Kivity <[email protected]>
|
|
Opcodes 0F 01 /0 and 0F 01 /1
Signed-off-by: Avi Kivity <[email protected]>
|
|
We correctly default to SS when BP is used as a base in 16-bit address mode,
but we don't do that for 32-bit mode.
Fix by adjusting the default to SS when either ESP or EBP is used as the base
register.
Signed-off-by: Avi Kivity <[email protected]>
|
|
memop is not initialized; this can lead to a two-byte operation
following a 4-byte operation to see garbage values. Usually
truncation fixes things fot us later on, but at least in one case
(call abs) it doesn't.
Fix by moving memop to the auto-initialized field area.
Signed-off-by: Avi Kivity <[email protected]>
|
|
Opcode c9; used by some variants of Windows during boot, in big real mode.
Signed-off-by: Avi Kivity <[email protected]>
|
|
Otherwise, if the guest ends up looping, we never exit the srcu critical
section, which causes synchronize_srcu() to hang.
Signed-off-by: Avi Kivity <[email protected]>
|
|
Some userspace (e.g. QEMU 1.1) munge the d and g bits of segment
descriptors, causing us not to recognize them as unusable segments
with emulate_invalid_guest_state=1. Relax the check by testing for
segment not present (a non-present segment cannot be usable).
Signed-off-by: Avi Kivity <[email protected]>
|
|
The operand size for these instructions is 8 bytes in long mode, even without
a REX prefix. Set it explicitly.
Triggered while booting Linux with emulate_invalid_guest_state=1.
Signed-off-by: Avi Kivity <[email protected]>
|
|
Null SS is valid in long mode; allow loading it.
Signed-off-by: Avi Kivity <[email protected]>
|
|
Opcode 0F A2.
Used by Linux during the mode change trampoline while in a state that is
not virtualizable on vmx without unrestricted_guest, so we need to emulate
it is emulate_invalid_guest_state=1.
Signed-off-by: Avi Kivity <[email protected]>
|
|
Instead of getting an exact leaf, follow the spec and fall back to the last
main leaf instead. This lets us easily emulate the cpuid instruction in the
emulator.
Signed-off-by: Avi Kivity <[email protected]>
|
|
Introduce kvm_cpuid() to perform the leaf limit check and calculate
register values, and let kvm_emulate_cpuid() just handle reading and
writing the registers from/to the vcpu. This allows us to reuse
kvm_cpuid() in a context where directly reading and writing registers
is not desired.
Signed-off-by: Avi Kivity <[email protected]>
|
|
In protected mode, the CPL is defined as the lower two bits of CS, as set by
the last far jump. But during the transition to protected mode, there is no
last far jump, so we need to return zero (the inherited real mode CPL).
Fix by reading CPL from the cache during the transition. This isn't 100%
correct since we don't set the CPL cache on a far jump, but since protected
mode transition will always jump to a segment with RPL=0, it will always
work.
Signed-off-by: Avi Kivity <[email protected]>
|
|
Currently the MMU's ->new_cr3() callback does nothing when guest paging
is disabled or when two-dimentional paging (e.g. EPT on Intel) is active.
This means that an emulated write to cr3 can be lost; kvm_set_cr3() will
write vcpu-arch.cr3, but the GUEST_CR3 field in the VMCS will retain its
old value and this is what the guest sees.
This bug did not have any effect until now because:
- with unrestricted guest, or with svm, we never emulate a mov cr3 instruction
- without unrestricted guest, and with paging enabled, we also never emulate a
mov cr3 instruction
- without unrestricted guest, but with paging disabled, the guest's cr3 is
ignored until the guest enables paging; at this point the value from arch.cr3
is loaded correctly my the mov cr0 instruction which turns on paging
However, the patchset that enables big real mode causes us to emulate mov cr3
instructions in protected mode sometimes (when guest state is not virtualizable
by vmx); this mov cr3 is effectively ignored and will crash the guest.
The fix is to make nonpaging_new_cr3() call mmu_free_roots() to force a cr3
reload. This is awkward because now all the new_cr3 callbacks to the same
thing, and because mmu_free_roots() is somewhat of an overkill; but fixing
that is more complicated and will be done after this minimal fix.
Observed in the Window XP 32-bit installer while bringing up secondary vcpus.
Signed-off-by: Avi Kivity <[email protected]>
|
|
If last_boosted_vcpu == 0, then we fall through all test cases and
may end up with all VCPUs pouncing on vcpu 0. With a large enough
guest, this can result in enormous runqueue lock contention, which
can prevent vcpu0 from running, leading to a livelock.
Changing < to <= makes sure we properly handle that case.
Signed-off-by: Rik van Riel <[email protected]>
Signed-off-by: Marcelo Tosatti <[email protected]>
|
|
If sigp sense doesn't have any status bits to report, it should set
cc 0 and leave the register as-is.
Since we know about the external call pending bit, we should report
it if it is set as well.
Acked-by: Heiko Carstens <[email protected]>
Reviewed-by: Christian Borntraeger <[email protected]>
Signed-off-by: Cornelia Huck <[email protected]>
Signed-off-by: Marcelo Tosatti <[email protected]>
|
|
Just use the defines instead of using plain numbers and adding
a comment behind each line.
Signed-off-by: Heiko Carstens <[email protected]>
Signed-off-by: Cornelia Huck <[email protected]>
Signed-off-by: Marcelo Tosatti <[email protected]>
|
|
If an invalid parameter is passed or the addressed cpu is in an
incorrect state sigp set prefix will store a status.
This status must only have bits set as defined by the architecture.
The current kvm implementation missed to clear bits and also did
not set the intended status bit ("and" instead of "or" operation).
Signed-off-by: Heiko Carstens <[email protected]>
Signed-off-by: Cornelia Huck <[email protected]>
Signed-off-by: Marcelo Tosatti <[email protected]>
|
|
Only if the sensed cpu is not running a status is stored, which
is reflected by condition code 1. If the cpu is running, condition
code 0 should be returned.
Just the opposite of what the code is doing.
Acked-by: Cornelia Huck <[email protected]>
Signed-off-by: Heiko Carstens <[email protected]>
Signed-off-by: Cornelia Huck <[email protected]>
Signed-off-by: Marcelo Tosatti <[email protected]>
|
|
The smp and the kvm code have different defines for the sigp order codes.
Let's just have a single place where these are defined.
Also move the sigp condition code and sigp cpu status bits to the new
sigp.h header file.
Signed-off-by: Heiko Carstens <[email protected]>
Signed-off-by: Martin Schwidefsky <[email protected]>
Signed-off-by: Cornelia Huck <[email protected]>
Signed-off-by: Marcelo Tosatti <[email protected]>
|
|
condition code "status stored" for sigp sense running always implies
that only the "not running" status bit is set. Therefore no need to
check if it is set.
Signed-off-by: Heiko Carstens <[email protected]>
Signed-off-by: Martin Schwidefsky <[email protected]>
Signed-off-by: Cornelia Huck <[email protected]>
Signed-off-by: Marcelo Tosatti <[email protected]>
|
|
In order to avoid compilation failure when KVM is not compiled in,
guard the mmu_notifier specific sections with both CONFIG_MMU_NOTIFIER
and KVM_ARCH_WANT_MMU_NOTIFIER, like it is being done in the rest of
the KVM code.
Signed-off-by: Marc Zyngier <[email protected]>
Signed-off-by: Christoffer Dall <[email protected]>
Signed-off-by: Marcelo Tosatti <[email protected]>
|
|
Signed-off-by: Guo Chao <[email protected]>
Signed-off-by: Marcelo Tosatti <[email protected]>
|
|
On UP i386, when APIC is disabled
# CONFIG_X86_UP_APIC is not set
# CONFIG_PCI_IOAPIC is not set
code looking at apicdrivers never has any effect but it
still gets compiled in. In particular, this causes
build failures with kvm, but it generally bloats the kernel
unnecessarily.
Fix by defining both __apicdrivers and __apicdrivers_end
to be NULL when CONFIG_X86_LOCAL_APIC is unset: I verified
that as the result any loop scanning __apicdrivers gets optimized out by
the compiler.
Warning: a .config with apic disabled doesn't seem to boot
for me (even without this patch). Still verifying why,
meanwhile this patch is compile-tested only.
Signed-off-by: Michael S. Tsirkin <[email protected]>
Reported-by: Randy Dunlap <[email protected]>
Acked-by: Randy Dunlap <[email protected]>
Acked-by: H. Peter Anvin <[email protected]>
Signed-off-by: Marcelo Tosatti <[email protected]>
|
|
Implementation of PV EOI using shared memory.
This reduces the number of exits an interrupt
causes as much as by half.
The idea is simple: there's a bit, per APIC, in guest memory,
that tells the guest that it does not need EOI.
We set it before injecting an interrupt and clear
before injecting a nested one. Guest tests it using
a test and clear operation - this is necessary
so that host can detect interrupt nesting -
and if set, it can skip the EOI MSR.
There's a new MSR to set the address of said register
in guest memory. Otherwise not much changed:
- Guest EOI is not required
- Register is tested & ISR is automatically cleared on exit
For testing results see description of previous patch
'kvm_para: guest side for eoi avoidance'.
Signed-off-by: Michael S. Tsirkin <[email protected]>
Signed-off-by: Avi Kivity <[email protected]>
|
|
Each time we need to cancel injection we invoke same code
(cancel_injection callback). Move it towards the end of function using
the familiar goto on error pattern.
Will make it easier to do more cleanups for PV EOI.
Signed-off-by: Michael S. Tsirkin <[email protected]>
Signed-off-by: Avi Kivity <[email protected]>
|
|
Commit eb0dc6d0368072236dcd086d7fdc17fd3c4574d4 introduced apic
attention bitmask but kvm still syncs lapic unconditionally.
As that commit suggested and in anticipation of adding more attention
bits, only sync lapic if(apic_attention).
Signed-off-by: Michael S. Tsirkin <[email protected]>
Signed-off-by: Avi Kivity <[email protected]>
|
|
Document the new EOI MSR. Couldn't decide whether this change belongs
conceptually on guest or host side, so a separate patch.
Signed-off-by: Michael S. Tsirkin <[email protected]>
Signed-off-by: Avi Kivity <[email protected]>
|
|
__test_and_clear_bit is actually atomic with respect
to the local CPU. Add a note saying that KVM on x86
relies on this behaviour so people don't accidentaly break it.
Also warn not to rely on this in portable code.
Signed-off-by: Michael S. Tsirkin <[email protected]>
Signed-off-by: Avi Kivity <[email protected]>
|
|
The idea is simple: there's a bit, per APIC, in guest memory,
that tells the guest that it does not need EOI.
Guest tests it using a single est and clear operation - this is
necessary so that host can detect interrupt nesting - and if set, it can
skip the EOI MSR.
I run a simple microbenchmark to show exit reduction
(note: for testing, need to apply follow-up patch
'kvm: host side for eoi optimization' + a qemu patch
I posted separately, on host):
Before:
Performance counter stats for 'sleep 1s':
47,357 kvm:kvm_entry [99.98%]
0 kvm:kvm_hypercall [99.98%]
0 kvm:kvm_hv_hypercall [99.98%]
5,001 kvm:kvm_pio [99.98%]
0 kvm:kvm_cpuid [99.98%]
22,124 kvm:kvm_apic [99.98%]
49,849 kvm:kvm_exit [99.98%]
21,115 kvm:kvm_inj_virq [99.98%]
0 kvm:kvm_inj_exception [99.98%]
0 kvm:kvm_page_fault [99.98%]
22,937 kvm:kvm_msr [99.98%]
0 kvm:kvm_cr [99.98%]
0 kvm:kvm_pic_set_irq [99.98%]
0 kvm:kvm_apic_ipi [99.98%]
22,207 kvm:kvm_apic_accept_irq [99.98%]
22,421 kvm:kvm_eoi [99.98%]
0 kvm:kvm_pv_eoi [99.99%]
0 kvm:kvm_nested_vmrun [99.99%]
0 kvm:kvm_nested_intercepts [99.99%]
0 kvm:kvm_nested_vmexit [99.99%]
0 kvm:kvm_nested_vmexit_inject [99.99%]
0 kvm:kvm_nested_intr_vmexit [99.99%]
0 kvm:kvm_invlpga [99.99%]
0 kvm:kvm_skinit [99.99%]
57 kvm:kvm_emulate_insn [99.99%]
0 kvm:vcpu_match_mmio [99.99%]
0 kvm:kvm_userspace_exit [99.99%]
2 kvm:kvm_set_irq [99.99%]
2 kvm:kvm_ioapic_set_irq [99.99%]
23,609 kvm:kvm_msi_set_irq [99.99%]
1 kvm:kvm_ack_irq [99.99%]
131 kvm:kvm_mmio [99.99%]
226 kvm:kvm_fpu [100.00%]
0 kvm:kvm_age_page [100.00%]
0 kvm:kvm_try_async_get_page [100.00%]
0 kvm:kvm_async_pf_doublefault [100.00%]
0 kvm:kvm_async_pf_not_present [100.00%]
0 kvm:kvm_async_pf_ready [100.00%]
0 kvm:kvm_async_pf_completed
1.002100578 seconds time elapsed
After:
Performance counter stats for 'sleep 1s':
28,354 kvm:kvm_entry [99.98%]
0 kvm:kvm_hypercall [99.98%]
0 kvm:kvm_hv_hypercall [99.98%]
1,347 kvm:kvm_pio [99.98%]
0 kvm:kvm_cpuid [99.98%]
1,931 kvm:kvm_apic [99.98%]
29,595 kvm:kvm_exit [99.98%]
24,884 kvm:kvm_inj_virq [99.98%]
0 kvm:kvm_inj_exception [99.98%]
0 kvm:kvm_page_fault [99.98%]
1,986 kvm:kvm_msr [99.98%]
0 kvm:kvm_cr [99.98%]
0 kvm:kvm_pic_set_irq [99.98%]
0 kvm:kvm_apic_ipi [99.99%]
25,953 kvm:kvm_apic_accept_irq [99.99%]
26,132 kvm:kvm_eoi [99.99%]
26,593 kvm:kvm_pv_eoi [99.99%]
0 kvm:kvm_nested_vmrun [99.99%]
0 kvm:kvm_nested_intercepts [99.99%]
0 kvm:kvm_nested_vmexit [99.99%]
0 kvm:kvm_nested_vmexit_inject [99.99%]
0 kvm:kvm_nested_intr_vmexit [99.99%]
0 kvm:kvm_invlpga [99.99%]
0 kvm:kvm_skinit [99.99%]
284 kvm:kvm_emulate_insn [99.99%]
68 kvm:vcpu_match_mmio [99.99%]
68 kvm:kvm_userspace_exit [99.99%]
2 kvm:kvm_set_irq [99.99%]
2 kvm:kvm_ioapic_set_irq [99.99%]
28,288 kvm:kvm_msi_set_irq [99.99%]
1 kvm:kvm_ack_irq [99.99%]
131 kvm:kvm_mmio [100.00%]
588 kvm:kvm_fpu [100.00%]
0 kvm:kvm_age_page [100.00%]
0 kvm:kvm_try_async_get_page [100.00%]
0 kvm:kvm_async_pf_doublefault [100.00%]
0 kvm:kvm_async_pf_not_present [100.00%]
0 kvm:kvm_async_pf_ready [100.00%]
0 kvm:kvm_async_pf_completed
1.002039622 seconds time elapsed
We see that # of exits is almost halved.
Signed-off-by: Michael S. Tsirkin <[email protected]>
Signed-off-by: Avi Kivity <[email protected]>
|
|
We perform ISR lookups twice: during interrupt
injection and on EOI. Typical workloads only have
a single bit set there. So we can avoid ISR scans by
1. counting bits as we set/clear them in ISR
2. on set, caching the injected vector number
3. on clear, invalidating the cache
The real purpose of this is enabling PV EOI
which needs to quickly validate the vector.
But non PV guests also benefit: with this patch,
and without interrupt nesting, apic_find_highest_isr
will always return immediately without scanning ISR.
Signed-off-by: Michael S. Tsirkin <[email protected]>
Signed-off-by: Avi Kivity <[email protected]>
|
|
Signed-off-by: Michael S. Tsirkin <[email protected]>
Signed-off-by: Avi Kivity <[email protected]>
|
|
The following commit did not care about the error handling path:
commit c1a7b32a14138f908df52d7c53b5ce3415ec6b50
KVM: Avoid wasting pages for small lpage_info arrays
If memory allocation fails, vfree() will be called with the address
returned by kzalloc(). This patch fixes this issue.
Signed-off-by: Takuya Yoshikawa <[email protected]>
Signed-off-by: Avi Kivity <[email protected]>
|