0001 .. SPDX-License-Identifier: GPL-2.0
0002
0003 =================================
0004 The PPC KVM paravirtual interface
0005 =================================
0006
0007 The basic execution principle by which KVM on PowerPC works is to run all kernel
0008 space code in PR=1 which is user space. This way we trap all privileged
0009 instructions and can emulate them accordingly.
0010
0011 Unfortunately that is also the downfall. There are quite some privileged
0012 instructions that needlessly return us to the hypervisor even though they
0013 could be handled differently.
0014
0015 This is what the PPC PV interface helps with. It takes privileged instructions
0016 and transforms them into unprivileged ones with some help from the hypervisor.
0017 This cuts down virtualization costs by about 50% on some of my benchmarks.
0018
0019 The code for that interface can be found in arch/powerpc/kernel/kvm*
0020
0021 Querying for existence
0022 ======================
0023
0024 To find out if we're running on KVM or not, we leverage the device tree. When
0025 Linux is running on KVM, a node /hypervisor exists. That node contains a
0026 compatible property with the value "linux,kvm".
0027
0028 Once you determined you're running under a PV capable KVM, you can now use
0029 hypercalls as described below.
0030
0031 KVM hypercalls
0032 ==============
0033
0034 Inside the device tree's /hypervisor node there's a property called
0035 'hypercall-instructions'. This property contains at most 4 opcodes that make
0036 up the hypercall. To call a hypercall, just call these instructions.
0037
0038 The parameters are as follows:
0039
0040 ======== ================ ================
0041 Register IN OUT
0042 ======== ================ ================
0043 r0 - volatile
0044 r3 1st parameter Return code
0045 r4 2nd parameter 1st output value
0046 r5 3rd parameter 2nd output value
0047 r6 4th parameter 3rd output value
0048 r7 5th parameter 4th output value
0049 r8 6th parameter 5th output value
0050 r9 7th parameter 6th output value
0051 r10 8th parameter 7th output value
0052 r11 hypercall number 8th output value
0053 r12 - volatile
0054 ======== ================ ================
0055
0056 Hypercall definitions are shared in generic code, so the same hypercall numbers
0057 apply for x86 and powerpc alike with the exception that each KVM hypercall
0058 also needs to be ORed with the KVM vendor code which is (42 << 16).
0059
0060 Return codes can be as follows:
0061
0062 ==== =========================
0063 Code Meaning
0064 ==== =========================
0065 0 Success
0066 12 Hypercall not implemented
0067 <0 Error
0068 ==== =========================
0069
0070 The magic page
0071 ==============
0072
0073 To enable communication between the hypervisor and guest there is a new shared
0074 page that contains parts of supervisor visible register state. The guest can
0075 map this shared page using the KVM hypercall KVM_HC_PPC_MAP_MAGIC_PAGE.
0076
0077 With this hypercall issued the guest always gets the magic page mapped at the
0078 desired location. The first parameter indicates the effective address when the
0079 MMU is enabled. The second parameter indicates the address in real mode, if
0080 applicable to the target. For now, we always map the page to -4096. This way we
0081 can access it using absolute load and store functions. The following
0082 instruction reads the first field of the magic page::
0083
0084 ld rX, -4096(0)
0085
0086 The interface is designed to be extensible should there be need later to add
0087 additional registers to the magic page. If you add fields to the magic page,
0088 also define a new hypercall feature to indicate that the host can give you more
0089 registers. Only if the host supports the additional features, make use of them.
0090
0091 The magic page layout is described by struct kvm_vcpu_arch_shared
0092 in arch/powerpc/include/asm/kvm_para.h.
0093
0094 Magic page features
0095 ===================
0096
0097 When mapping the magic page using the KVM hypercall KVM_HC_PPC_MAP_MAGIC_PAGE,
0098 a second return value is passed to the guest. This second return value contains
0099 a bitmap of available features inside the magic page.
0100
0101 The following enhancements to the magic page are currently available:
0102
0103 ============================ =======================================
0104 KVM_MAGIC_FEAT_SR Maps SR registers r/w in the magic page
0105 KVM_MAGIC_FEAT_MAS0_TO_SPRG7 Maps MASn, ESR, PIR and high SPRGs
0106 ============================ =======================================
0107
0108 For enhanced features in the magic page, please check for the existence of the
0109 feature before using them!
0110
0111 Magic page flags
0112 ================
0113
0114 In addition to features that indicate whether a host is capable of a particular
0115 feature we also have a channel for a guest to tell the guest whether it's capable
0116 of something. This is what we call "flags".
0117
0118 Flags are passed to the host in the low 12 bits of the Effective Address.
0119
0120 The following flags are currently available for a guest to expose:
0121
0122 MAGIC_PAGE_FLAG_NOT_MAPPED_NX Guest handles NX bits correctly wrt magic page
0123
0124 MSR bits
0125 ========
0126
0127 The MSR contains bits that require hypervisor intervention and bits that do
0128 not require direct hypervisor intervention because they only get interpreted
0129 when entering the guest or don't have any impact on the hypervisor's behavior.
0130
0131 The following bits are safe to be set inside the guest:
0132
0133 - MSR_EE
0134 - MSR_RI
0135
0136 If any other bit changes in the MSR, please still use mtmsr(d).
0137
0138 Patched instructions
0139 ====================
0140
0141 The "ld" and "std" instructions are transformed to "lwz" and "stw" instructions
0142 respectively on 32 bit systems with an added offset of 4 to accommodate for big
0143 endianness.
0144
0145 The following is a list of mapping the Linux kernel performs when running as
0146 guest. Implementing any of those mappings is optional, as the instruction traps
0147 also act on the shared page. So calling privileged instructions still works as
0148 before.
0149
0150 ======================= ================================
0151 From To
0152 ======================= ================================
0153 mfmsr rX ld rX, magic_page->msr
0154 mfsprg rX, 0 ld rX, magic_page->sprg0
0155 mfsprg rX, 1 ld rX, magic_page->sprg1
0156 mfsprg rX, 2 ld rX, magic_page->sprg2
0157 mfsprg rX, 3 ld rX, magic_page->sprg3
0158 mfsrr0 rX ld rX, magic_page->srr0
0159 mfsrr1 rX ld rX, magic_page->srr1
0160 mfdar rX ld rX, magic_page->dar
0161 mfdsisr rX lwz rX, magic_page->dsisr
0162
0163 mtmsr rX std rX, magic_page->msr
0164 mtsprg 0, rX std rX, magic_page->sprg0
0165 mtsprg 1, rX std rX, magic_page->sprg1
0166 mtsprg 2, rX std rX, magic_page->sprg2
0167 mtsprg 3, rX std rX, magic_page->sprg3
0168 mtsrr0 rX std rX, magic_page->srr0
0169 mtsrr1 rX std rX, magic_page->srr1
0170 mtdar rX std rX, magic_page->dar
0171 mtdsisr rX stw rX, magic_page->dsisr
0172
0173 tlbsync nop
0174
0175 mtmsrd rX, 0 b <special mtmsr section>
0176 mtmsr rX b <special mtmsr section>
0177
0178 mtmsrd rX, 1 b <special mtmsrd section>
0179
0180 [Book3S only]
0181 mtsrin rX, rY b <special mtsrin section>
0182
0183 [BookE only]
0184 wrteei [0|1] b <special wrteei section>
0185 ======================= ================================
0186
0187 Some instructions require more logic to determine what's going on than a load
0188 or store instruction can deliver. To enable patching of those, we keep some
0189 RAM around where we can live translate instructions to. What happens is the
0190 following:
0191
0192 1) copy emulation code to memory
0193 2) patch that code to fit the emulated instruction
0194 3) patch that code to return to the original pc + 4
0195 4) patch the original instruction to branch to the new code
0196
0197 That way we can inject an arbitrary amount of code as replacement for a single
0198 instruction. This allows us to check for pending interrupts when setting EE=1
0199 for example.
0200
0201 Hypercall ABIs in KVM on PowerPC
0202 =================================
0203
0204 1) KVM hypercalls (ePAPR)
0205
0206 These are ePAPR compliant hypercall implementation (mentioned above). Even
0207 generic hypercalls are implemented here, like the ePAPR idle hcall. These are
0208 available on all targets.
0209
0210 2) PAPR hypercalls
0211
0212 PAPR hypercalls are needed to run server PowerPC PAPR guests (-M pseries in QEMU).
0213 These are the same hypercalls that pHyp, the POWER hypervisor implements. Some of
0214 them are handled in the kernel, some are handled in user space. This is only
0215 available on book3s_64.
0216
0217 3) OSI hypercalls
0218
0219 Mac-on-Linux is another user of KVM on PowerPC, which has its own hypercall (long
0220 before KVM). This is supported to maintain compatibility. All these hypercalls get
0221 forwarded to user space. This is only useful on book3s_32, but can be used with
0222 book3s_64 as well.
