0001 .. SPDX-License-Identifier: GPL-2.0
0002
0003 =====================================
0004 Intel Trust Domain Extensions (TDX)
0005 =====================================
0006
0007 Intel's Trust Domain Extensions (TDX) protect confidential guest VMs from
0008 the host and physical attacks by isolating the guest register state and by
0009 encrypting the guest memory. In TDX, a special module running in a special
0010 mode sits between the host and the guest and manages the guest/host
0011 separation.
0012
0013 Since the host cannot directly access guest registers or memory, much
0014 normal functionality of a hypervisor must be moved into the guest. This is
0015 implemented using a Virtualization Exception (#VE) that is handled by the
0016 guest kernel. A #VE is handled entirely inside the guest kernel, but some
0017 require the hypervisor to be consulted.
0018
0019 TDX includes new hypercall-like mechanisms for communicating from the
0020 guest to the hypervisor or the TDX module.
0021
0022 New TDX Exceptions
0023 ==================
0024
0025 TDX guests behave differently from bare-metal and traditional VMX guests.
0026 In TDX guests, otherwise normal instructions or memory accesses can cause
0027 #VE or #GP exceptions.
0028
0029 Instructions marked with an '*' conditionally cause exceptions. The
0030 details for these instructions are discussed below.
0031
0032 Instruction-based #VE
0033 ---------------------
0034
0035 - Port I/O (INS, OUTS, IN, OUT)
0036 - HLT
0037 - MONITOR, MWAIT
0038 - WBINVD, INVD
0039 - VMCALL
0040 - RDMSR*,WRMSR*
0041 - CPUID*
0042
0043 Instruction-based #GP
0044 ---------------------
0045
0046 - All VMX instructions: INVEPT, INVVPID, VMCLEAR, VMFUNC, VMLAUNCH,
0047 VMPTRLD, VMPTRST, VMREAD, VMRESUME, VMWRITE, VMXOFF, VMXON
0048 - ENCLS, ENCLU
0049 - GETSEC
0050 - RSM
0051 - ENQCMD
0052 - RDMSR*,WRMSR*
0053
0054 RDMSR/WRMSR Behavior
0055 --------------------
0056
0057 MSR access behavior falls into three categories:
0058
0059 - #GP generated
0060 - #VE generated
0061 - "Just works"
0062
0063 In general, the #GP MSRs should not be used in guests. Their use likely
0064 indicates a bug in the guest. The guest may try to handle the #GP with a
0065 hypercall but it is unlikely to succeed.
0066
0067 The #VE MSRs are typically able to be handled by the hypervisor. Guests
0068 can make a hypercall to the hypervisor to handle the #VE.
0069
0070 The "just works" MSRs do not need any special guest handling. They might
0071 be implemented by directly passing through the MSR to the hardware or by
0072 trapping and handling in the TDX module. Other than possibly being slow,
0073 these MSRs appear to function just as they would on bare metal.
0074
0075 CPUID Behavior
0076 --------------
0077
0078 For some CPUID leaves and sub-leaves, the virtualized bit fields of CPUID
0079 return values (in guest EAX/EBX/ECX/EDX) are configurable by the
0080 hypervisor. For such cases, the Intel TDX module architecture defines two
0081 virtualization types:
0082
0083 - Bit fields for which the hypervisor controls the value seen by the guest
0084 TD.
0085
0086 - Bit fields for which the hypervisor configures the value such that the
0087 guest TD either sees their native value or a value of 0. For these bit
0088 fields, the hypervisor can mask off the native values, but it can not
0089 turn *on* values.
0090
0091 A #VE is generated for CPUID leaves and sub-leaves that the TDX module does
0092 not know how to handle. The guest kernel may ask the hypervisor for the
0093 value with a hypercall.
0094
0095 #VE on Memory Accesses
0096 ======================
0097
0098 There are essentially two classes of TDX memory: private and shared.
0099 Private memory receives full TDX protections. Its content is protected
0100 against access from the hypervisor. Shared memory is expected to be
0101 shared between guest and hypervisor and does not receive full TDX
0102 protections.
0103
0104 A TD guest is in control of whether its memory accesses are treated as
0105 private or shared. It selects the behavior with a bit in its page table
0106 entries. This helps ensure that a guest does not place sensitive
0107 information in shared memory, exposing it to the untrusted hypervisor.
0108
0109 #VE on Shared Memory
0110 --------------------
0111
0112 Access to shared mappings can cause a #VE. The hypervisor ultimately
0113 controls whether a shared memory access causes a #VE, so the guest must be
0114 careful to only reference shared pages it can safely handle a #VE. For
0115 instance, the guest should be careful not to access shared memory in the
0116 #VE handler before it reads the #VE info structure (TDG.VP.VEINFO.GET).
0117
0118 Shared mapping content is entirely controlled by the hypervisor. The guest
0119 should only use shared mappings for communicating with the hypervisor.
0120 Shared mappings must never be used for sensitive memory content like kernel
0121 stacks. A good rule of thumb is that hypervisor-shared memory should be
0122 treated the same as memory mapped to userspace. Both the hypervisor and
0123 userspace are completely untrusted.
0124
0125 MMIO for virtual devices is implemented as shared memory. The guest must
0126 be careful not to access device MMIO regions unless it is also prepared to
0127 handle a #VE.
0128
0129 #VE on Private Pages
0130 --------------------
0131
0132 An access to private mappings can also cause a #VE. Since all kernel
0133 memory is also private memory, the kernel might theoretically need to
0134 handle a #VE on arbitrary kernel memory accesses. This is not feasible, so
0135 TDX guests ensure that all guest memory has been "accepted" before memory
0136 is used by the kernel.
0137
0138 A modest amount of memory (typically 512M) is pre-accepted by the firmware
0139 before the kernel runs to ensure that the kernel can start up without
0140 being subjected to a #VE.
0141
0142 The hypervisor is permitted to unilaterally move accepted pages to a
0143 "blocked" state. However, if it does this, page access will not generate a
0144 #VE. It will, instead, cause a "TD Exit" where the hypervisor is required
0145 to handle the exception.
0146
0147 Linux #VE handler
0148 =================
0149
0150 Just like page faults or #GP's, #VE exceptions can be either handled or be
0151 fatal. Typically, an unhandled userspace #VE results in a SIGSEGV.
0152 An unhandled kernel #VE results in an oops.
0153
0154 Handling nested exceptions on x86 is typically nasty business. A #VE
0155 could be interrupted by an NMI which triggers another #VE and hilarity
0156 ensues. The TDX #VE architecture anticipated this scenario and includes a
0157 feature to make it slightly less nasty.
0158
0159 During #VE handling, the TDX module ensures that all interrupts (including
0160 NMIs) are blocked. The block remains in place until the guest makes a
0161 TDG.VP.VEINFO.GET TDCALL. This allows the guest to control when interrupts
0162 or a new #VE can be delivered.
0163
0164 However, the guest kernel must still be careful to avoid potential
0165 #VE-triggering actions (discussed above) while this block is in place.
0166 While the block is in place, any #VE is elevated to a double fault (#DF)
0167 which is not recoverable.
0168
0169 MMIO handling
0170 =============
0171
0172 In non-TDX VMs, MMIO is usually implemented by giving a guest access to a
0173 mapping which will cause a VMEXIT on access, and then the hypervisor
0174 emulates the access. That is not possible in TDX guests because VMEXIT
0175 will expose the register state to the host. TDX guests don't trust the host
0176 and can't have their state exposed to the host.
0177
0178 In TDX, MMIO regions typically trigger a #VE exception in the guest. The
0179 guest #VE handler then emulates the MMIO instruction inside the guest and
0180 converts it into a controlled TDCALL to the host, rather than exposing
0181 guest state to the host.
0182
0183 MMIO addresses on x86 are just special physical addresses. They can
0184 theoretically be accessed with any instruction that accesses memory.
0185 However, the kernel instruction decoding method is limited. It is only
0186 designed to decode instructions like those generated by io.h macros.
0187
0188 MMIO access via other means (like structure overlays) may result in an
0189 oops.
0190
0191 Shared Memory Conversions
0192 =========================
0193
0194 All TDX guest memory starts out as private at boot. This memory can not
0195 be accessed by the hypervisor. However, some kernel users like device
0196 drivers might have a need to share data with the hypervisor. To do this,
0197 memory must be converted between shared and private. This can be
0198 accomplished using some existing memory encryption helpers:
0199
0200 * set_memory_decrypted() converts a range of pages to shared.
0201 * set_memory_encrypted() converts memory back to private.
0202
0203 Device drivers are the primary user of shared memory, but there's no need
0204 to touch every driver. DMA buffers and ioremap() do the conversions
0205 automatically.
0206
0207 TDX uses SWIOTLB for most DMA allocations. The SWIOTLB buffer is
0208 converted to shared on boot.
0209
0210 For coherent DMA allocation, the DMA buffer gets converted on the
0211 allocation. Check force_dma_unencrypted() for details.
0212
0213 References
0214 ==========
0215
0216 TDX reference material is collected here:
0217
0218 https://www.intel.com/content/www/us/en/developer/articles/technical/intel-trust-domain-extensions.html