0001 .. SPDX-License-Identifier: GPL-2.0
0002
0003 ==============
0004 Kernel Entries
0005 ==============
0006
0007 This file documents some of the kernel entries in
0008 arch/x86/entry/entry_64.S. A lot of this explanation is adapted from
0009 an email from Ingo Molnar:
0010
0011 https://lore.kernel.org/r/20110529191055.GC9835%40elte.hu
0012
0013 The x86 architecture has quite a few different ways to jump into
0014 kernel code. Most of these entry points are registered in
0015 arch/x86/kernel/traps.c and implemented in arch/x86/entry/entry_64.S
0016 for 64-bit, arch/x86/entry/entry_32.S for 32-bit and finally
0017 arch/x86/entry/entry_64_compat.S which implements the 32-bit compatibility
0018 syscall entry points and thus provides for 32-bit processes the
0019 ability to execute syscalls when running on 64-bit kernels.
0020
0021 The IDT vector assignments are listed in arch/x86/include/asm/irq_vectors.h.
0022
0023 Some of these entries are:
0024
0025 - system_call: syscall instruction from 64-bit code.
0026
0027 - entry_INT80_compat: int 0x80 from 32-bit or 64-bit code; compat syscall
0028 either way.
0029
0030 - entry_INT80_compat, ia32_sysenter: syscall and sysenter from 32-bit
0031 code
0032
0033 - interrupt: An array of entries. Every IDT vector that doesn't
0034 explicitly point somewhere else gets set to the corresponding
0035 value in interrupts. These point to a whole array of
0036 magically-generated functions that make their way to do_IRQ with
0037 the interrupt number as a parameter.
0038
0039 - APIC interrupts: Various special-purpose interrupts for things
0040 like TLB shootdown.
0041
0042 - Architecturally-defined exceptions like divide_error.
0043
0044 There are a few complexities here. The different x86-64 entries
0045 have different calling conventions. The syscall and sysenter
0046 instructions have their own peculiar calling conventions. Some of
0047 the IDT entries push an error code onto the stack; others don't.
0048 IDT entries using the IST alternative stack mechanism need their own
0049 magic to get the stack frames right. (You can find some
0050 documentation in the AMD APM, Volume 2, Chapter 8 and the Intel SDM,
0051 Volume 3, Chapter 6.)
0052
0053 Dealing with the swapgs instruction is especially tricky. Swapgs
0054 toggles whether gs is the kernel gs or the user gs. The swapgs
0055 instruction is rather fragile: it must nest perfectly and only in
0056 single depth, it should only be used if entering from user mode to
0057 kernel mode and then when returning to user-space, and precisely
0058 so. If we mess that up even slightly, we crash.
0059
0060 So when we have a secondary entry, already in kernel mode, we *must
0061 not* use SWAPGS blindly - nor must we forget doing a SWAPGS when it's
0062 not switched/swapped yet.
0063
0064 Now, there's a secondary complication: there's a cheap way to test
0065 which mode the CPU is in and an expensive way.
0066
0067 The cheap way is to pick this info off the entry frame on the kernel
0068 stack, from the CS of the ptregs area of the kernel stack::
0069
0070 xorl %ebx,%ebx
0071 testl $3,CS+8(%rsp)
0072 je error_kernelspace
0073 SWAPGS
0074
0075 The expensive (paranoid) way is to read back the MSR_GS_BASE value
0076 (which is what SWAPGS modifies)::
0077
0078 movl $1,%ebx
0079 movl $MSR_GS_BASE,%ecx
0080 rdmsr
0081 testl %edx,%edx
0082 js 1f /* negative -> in kernel */
0083 SWAPGS
0084 xorl %ebx,%ebx
0085 1: ret
0086
0087 If we are at an interrupt or user-trap/gate-alike boundary then we can
0088 use the faster check: the stack will be a reliable indicator of
0089 whether SWAPGS was already done: if we see that we are a secondary
0090 entry interrupting kernel mode execution, then we know that the GS
0091 base has already been switched. If it says that we interrupted
0092 user-space execution then we must do the SWAPGS.
0093
0094 But if we are in an NMI/MCE/DEBUG/whatever super-atomic entry context,
0095 which might have triggered right after a normal entry wrote CS to the
0096 stack but before we executed SWAPGS, then the only safe way to check
0097 for GS is the slower method: the RDMSR.
0098
0099 Therefore, super-atomic entries (except NMI, which is handled separately)
0100 must use idtentry with paranoid=1 to handle gsbase correctly. This
0101 triggers three main behavior changes:
0102
0103 - Interrupt entry will use the slower gsbase check.
0104 - Interrupt entry from user mode will switch off the IST stack.
0105 - Interrupt exit to kernel mode will not attempt to reschedule.
0106
0107 We try to only use IST entries and the paranoid entry code for vectors
0108 that absolutely need the more expensive check for the GS base - and we
0109 generate all 'normal' entry points with the regular (faster) paranoid=0
0110 variant.
