0001 .. SPDX-License-Identifier: GPL-2.0
0002 .. include:: <isonum.txt>
0003
0004 =========================
0005 System Suspend Code Flows
0006 =========================
0007
0008 :Copyright: |copy| 2020 Intel Corporation
0009
0010 :Author: Rafael J. Wysocki <rafael.j.wysocki@intel.com>
0011
0012 At least one global system-wide transition needs to be carried out for the
0013 system to get from the working state into one of the supported
0014 :doc:`sleep states <sleep-states>`. Hibernation requires more than one
0015 transition to occur for this purpose, but the other sleep states, commonly
0016 referred to as *system-wide suspend* (or simply *system suspend*) states, need
0017 only one.
0018
0019 For those sleep states, the transition from the working state of the system into
0020 the target sleep state is referred to as *system suspend* too (in the majority
0021 of cases, whether this means a transition or a sleep state of the system should
0022 be clear from the context) and the transition back from the sleep state into the
0023 working state is referred to as *system resume*.
0024
0025 The kernel code flows associated with the suspend and resume transitions for
0026 different sleep states of the system are quite similar, but there are some
0027 significant differences between the :ref:`suspend-to-idle <s2idle>` code flows
0028 and the code flows related to the :ref:`suspend-to-RAM <s2ram>` and
0029 :ref:`standby <standby>` sleep states.
0030
0031 The :ref:`suspend-to-RAM <s2ram>` and :ref:`standby <standby>` sleep states
0032 cannot be implemented without platform support and the difference between them
0033 boils down to the platform-specific actions carried out by the suspend and
0034 resume hooks that need to be provided by the platform driver to make them
0035 available. Apart from that, the suspend and resume code flows for these sleep
0036 states are mostly identical, so they both together will be referred to as
0037 *platform-dependent suspend* states in what follows.
0038
0039
0040 .. _s2idle_suspend:
0041
0042 Suspend-to-idle Suspend Code Flow
0043 =================================
0044
0045 The following steps are taken in order to transition the system from the working
0046 state to the :ref:`suspend-to-idle <s2idle>` sleep state:
0047
0048 1. Invoking system-wide suspend notifiers.
0049
0050 Kernel subsystems can register callbacks to be invoked when the suspend
0051 transition is about to occur and when the resume transition has finished.
0052
0053 That allows them to prepare for the change of the system state and to clean
0054 up after getting back to the working state.
0055
0056 2. Freezing tasks.
0057
0058 Tasks are frozen primarily in order to avoid unchecked hardware accesses
0059 from user space through MMIO regions or I/O registers exposed directly to
0060 it and to prevent user space from entering the kernel while the next step
0061 of the transition is in progress (which might have been problematic for
0062 various reasons).
0063
0064 All user space tasks are intercepted as though they were sent a signal and
0065 put into uninterruptible sleep until the end of the subsequent system resume
0066 transition.
0067
0068 The kernel threads that choose to be frozen during system suspend for
0069 specific reasons are frozen subsequently, but they are not intercepted.
0070 Instead, they are expected to periodically check whether or not they need
0071 to be frozen and to put themselves into uninterruptible sleep if so. [Note,
0072 however, that kernel threads can use locking and other concurrency controls
0073 available in kernel space to synchronize themselves with system suspend and
0074 resume, which can be much more precise than the freezing, so the latter is
0075 not a recommended option for kernel threads.]
0076
0077 3. Suspending devices and reconfiguring IRQs.
0078
0079 Devices are suspended in four phases called *prepare*, *suspend*,
0080 *late suspend* and *noirq suspend* (see :ref:`driverapi_pm_devices` for more
0081 information on what exactly happens in each phase).
0082
0083 Every device is visited in each phase, but typically it is not physically
0084 accessed in more than two of them.
0085
0086 The runtime PM API is disabled for every device during the *late* suspend
0087 phase and high-level ("action") interrupt handlers are prevented from being
0088 invoked before the *noirq* suspend phase.
0089
0090 Interrupts are still handled after that, but they are only acknowledged to
0091 interrupt controllers without performing any device-specific actions that
0092 would be triggered in the working state of the system (those actions are
0093 deferred till the subsequent system resume transition as described
0094 `below <s2idle_resume_>`_).
0095
0096 IRQs associated with system wakeup devices are "armed" so that the resume
0097 transition of the system is started when one of them signals an event.
0098
0099 4. Freezing the scheduler tick and suspending timekeeping.
0100
0101 When all devices have been suspended, CPUs enter the idle loop and are put
0102 into the deepest available idle state. While doing that, each of them
0103 "freezes" its own scheduler tick so that the timer events associated with
0104 the tick do not occur until the CPU is woken up by another interrupt source.
0105
0106 The last CPU to enter the idle state also stops the timekeeping which
0107 (among other things) prevents high resolution timers from triggering going
0108 forward until the first CPU that is woken up restarts the timekeeping.
0109 That allows the CPUs to stay in the deep idle state relatively long in one
0110 go.
0111
0112 From this point on, the CPUs can only be woken up by non-timer hardware
0113 interrupts. If that happens, they go back to the idle state unless the
0114 interrupt that woke up one of them comes from an IRQ that has been armed for
0115 system wakeup, in which case the system resume transition is started.
0116
0117
0118 .. _s2idle_resume:
0119
0120 Suspend-to-idle Resume Code Flow
0121 ================================
0122
0123 The following steps are taken in order to transition the system from the
0124 :ref:`suspend-to-idle <s2idle>` sleep state into the working state:
0125
0126 1. Resuming timekeeping and unfreezing the scheduler tick.
0127
0128 When one of the CPUs is woken up (by a non-timer hardware interrupt), it
0129 leaves the idle state entered in the last step of the preceding suspend
0130 transition, restarts the timekeeping (unless it has been restarted already
0131 by another CPU that woke up earlier) and the scheduler tick on that CPU is
0132 unfrozen.
0133
0134 If the interrupt that has woken up the CPU was armed for system wakeup,
0135 the system resume transition begins.
0136
0137 2. Resuming devices and restoring the working-state configuration of IRQs.
0138
0139 Devices are resumed in four phases called *noirq resume*, *early resume*,
0140 *resume* and *complete* (see :ref:`driverapi_pm_devices` for more
0141 information on what exactly happens in each phase).
0142
0143 Every device is visited in each phase, but typically it is not physically
0144 accessed in more than two of them.
0145
0146 The working-state configuration of IRQs is restored after the *noirq* resume
0147 phase and the runtime PM API is re-enabled for every device whose driver
0148 supports it during the *early* resume phase.
0149
0150 3. Thawing tasks.
0151
0152 Tasks frozen in step 2 of the preceding `suspend <s2idle_suspend_>`_
0153 transition are "thawed", which means that they are woken up from the
0154 uninterruptible sleep that they went into at that time and user space tasks
0155 are allowed to exit the kernel.
0156
0157 4. Invoking system-wide resume notifiers.
0158
0159 This is analogous to step 1 of the `suspend <s2idle_suspend_>`_ transition
0160 and the same set of callbacks is invoked at this point, but a different
0161 "notification type" parameter value is passed to them.
0162
0163
0164 Platform-dependent Suspend Code Flow
0165 ====================================
0166
0167 The following steps are taken in order to transition the system from the working
0168 state to platform-dependent suspend state:
0169
0170 1. Invoking system-wide suspend notifiers.
0171
0172 This step is the same as step 1 of the suspend-to-idle suspend transition
0173 described `above <s2idle_suspend_>`_.
0174
0175 2. Freezing tasks.
0176
0177 This step is the same as step 2 of the suspend-to-idle suspend transition
0178 described `above <s2idle_suspend_>`_.
0179
0180 3. Suspending devices and reconfiguring IRQs.
0181
0182 This step is analogous to step 3 of the suspend-to-idle suspend transition
0183 described `above <s2idle_suspend_>`_, but the arming of IRQs for system
0184 wakeup generally does not have any effect on the platform.
0185
0186 There are platforms that can go into a very deep low-power state internally
0187 when all CPUs in them are in sufficiently deep idle states and all I/O
0188 devices have been put into low-power states. On those platforms,
0189 suspend-to-idle can reduce system power very effectively.
0190
0191 On the other platforms, however, low-level components (like interrupt
0192 controllers) need to be turned off in a platform-specific way (implemented
0193 in the hooks provided by the platform driver) to achieve comparable power
0194 reduction.
0195
0196 That usually prevents in-band hardware interrupts from waking up the system,
0197 which must be done in a special platform-dependent way. Then, the
0198 configuration of system wakeup sources usually starts when system wakeup
0199 devices are suspended and is finalized by the platform suspend hooks later
0200 on.
0201
0202 4. Disabling non-boot CPUs.
0203
0204 On some platforms the suspend hooks mentioned above must run in a one-CPU
0205 configuration of the system (in particular, the hardware cannot be accessed
0206 by any code running in parallel with the platform suspend hooks that may,
0207 and often do, trap into the platform firmware in order to finalize the
0208 suspend transition).
0209
0210 For this reason, the CPU offline/online (CPU hotplug) framework is used
0211 to take all of the CPUs in the system, except for one (the boot CPU),
0212 offline (typically, the CPUs that have been taken offline go into deep idle
0213 states).
0214
0215 This means that all tasks are migrated away from those CPUs and all IRQs are
0216 rerouted to the only CPU that remains online.
0217
0218 5. Suspending core system components.
0219
0220 This prepares the core system components for (possibly) losing power going
0221 forward and suspends the timekeeping.
0222
0223 6. Platform-specific power removal.
0224
0225 This is expected to remove power from all of the system components except
0226 for the memory controller and RAM (in order to preserve the contents of the
0227 latter) and some devices designated for system wakeup.
0228
0229 In many cases control is passed to the platform firmware which is expected
0230 to finalize the suspend transition as needed.
0231
0232
0233 Platform-dependent Resume Code Flow
0234 ===================================
0235
0236 The following steps are taken in order to transition the system from a
0237 platform-dependent suspend state into the working state:
0238
0239 1. Platform-specific system wakeup.
0240
0241 The platform is woken up by a signal from one of the designated system
0242 wakeup devices (which need not be an in-band hardware interrupt) and
0243 control is passed back to the kernel (the working configuration of the
0244 platform may need to be restored by the platform firmware before the
0245 kernel gets control again).
0246
0247 2. Resuming core system components.
0248
0249 The suspend-time configuration of the core system components is restored and
0250 the timekeeping is resumed.
0251
0252 3. Re-enabling non-boot CPUs.
0253
0254 The CPUs disabled in step 4 of the preceding suspend transition are taken
0255 back online and their suspend-time configuration is restored.
0256
0257 4. Resuming devices and restoring the working-state configuration of IRQs.
0258
0259 This step is the same as step 2 of the suspend-to-idle suspend transition
0260 described `above <s2idle_resume_>`_.
0261
0262 5. Thawing tasks.
0263
0264 This step is the same as step 3 of the suspend-to-idle suspend transition
0265 described `above <s2idle_resume_>`_.
0266
0267 6. Invoking system-wide resume notifiers.
0268
0269 This step is the same as step 4 of the suspend-to-idle suspend transition
0270 described `above <s2idle_resume_>`_.
