0001 .. SPDX-License-Identifier: GPL-2.0
0002
0003 =======================
0004 DAMON-based Reclamation
0005 =======================
0006
0007 DAMON-based Reclamation (DAMON_RECLAIM) is a static kernel module that aimed to
0008 be used for proactive and lightweight reclamation under light memory pressure.
0009 It doesn't aim to replace the LRU-list based page_granularity reclamation, but
0010 to be selectively used for different level of memory pressure and requirements.
0011
0012 Where Proactive Reclamation is Required?
0013 ========================================
0014
0015 On general memory over-committed systems, proactively reclaiming cold pages
0016 helps saving memory and reducing latency spikes that incurred by the direct
0017 reclaim of the process or CPU consumption of kswapd, while incurring only
0018 minimal performance degradation [1]_ [2]_ .
0019
0020 Free Pages Reporting [3]_ based memory over-commit virtualization systems are
0021 good example of the cases. In such systems, the guest VMs reports their free
0022 memory to host, and the host reallocates the reported memory to other guests.
0023 As a result, the memory of the systems are fully utilized. However, the
0024 guests could be not so memory-frugal, mainly because some kernel subsystems and
0025 user-space applications are designed to use as much memory as available. Then,
0026 guests could report only small amount of memory as free to host, results in
0027 memory utilization drop of the systems. Running the proactive reclamation in
0028 guests could mitigate this problem.
0029
0030 How It Works?
0031 =============
0032
0033 DAMON_RECLAIM finds memory regions that didn't accessed for specific time
0034 duration and page out. To avoid it consuming too much CPU for the paging out
0035 operation, a speed limit can be configured. Under the speed limit, it pages
0036 out memory regions that didn't accessed longer time first. System
0037 administrators can also configure under what situation this scheme should
0038 automatically activated and deactivated with three memory pressure watermarks.
0039
0040 Interface: Module Parameters
0041 ============================
0042
0043 To use this feature, you should first ensure your system is running on a kernel
0044 that is built with ``CONFIG_DAMON_RECLAIM=y``.
0045
0046 To let sysadmins enable or disable it and tune for the given system,
0047 DAMON_RECLAIM utilizes module parameters. That is, you can put
0048 ``damon_reclaim.<parameter>=<value>`` on the kernel boot command line or write
0049 proper values to ``/sys/modules/damon_reclaim/parameters/<parameter>`` files.
0050
0051 Below are the description of each parameter.
0052
0053 enabled
0054 -------
0055
0056 Enable or disable DAMON_RECLAIM.
0057
0058 You can enable DAMON_RCLAIM by setting the value of this parameter as ``Y``.
0059 Setting it as ``N`` disables DAMON_RECLAIM. Note that DAMON_RECLAIM could do
0060 no real monitoring and reclamation due to the watermarks-based activation
0061 condition. Refer to below descriptions for the watermarks parameter for this.
0062
0063 commit_inputs
0064 -------------
0065
0066 Make DAMON_RECLAIM reads the input parameters again, except ``enabled``.
0067
0068 Input parameters that updated while DAMON_RECLAIM is running are not applied
0069 by default. Once this parameter is set as ``Y``, DAMON_RECLAIM reads values
0070 of parametrs except ``enabled`` again. Once the re-reading is done, this
0071 parameter is set as ``N``. If invalid parameters are found while the
0072 re-reading, DAMON_RECLAIM will be disabled.
0073
0074 min_age
0075 -------
0076
0077 Time threshold for cold memory regions identification in microseconds.
0078
0079 If a memory region is not accessed for this or longer time, DAMON_RECLAIM
0080 identifies the region as cold, and reclaims it.
0081
0082 120 seconds by default.
0083
0084 quota_ms
0085 --------
0086
0087 Limit of time for the reclamation in milliseconds.
0088
0089 DAMON_RECLAIM tries to use only up to this time within a time window
0090 (quota_reset_interval_ms) for trying reclamation of cold pages. This can be
0091 used for limiting CPU consumption of DAMON_RECLAIM. If the value is zero, the
0092 limit is disabled.
0093
0094 10 ms by default.
0095
0096 quota_sz
0097 --------
0098
0099 Limit of size of memory for the reclamation in bytes.
0100
0101 DAMON_RECLAIM charges amount of memory which it tried to reclaim within a time
0102 window (quota_reset_interval_ms) and makes no more than this limit is tried.
0103 This can be used for limiting consumption of CPU and IO. If this value is
0104 zero, the limit is disabled.
0105
0106 128 MiB by default.
0107
0108 quota_reset_interval_ms
0109 -----------------------
0110
0111 The time/size quota charge reset interval in milliseconds.
0112
0113 The charget reset interval for the quota of time (quota_ms) and size
0114 (quota_sz). That is, DAMON_RECLAIM does not try reclamation for more than
0115 quota_ms milliseconds or quota_sz bytes within quota_reset_interval_ms
0116 milliseconds.
0117
0118 1 second by default.
0119
0120 wmarks_interval
0121 ---------------
0122
0123 Minimal time to wait before checking the watermarks, when DAMON_RECLAIM is
0124 enabled but inactive due to its watermarks rule.
0125
0126 wmarks_high
0127 -----------
0128
0129 Free memory rate (per thousand) for the high watermark.
0130
0131 If free memory of the system in bytes per thousand bytes is higher than this,
0132 DAMON_RECLAIM becomes inactive, so it does nothing but only periodically checks
0133 the watermarks.
0134
0135 wmarks_mid
0136 ----------
0137
0138 Free memory rate (per thousand) for the middle watermark.
0139
0140 If free memory of the system in bytes per thousand bytes is between this and
0141 the low watermark, DAMON_RECLAIM becomes active, so starts the monitoring and
0142 the reclaiming.
0143
0144 wmarks_low
0145 ----------
0146
0147 Free memory rate (per thousand) for the low watermark.
0148
0149 If free memory of the system in bytes per thousand bytes is lower than this,
0150 DAMON_RECLAIM becomes inactive, so it does nothing but periodically checks the
0151 watermarks. In the case, the system falls back to the LRU-list based page
0152 granularity reclamation logic.
0153
0154 sample_interval
0155 ---------------
0156
0157 Sampling interval for the monitoring in microseconds.
0158
0159 The sampling interval of DAMON for the cold memory monitoring. Please refer to
0160 the DAMON documentation (:doc:`usage`) for more detail.
0161
0162 aggr_interval
0163 -------------
0164
0165 Aggregation interval for the monitoring in microseconds.
0166
0167 The aggregation interval of DAMON for the cold memory monitoring. Please
0168 refer to the DAMON documentation (:doc:`usage`) for more detail.
0169
0170 min_nr_regions
0171 --------------
0172
0173 Minimum number of monitoring regions.
0174
0175 The minimal number of monitoring regions of DAMON for the cold memory
0176 monitoring. This can be used to set lower-bound of the monitoring quality.
0177 But, setting this too high could result in increased monitoring overhead.
0178 Please refer to the DAMON documentation (:doc:`usage`) for more detail.
0179
0180 max_nr_regions
0181 --------------
0182
0183 Maximum number of monitoring regions.
0184
0185 The maximum number of monitoring regions of DAMON for the cold memory
0186 monitoring. This can be used to set upper-bound of the monitoring overhead.
0187 However, setting this too low could result in bad monitoring quality. Please
0188 refer to the DAMON documentation (:doc:`usage`) for more detail.
0189
0190 monitor_region_start
0191 --------------------
0192
0193 Start of target memory region in physical address.
0194
0195 The start physical address of memory region that DAMON_RECLAIM will do work
0196 against. That is, DAMON_RECLAIM will find cold memory regions in this region
0197 and reclaims. By default, biggest System RAM is used as the region.
0198
0199 monitor_region_end
0200 ------------------
0201
0202 End of target memory region in physical address.
0203
0204 The end physical address of memory region that DAMON_RECLAIM will do work
0205 against. That is, DAMON_RECLAIM will find cold memory regions in this region
0206 and reclaims. By default, biggest System RAM is used as the region.
0207
0208 kdamond_pid
0209 -----------
0210
0211 PID of the DAMON thread.
0212
0213 If DAMON_RECLAIM is enabled, this becomes the PID of the worker thread. Else,
0214 -1.
0215
0216 nr_reclaim_tried_regions
0217 ------------------------
0218
0219 Number of memory regions that tried to be reclaimed by DAMON_RECLAIM.
0220
0221 bytes_reclaim_tried_regions
0222 ---------------------------
0223
0224 Total bytes of memory regions that tried to be reclaimed by DAMON_RECLAIM.
0225
0226 nr_reclaimed_regions
0227 --------------------
0228
0229 Number of memory regions that successfully be reclaimed by DAMON_RECLAIM.
0230
0231 bytes_reclaimed_regions
0232 -----------------------
0233
0234 Total bytes of memory regions that successfully be reclaimed by DAMON_RECLAIM.
0235
0236 nr_quota_exceeds
0237 ----------------
0238
0239 Number of times that the time/space quota limits have exceeded.
0240
0241 Example
0242 =======
0243
0244 Below runtime example commands make DAMON_RECLAIM to find memory regions that
0245 not accessed for 30 seconds or more and pages out. The reclamation is limited
0246 to be done only up to 1 GiB per second to avoid DAMON_RECLAIM consuming too
0247 much CPU time for the paging out operation. It also asks DAMON_RECLAIM to do
0248 nothing if the system's free memory rate is more than 50%, but start the real
0249 works if it becomes lower than 40%. If DAMON_RECLAIM doesn't make progress and
0250 therefore the free memory rate becomes lower than 20%, it asks DAMON_RECLAIM to
0251 do nothing again, so that we can fall back to the LRU-list based page
0252 granularity reclamation. ::
0253
0254 # cd /sys/modules/damon_reclaim/parameters
0255 # echo 30000000 > min_age
0256 # echo $((1 * 1024 * 1024 * 1024)) > quota_sz
0257 # echo 1000 > quota_reset_interval_ms
0258 # echo 500 > wmarks_high
0259 # echo 400 > wmarks_mid
0260 # echo 200 > wmarks_low
0261 # echo Y > enabled
0262
0263 .. [1] https://research.google/pubs/pub48551/
0264 .. [2] https://lwn.net/Articles/787611/
0265 .. [3] https://www.kernel.org/doc/html/latest/mm/free_page_reporting.html
