0001 =============================
0002 Guidance for writing policies
0003 =============================
0004
0005 Try to keep transactionality out of it. The core is careful to
0006 avoid asking about anything that is migrating. This is a pain, but
0007 makes it easier to write the policies.
0008
0009 Mappings are loaded into the policy at construction time.
0010
0011 Every bio that is mapped by the target is referred to the policy.
0012 The policy can return a simple HIT or MISS or issue a migration.
0013
0014 Currently there's no way for the policy to issue background work,
0015 e.g. to start writing back dirty blocks that are going to be evicted
0016 soon.
0017
0018 Because we map bios, rather than requests it's easy for the policy
0019 to get fooled by many small bios. For this reason the core target
0020 issues periodic ticks to the policy. It's suggested that the policy
0021 doesn't update states (eg, hit counts) for a block more than once
0022 for each tick. The core ticks by watching bios complete, and so
0023 trying to see when the io scheduler has let the ios run.
0024
0025
0026 Overview of supplied cache replacement policies
0027 ===============================================
0028
0029 multiqueue (mq)
0030 ---------------
0031
0032 This policy is now an alias for smq (see below).
0033
0034 The following tunables are accepted, but have no effect::
0035
0036 'sequential_threshold <#nr_sequential_ios>'
0037 'random_threshold <#nr_random_ios>'
0038 'read_promote_adjustment <value>'
0039 'write_promote_adjustment <value>'
0040 'discard_promote_adjustment <value>'
0041
0042 Stochastic multiqueue (smq)
0043 ---------------------------
0044
0045 This policy is the default.
0046
0047 The stochastic multi-queue (smq) policy addresses some of the problems
0048 with the multiqueue (mq) policy.
0049
0050 The smq policy (vs mq) offers the promise of less memory utilization,
0051 improved performance and increased adaptability in the face of changing
0052 workloads. smq also does not have any cumbersome tuning knobs.
0053
0054 Users may switch from "mq" to "smq" simply by appropriately reloading a
0055 DM table that is using the cache target. Doing so will cause all of the
0056 mq policy's hints to be dropped. Also, performance of the cache may
0057 degrade slightly until smq recalculates the origin device's hotspots
0058 that should be cached.
0059
0060 Memory usage
0061 ^^^^^^^^^^^^
0062
0063 The mq policy used a lot of memory; 88 bytes per cache block on a 64
0064 bit machine.
0065
0066 smq uses 28bit indexes to implement its data structures rather than
0067 pointers. It avoids storing an explicit hit count for each block. It
0068 has a 'hotspot' queue, rather than a pre-cache, which uses a quarter of
0069 the entries (each hotspot block covers a larger area than a single
0070 cache block).
0071
0072 All this means smq uses ~25bytes per cache block. Still a lot of
0073 memory, but a substantial improvement nontheless.
0074
0075 Level balancing
0076 ^^^^^^^^^^^^^^^
0077
0078 mq placed entries in different levels of the multiqueue structures
0079 based on their hit count (~ln(hit count)). This meant the bottom
0080 levels generally had the most entries, and the top ones had very
0081 few. Having unbalanced levels like this reduced the efficacy of the
0082 multiqueue.
0083
0084 smq does not maintain a hit count, instead it swaps hit entries with
0085 the least recently used entry from the level above. The overall
0086 ordering being a side effect of this stochastic process. With this
0087 scheme we can decide how many entries occupy each multiqueue level,
0088 resulting in better promotion/demotion decisions.
0089
0090 Adaptability:
0091 The mq policy maintained a hit count for each cache block. For a
0092 different block to get promoted to the cache its hit count has to
0093 exceed the lowest currently in the cache. This meant it could take a
0094 long time for the cache to adapt between varying IO patterns.
0095
0096 smq doesn't maintain hit counts, so a lot of this problem just goes
0097 away. In addition it tracks performance of the hotspot queue, which
0098 is used to decide which blocks to promote. If the hotspot queue is
0099 performing badly then it starts moving entries more quickly between
0100 levels. This lets it adapt to new IO patterns very quickly.
0101
0102 Performance
0103 ^^^^^^^^^^^
0104
0105 Testing smq shows substantially better performance than mq.
0106
0107 cleaner
0108 -------
0109
0110 The cleaner writes back all dirty blocks in a cache to decommission it.
0111
0112 Examples
0113 ========
0114
0115 The syntax for a table is::
0116
0117 cache <metadata dev> <cache dev> <origin dev> <block size>
0118 <#feature_args> [<feature arg>]*
0119 <policy> <#policy_args> [<policy arg>]*
0120
0121 The syntax to send a message using the dmsetup command is::
0122
0123 dmsetup message <mapped device> 0 sequential_threshold 1024
0124 dmsetup message <mapped device> 0 random_threshold 8
0125
0126 Using dmsetup::
0127
0128 dmsetup create blah --table "0 268435456 cache /dev/sdb /dev/sdc \
0129 /dev/sdd 512 0 mq 4 sequential_threshold 1024 random_threshold 8"
0130 creates a 128GB large mapped device named 'blah' with the
0131 sequential threshold set to 1024 and the random_threshold set to 8.
