Back to home page

LXR

 
 

    


0001 Say you've got a big slow raid 6, and an ssd or three. Wouldn't it be
0002 nice if you could use them as cache... Hence bcache.
0003 
0004 Wiki and git repositories are at:
0005   http://bcache.evilpiepirate.org
0006   http://evilpiepirate.org/git/linux-bcache.git
0007   http://evilpiepirate.org/git/bcache-tools.git
0008 
0009 It's designed around the performance characteristics of SSDs - it only allocates
0010 in erase block sized buckets, and it uses a hybrid btree/log to track cached
0011 extents (which can be anywhere from a single sector to the bucket size). It's
0012 designed to avoid random writes at all costs; it fills up an erase block
0013 sequentially, then issues a discard before reusing it.
0014 
0015 Both writethrough and writeback caching are supported. Writeback defaults to
0016 off, but can be switched on and off arbitrarily at runtime. Bcache goes to
0017 great lengths to protect your data - it reliably handles unclean shutdown. (It
0018 doesn't even have a notion of a clean shutdown; bcache simply doesn't return
0019 writes as completed until they're on stable storage).
0020 
0021 Writeback caching can use most of the cache for buffering writes - writing
0022 dirty data to the backing device is always done sequentially, scanning from the
0023 start to the end of the index.
0024 
0025 Since random IO is what SSDs excel at, there generally won't be much benefit
0026 to caching large sequential IO. Bcache detects sequential IO and skips it;
0027 it also keeps a rolling average of the IO sizes per task, and as long as the
0028 average is above the cutoff it will skip all IO from that task - instead of
0029 caching the first 512k after every seek. Backups and large file copies should
0030 thus entirely bypass the cache.
0031 
0032 In the event of a data IO error on the flash it will try to recover by reading
0033 from disk or invalidating cache entries.  For unrecoverable errors (meta data
0034 or dirty data), caching is automatically disabled; if dirty data was present
0035 in the cache it first disables writeback caching and waits for all dirty data
0036 to be flushed.
0037 
0038 Getting started:
0039 You'll need make-bcache from the bcache-tools repository. Both the cache device
0040 and backing device must be formatted before use.
0041   make-bcache -B /dev/sdb
0042   make-bcache -C /dev/sdc
0043 
0044 make-bcache has the ability to format multiple devices at the same time - if
0045 you format your backing devices and cache device at the same time, you won't
0046 have to manually attach:
0047   make-bcache -B /dev/sda /dev/sdb -C /dev/sdc
0048 
0049 bcache-tools now ships udev rules, and bcache devices are known to the kernel
0050 immediately.  Without udev, you can manually register devices like this:
0051 
0052   echo /dev/sdb > /sys/fs/bcache/register
0053   echo /dev/sdc > /sys/fs/bcache/register
0054 
0055 Registering the backing device makes the bcache device show up in /dev; you can
0056 now format it and use it as normal. But the first time using a new bcache
0057 device, it'll be running in passthrough mode until you attach it to a cache.
0058 If you are thinking about using bcache later, it is recommended to setup all your
0059 slow devices as bcache backing devices without a cache, and you can choose to add
0060 a caching device later.
0061 See 'ATTACHING' section below.
0062 
0063 The devices show up as:
0064 
0065   /dev/bcache<N>
0066 
0067 As well as (with udev):
0068 
0069   /dev/bcache/by-uuid/<uuid>
0070   /dev/bcache/by-label/<label>
0071 
0072 To get started:
0073 
0074   mkfs.ext4 /dev/bcache0
0075   mount /dev/bcache0 /mnt
0076 
0077 You can control bcache devices through sysfs at /sys/block/bcache<N>/bcache .
0078 You can also control them through /sys/fs//bcache/<cset-uuid>/ .
0079 
0080 Cache devices are managed as sets; multiple caches per set isn't supported yet
0081 but will allow for mirroring of metadata and dirty data in the future. Your new
0082 cache set shows up as /sys/fs/bcache/<UUID>
0083 
0084 ATTACHING
0085 ---------
0086 
0087 After your cache device and backing device are registered, the backing device
0088 must be attached to your cache set to enable caching. Attaching a backing
0089 device to a cache set is done thusly, with the UUID of the cache set in
0090 /sys/fs/bcache:
0091 
0092   echo <CSET-UUID> > /sys/block/bcache0/bcache/attach
0093 
0094 This only has to be done once. The next time you reboot, just reregister all
0095 your bcache devices. If a backing device has data in a cache somewhere, the
0096 /dev/bcache<N> device won't be created until the cache shows up - particularly
0097 important if you have writeback caching turned on.
0098 
0099 If you're booting up and your cache device is gone and never coming back, you
0100 can force run the backing device:
0101 
0102   echo 1 > /sys/block/sdb/bcache/running
0103 
0104 (You need to use /sys/block/sdb (or whatever your backing device is called), not
0105 /sys/block/bcache0, because bcache0 doesn't exist yet. If you're using a
0106 partition, the bcache directory would be at /sys/block/sdb/sdb2/bcache)
0107 
0108 The backing device will still use that cache set if it shows up in the future,
0109 but all the cached data will be invalidated. If there was dirty data in the
0110 cache, don't expect the filesystem to be recoverable - you will have massive
0111 filesystem corruption, though ext4's fsck does work miracles.
0112 
0113 ERROR HANDLING
0114 --------------
0115 
0116 Bcache tries to transparently handle IO errors to/from the cache device without
0117 affecting normal operation; if it sees too many errors (the threshold is
0118 configurable, and defaults to 0) it shuts down the cache device and switches all
0119 the backing devices to passthrough mode.
0120 
0121  - For reads from the cache, if they error we just retry the read from the
0122    backing device.
0123 
0124  - For writethrough writes, if the write to the cache errors we just switch to
0125    invalidating the data at that lba in the cache (i.e. the same thing we do for
0126    a write that bypasses the cache)
0127 
0128  - For writeback writes, we currently pass that error back up to the
0129    filesystem/userspace. This could be improved - we could retry it as a write
0130    that skips the cache so we don't have to error the write.
0131 
0132  - When we detach, we first try to flush any dirty data (if we were running in
0133    writeback mode). It currently doesn't do anything intelligent if it fails to
0134    read some of the dirty data, though.
0135 
0136 
0137 HOWTO/COOKBOOK
0138 --------------
0139 
0140 A) Starting a bcache with a missing caching device
0141 
0142 If registering the backing device doesn't help, it's already there, you just need
0143 to force it to run without the cache:
0144         host:~# echo /dev/sdb1 > /sys/fs/bcache/register
0145         [  119.844831] bcache: register_bcache() error opening /dev/sdb1: device already registered
0146 
0147 Next, you try to register your caching device if it's present. However
0148 if it's absent, or registration fails for some reason, you can still
0149 start your bcache without its cache, like so:
0150         host:/sys/block/sdb/sdb1/bcache# echo 1 > running
0151 
0152 Note that this may cause data loss if you were running in writeback mode.
0153 
0154 
0155 B) Bcache does not find its cache
0156 
0157         host:/sys/block/md5/bcache# echo 0226553a-37cf-41d5-b3ce-8b1e944543a8 > attach
0158         [ 1933.455082] bcache: bch_cached_dev_attach() Couldn't find uuid for md5 in set
0159         [ 1933.478179] bcache: __cached_dev_store() Can't attach 0226553a-37cf-41d5-b3ce-8b1e944543a8
0160         [ 1933.478179] : cache set not found
0161 
0162 In this case, the caching device was simply not registered at boot
0163 or disappeared and came back, and needs to be (re-)registered:
0164         host:/sys/block/md5/bcache# echo /dev/sdh2 > /sys/fs/bcache/register
0165 
0166 
0167 C) Corrupt bcache crashes the kernel at device registration time:
0168 
0169 This should never happen.  If it does happen, then you have found a bug!
0170 Please report it to the bcache development list: linux-bcache@vger.kernel.org
0171 
0172 Be sure to provide as much information that you can including kernel dmesg
0173 output if available so that we may assist.
0174 
0175 
0176 D) Recovering data without bcache:
0177 
0178 If bcache is not available in the kernel, a filesystem on the backing
0179 device is still available at an 8KiB offset. So either via a loopdev
0180 of the backing device created with --offset 8K, or any value defined by
0181 --data-offset when you originally formatted bcache with `make-bcache`.
0182 
0183 For example:
0184         losetup -o 8192 /dev/loop0 /dev/your_bcache_backing_dev
0185 
0186 This should present your unmodified backing device data in /dev/loop0
0187 
0188 If your cache is in writethrough mode, then you can safely discard the
0189 cache device without loosing data.
0190 
0191 
0192 E) Wiping a cache device
0193 
0194 host:~# wipefs -a /dev/sdh2
0195 16 bytes were erased at offset 0x1018 (bcache)
0196 they were: c6 85 73 f6 4e 1a 45 ca 82 65 f5 7f 48 ba 6d 81
0197 
0198 After you boot back with bcache enabled, you recreate the cache and attach it:
0199 host:~# make-bcache -C /dev/sdh2
0200 UUID:                   7be7e175-8f4c-4f99-94b2-9c904d227045
0201 Set UUID:               5bc072a8-ab17-446d-9744-e247949913c1
0202 version:                0
0203 nbuckets:               106874
0204 block_size:             1
0205 bucket_size:            1024
0206 nr_in_set:              1
0207 nr_this_dev:            0
0208 first_bucket:           1
0209 [  650.511912] bcache: run_cache_set() invalidating existing data
0210 [  650.549228] bcache: register_cache() registered cache device sdh2
0211 
0212 start backing device with missing cache:
0213 host:/sys/block/md5/bcache# echo 1 > running
0214 
0215 attach new cache:
0216 host:/sys/block/md5/bcache# echo 5bc072a8-ab17-446d-9744-e247949913c1 > attach
0217 [  865.276616] bcache: bch_cached_dev_attach() Caching md5 as bcache0 on set 5bc072a8-ab17-446d-9744-e247949913c1
0218 
0219 
0220 F) Remove or replace a caching device
0221 
0222         host:/sys/block/sda/sda7/bcache# echo 1 > detach
0223         [  695.872542] bcache: cached_dev_detach_finish() Caching disabled for sda7
0224 
0225         host:~# wipefs -a /dev/nvme0n1p4
0226         wipefs: error: /dev/nvme0n1p4: probing initialization failed: Device or resource busy
0227         Ooops, it's disabled, but not unregistered, so it's still protected
0228 
0229 We need to go and unregister it:
0230         host:/sys/fs/bcache/b7ba27a1-2398-4649-8ae3-0959f57ba128# ls -l cache0
0231         lrwxrwxrwx 1 root root 0 Feb 25 18:33 cache0 -> ../../../devices/pci0000:00/0000:00:1d.0/0000:70:00.0/nvme/nvme0/nvme0n1/nvme0n1p4/bcache/
0232         host:/sys/fs/bcache/b7ba27a1-2398-4649-8ae3-0959f57ba128# echo 1 > stop
0233         kernel: [  917.041908] bcache: cache_set_free() Cache set b7ba27a1-2398-4649-8ae3-0959f57ba128 unregistered
0234 
0235 Now we can wipe it:
0236         host:~# wipefs -a /dev/nvme0n1p4
0237         /dev/nvme0n1p4: 16 bytes were erased at offset 0x00001018 (bcache): c6 85 73 f6 4e 1a 45 ca 82 65 f5 7f 48 ba 6d 81
0238 
0239 
0240 G) dm-crypt and bcache
0241 
0242 First setup bcache unencrypted and then install dmcrypt on top of
0243 /dev/bcache<N> This will work faster than if you dmcrypt both the backing
0244 and caching devices and then install bcache on top. [benchmarks?]
0245 
0246 
0247 H) Stop/free a registered bcache to wipe and/or recreate it
0248 
0249 Suppose that you need to free up all bcache references so that you can
0250 fdisk run and re-register a changed partition table, which won't work
0251 if there are any active backing or caching devices left on it:
0252 
0253 1) Is it present in /dev/bcache* ? (there are times where it won't be)
0254 
0255 If so, it's easy:
0256         host:/sys/block/bcache0/bcache# echo 1 > stop
0257 
0258 2) But if your backing device is gone, this won't work:
0259         host:/sys/block/bcache0# cd bcache
0260         bash: cd: bcache: No such file or directory
0261 
0262 In this case, you may have to unregister the dmcrypt block device that
0263 references this bcache to free it up:
0264         host:~# dmsetup remove oldds1
0265         bcache: bcache_device_free() bcache0 stopped
0266         bcache: cache_set_free() Cache set 5bc072a8-ab17-446d-9744-e247949913c1 unregistered
0267 
0268 This causes the backing bcache to be removed from /sys/fs/bcache and
0269 then it can be reused.  This would be true of any block device stacking
0270 where bcache is a lower device.
0271 
0272 3) In other cases, you can also look in /sys/fs/bcache/:
0273 
0274 host:/sys/fs/bcache# ls -l */{cache?,bdev?}
0275 lrwxrwxrwx 1 root root 0 Mar  5 09:39 0226553a-37cf-41d5-b3ce-8b1e944543a8/bdev1 -> ../../../devices/virtual/block/dm-1/bcache/
0276 lrwxrwxrwx 1 root root 0 Mar  5 09:39 0226553a-37cf-41d5-b3ce-8b1e944543a8/cache0 -> ../../../devices/virtual/block/dm-4/bcache/
0277 lrwxrwxrwx 1 root root 0 Mar  5 09:39 5bc072a8-ab17-446d-9744-e247949913c1/cache0 -> ../../../devices/pci0000:00/0000:00:01.0/0000:01:00.0/ata10/host9/target9:0:0/9:0:0:0/block/sdl/sdl2/bcache/
0278 
0279 The device names will show which UUID is relevant, cd in that directory
0280 and stop the cache:
0281         host:/sys/fs/bcache/5bc072a8-ab17-446d-9744-e247949913c1# echo 1 > stop
0282 
0283 This will free up bcache references and let you reuse the partition for
0284 other purposes.
0285 
0286 
0287 
0288 TROUBLESHOOTING PERFORMANCE
0289 ---------------------------
0290 
0291 Bcache has a bunch of config options and tunables. The defaults are intended to
0292 be reasonable for typical desktop and server workloads, but they're not what you
0293 want for getting the best possible numbers when benchmarking.
0294 
0295  - Backing device alignment
0296 
0297    The default metadata size in bcache is 8k.  If your backing device is
0298    RAID based, then be sure to align this by a multiple of your stride
0299    width using `make-bcache --data-offset`. If you intend to expand your
0300    disk array in the future, then multiply a series of primes by your
0301    raid stripe size to get the disk multiples that you would like.
0302 
0303    For example:  If you have a 64k stripe size, then the following offset
0304    would provide alignment for many common RAID5 data spindle counts:
0305         64k * 2*2*2*3*3*5*7 bytes = 161280k
0306 
0307    That space is wasted, but for only 157.5MB you can grow your RAID 5
0308    volume to the following data-spindle counts without re-aligning:
0309         3,4,5,6,7,8,9,10,12,14,15,18,20,21 ...
0310 
0311  - Bad write performance
0312 
0313    If write performance is not what you expected, you probably wanted to be
0314    running in writeback mode, which isn't the default (not due to a lack of
0315    maturity, but simply because in writeback mode you'll lose data if something
0316    happens to your SSD)
0317 
0318    # echo writeback > /sys/block/bcache0/bcache/cache_mode
0319 
0320  - Bad performance, or traffic not going to the SSD that you'd expect
0321 
0322    By default, bcache doesn't cache everything. It tries to skip sequential IO -
0323    because you really want to be caching the random IO, and if you copy a 10
0324    gigabyte file you probably don't want that pushing 10 gigabytes of randomly
0325    accessed data out of your cache.
0326 
0327    But if you want to benchmark reads from cache, and you start out with fio
0328    writing an 8 gigabyte test file - so you want to disable that.
0329 
0330    # echo 0 > /sys/block/bcache0/bcache/sequential_cutoff
0331 
0332    To set it back to the default (4 mb), do
0333 
0334    # echo 4M > /sys/block/bcache0/bcache/sequential_cutoff
0335 
0336  - Traffic's still going to the spindle/still getting cache misses
0337 
0338    In the real world, SSDs don't always keep up with disks - particularly with
0339    slower SSDs, many disks being cached by one SSD, or mostly sequential IO. So
0340    you want to avoid being bottlenecked by the SSD and having it slow everything
0341    down.
0342 
0343    To avoid that bcache tracks latency to the cache device, and gradually
0344    throttles traffic if the latency exceeds a threshold (it does this by
0345    cranking down the sequential bypass).
0346 
0347    You can disable this if you need to by setting the thresholds to 0:
0348 
0349    # echo 0 > /sys/fs/bcache/<cache set>/congested_read_threshold_us
0350    # echo 0 > /sys/fs/bcache/<cache set>/congested_write_threshold_us
0351 
0352    The default is 2000 us (2 milliseconds) for reads, and 20000 for writes.
0353 
0354  - Still getting cache misses, of the same data
0355 
0356    One last issue that sometimes trips people up is actually an old bug, due to
0357    the way cache coherency is handled for cache misses. If a btree node is full,
0358    a cache miss won't be able to insert a key for the new data and the data
0359    won't be written to the cache.
0360 
0361    In practice this isn't an issue because as soon as a write comes along it'll
0362    cause the btree node to be split, and you need almost no write traffic for
0363    this to not show up enough to be noticeable (especially since bcache's btree
0364    nodes are huge and index large regions of the device). But when you're
0365    benchmarking, if you're trying to warm the cache by reading a bunch of data
0366    and there's no other traffic - that can be a problem.
0367 
0368    Solution: warm the cache by doing writes, or use the testing branch (there's
0369    a fix for the issue there).
0370 
0371 
0372 SYSFS - BACKING DEVICE
0373 ----------------------
0374 
0375 Available at /sys/block/<bdev>/bcache, /sys/block/bcache*/bcache and
0376 (if attached) /sys/fs/bcache/<cset-uuid>/bdev*
0377 
0378 attach
0379   Echo the UUID of a cache set to this file to enable caching.
0380 
0381 cache_mode
0382   Can be one of either writethrough, writeback, writearound or none.
0383 
0384 clear_stats
0385   Writing to this file resets the running total stats (not the day/hour/5 minute
0386   decaying versions).
0387 
0388 detach
0389   Write to this file to detach from a cache set. If there is dirty data in the
0390   cache, it will be flushed first.
0391 
0392 dirty_data
0393   Amount of dirty data for this backing device in the cache. Continuously
0394   updated unlike the cache set's version, but may be slightly off.
0395 
0396 label
0397   Name of underlying device.
0398 
0399 readahead
0400   Size of readahead that should be performed.  Defaults to 0.  If set to e.g.
0401   1M, it will round cache miss reads up to that size, but without overlapping
0402   existing cache entries.
0403 
0404 running
0405   1 if bcache is running (i.e. whether the /dev/bcache device exists, whether
0406   it's in passthrough mode or caching).
0407 
0408 sequential_cutoff
0409   A sequential IO will bypass the cache once it passes this threshold; the
0410   most recent 128 IOs are tracked so sequential IO can be detected even when
0411   it isn't all done at once.
0412 
0413 sequential_merge
0414   If non zero, bcache keeps a list of the last 128 requests submitted to compare
0415   against all new requests to determine which new requests are sequential
0416   continuations of previous requests for the purpose of determining sequential
0417   cutoff. This is necessary if the sequential cutoff value is greater than the
0418   maximum acceptable sequential size for any single request.
0419 
0420 state
0421   The backing device can be in one of four different states:
0422 
0423   no cache: Has never been attached to a cache set.
0424 
0425   clean: Part of a cache set, and there is no cached dirty data.
0426 
0427   dirty: Part of a cache set, and there is cached dirty data.
0428 
0429   inconsistent: The backing device was forcibly run by the user when there was
0430   dirty data cached but the cache set was unavailable; whatever data was on the
0431   backing device has likely been corrupted.
0432 
0433 stop
0434   Write to this file to shut down the bcache device and close the backing
0435   device.
0436 
0437 writeback_delay
0438   When dirty data is written to the cache and it previously did not contain
0439   any, waits some number of seconds before initiating writeback. Defaults to
0440   30.
0441 
0442 writeback_percent
0443   If nonzero, bcache tries to keep around this percentage of the cache dirty by
0444   throttling background writeback and using a PD controller to smoothly adjust
0445   the rate.
0446 
0447 writeback_rate
0448   Rate in sectors per second - if writeback_percent is nonzero, background
0449   writeback is throttled to this rate. Continuously adjusted by bcache but may
0450   also be set by the user.
0451 
0452 writeback_running
0453   If off, writeback of dirty data will not take place at all. Dirty data will
0454   still be added to the cache until it is mostly full; only meant for
0455   benchmarking. Defaults to on.
0456 
0457 SYSFS - BACKING DEVICE STATS:
0458 
0459 There are directories with these numbers for a running total, as well as
0460 versions that decay over the past day, hour and 5 minutes; they're also
0461 aggregated in the cache set directory as well.
0462 
0463 bypassed
0464   Amount of IO (both reads and writes) that has bypassed the cache
0465 
0466 cache_hits
0467 cache_misses
0468 cache_hit_ratio
0469   Hits and misses are counted per individual IO as bcache sees them; a
0470   partial hit is counted as a miss.
0471 
0472 cache_bypass_hits
0473 cache_bypass_misses
0474   Hits and misses for IO that is intended to skip the cache are still counted,
0475   but broken out here.
0476 
0477 cache_miss_collisions
0478   Counts instances where data was going to be inserted into the cache from a
0479   cache miss, but raced with a write and data was already present (usually 0
0480   since the synchronization for cache misses was rewritten)
0481 
0482 cache_readaheads
0483   Count of times readahead occurred.
0484 
0485 SYSFS - CACHE SET:
0486 
0487 Available at /sys/fs/bcache/<cset-uuid>
0488 
0489 average_key_size
0490   Average data per key in the btree.
0491 
0492 bdev<0..n>
0493   Symlink to each of the attached backing devices.
0494 
0495 block_size
0496   Block size of the cache devices.
0497 
0498 btree_cache_size
0499   Amount of memory currently used by the btree cache
0500 
0501 bucket_size
0502   Size of buckets
0503 
0504 cache<0..n>
0505   Symlink to each of the cache devices comprising this cache set.
0506 
0507 cache_available_percent
0508   Percentage of cache device which doesn't contain dirty data, and could
0509   potentially be used for writeback.  This doesn't mean this space isn't used
0510   for clean cached data; the unused statistic (in priority_stats) is typically
0511   much lower.
0512 
0513 clear_stats
0514   Clears the statistics associated with this cache
0515 
0516 dirty_data
0517   Amount of dirty data is in the cache (updated when garbage collection runs).
0518 
0519 flash_vol_create
0520   Echoing a size to this file (in human readable units, k/M/G) creates a thinly
0521   provisioned volume backed by the cache set.
0522 
0523 io_error_halflife
0524 io_error_limit
0525   These determines how many errors we accept before disabling the cache.
0526   Each error is decayed by the half life (in # ios).  If the decaying count
0527   reaches io_error_limit dirty data is written out and the cache is disabled.
0528 
0529 journal_delay_ms
0530   Journal writes will delay for up to this many milliseconds, unless a cache
0531   flush happens sooner. Defaults to 100.
0532 
0533 root_usage_percent
0534   Percentage of the root btree node in use.  If this gets too high the node
0535   will split, increasing the tree depth.
0536 
0537 stop
0538   Write to this file to shut down the cache set - waits until all attached
0539   backing devices have been shut down.
0540 
0541 tree_depth
0542   Depth of the btree (A single node btree has depth 0).
0543 
0544 unregister
0545   Detaches all backing devices and closes the cache devices; if dirty data is
0546   present it will disable writeback caching and wait for it to be flushed.
0547 
0548 SYSFS - CACHE SET INTERNAL:
0549 
0550 This directory also exposes timings for a number of internal operations, with
0551 separate files for average duration, average frequency, last occurrence and max
0552 duration: garbage collection, btree read, btree node sorts and btree splits.
0553 
0554 active_journal_entries
0555   Number of journal entries that are newer than the index.
0556 
0557 btree_nodes
0558   Total nodes in the btree.
0559 
0560 btree_used_percent
0561   Average fraction of btree in use.
0562 
0563 bset_tree_stats
0564   Statistics about the auxiliary search trees
0565 
0566 btree_cache_max_chain
0567   Longest chain in the btree node cache's hash table
0568 
0569 cache_read_races
0570   Counts instances where while data was being read from the cache, the bucket
0571   was reused and invalidated - i.e. where the pointer was stale after the read
0572   completed. When this occurs the data is reread from the backing device.
0573 
0574 trigger_gc
0575   Writing to this file forces garbage collection to run.
0576 
0577 SYSFS - CACHE DEVICE:
0578 
0579 Available at /sys/block/<cdev>/bcache
0580 
0581 block_size
0582   Minimum granularity of writes - should match hardware sector size.
0583 
0584 btree_written
0585   Sum of all btree writes, in (kilo/mega/giga) bytes
0586 
0587 bucket_size
0588   Size of buckets
0589 
0590 cache_replacement_policy
0591   One of either lru, fifo or random.
0592 
0593 discard
0594   Boolean; if on a discard/TRIM will be issued to each bucket before it is
0595   reused. Defaults to off, since SATA TRIM is an unqueued command (and thus
0596   slow).
0597 
0598 freelist_percent
0599   Size of the freelist as a percentage of nbuckets. Can be written to to
0600   increase the number of buckets kept on the freelist, which lets you
0601   artificially reduce the size of the cache at runtime. Mostly for testing
0602   purposes (i.e. testing how different size caches affect your hit rate), but
0603   since buckets are discarded when they move on to the freelist will also make
0604   the SSD's garbage collection easier by effectively giving it more reserved
0605   space.
0606 
0607 io_errors
0608   Number of errors that have occurred, decayed by io_error_halflife.
0609 
0610 metadata_written
0611   Sum of all non data writes (btree writes and all other metadata).
0612 
0613 nbuckets
0614   Total buckets in this cache
0615 
0616 priority_stats
0617   Statistics about how recently data in the cache has been accessed.
0618   This can reveal your working set size.  Unused is the percentage of
0619   the cache that doesn't contain any data.  Metadata is bcache's
0620   metadata overhead.  Average is the average priority of cache buckets.
0621   Next is a list of quantiles with the priority threshold of each.
0622 
0623 written
0624   Sum of all data that has been written to the cache; comparison with
0625   btree_written gives the amount of write inflation in bcache.