0001 ================================
0002 Device-mapper "unstriped" target
0003 ================================
0004
0005 Introduction
0006 ============
0007
0008 The device-mapper "unstriped" target provides a transparent mechanism to
0009 unstripe a device-mapper "striped" target to access the underlying disks
0010 without having to touch the true backing block-device. It can also be
0011 used to unstripe a hardware RAID-0 to access backing disks.
0012
0013 Parameters:
0014 <number of stripes> <chunk size> <stripe #> <dev_path> <offset>
0015
0016 <number of stripes>
0017 The number of stripes in the RAID 0.
0018
0019 <chunk size>
0020 The amount of 512B sectors in the chunk striping.
0021
0022 <dev_path>
0023 The block device you wish to unstripe.
0024
0025 <stripe #>
0026 The stripe number within the device that corresponds to physical
0027 drive you wish to unstripe. This must be 0 indexed.
0028
0029
0030 Why use this module?
0031 ====================
0032
0033 An example of undoing an existing dm-stripe
0034 -------------------------------------------
0035
0036 This small bash script will setup 4 loop devices and use the existing
0037 striped target to combine the 4 devices into one. It then will use
0038 the unstriped target ontop of the striped device to access the
0039 individual backing loop devices. We write data to the newly exposed
0040 unstriped devices and verify the data written matches the correct
0041 underlying device on the striped array::
0042
0043 #!/bin/bash
0044
0045 MEMBER_SIZE=$((128 * 1024 * 1024))
0046 NUM=4
0047 SEQ_END=$((${NUM}-1))
0048 CHUNK=256
0049 BS=4096
0050
0051 RAID_SIZE=$((${MEMBER_SIZE}*${NUM}/512))
0052 DM_PARMS="0 ${RAID_SIZE} striped ${NUM} ${CHUNK}"
0053 COUNT=$((${MEMBER_SIZE} / ${BS}))
0054
0055 for i in $(seq 0 ${SEQ_END}); do
0056 dd if=/dev/zero of=member-${i} bs=${MEMBER_SIZE} count=1 oflag=direct
0057 losetup /dev/loop${i} member-${i}
0058 DM_PARMS+=" /dev/loop${i} 0"
0059 done
0060
0061 echo $DM_PARMS | dmsetup create raid0
0062 for i in $(seq 0 ${SEQ_END}); do
0063 echo "0 1 unstriped ${NUM} ${CHUNK} ${i} /dev/mapper/raid0 0" | dmsetup create set-${i}
0064 done;
0065
0066 for i in $(seq 0 ${SEQ_END}); do
0067 dd if=/dev/urandom of=/dev/mapper/set-${i} bs=${BS} count=${COUNT} oflag=direct
0068 diff /dev/mapper/set-${i} member-${i}
0069 done;
0070
0071 for i in $(seq 0 ${SEQ_END}); do
0072 dmsetup remove set-${i}
0073 done
0074
0075 dmsetup remove raid0
0076
0077 for i in $(seq 0 ${SEQ_END}); do
0078 losetup -d /dev/loop${i}
0079 rm -f member-${i}
0080 done
0081
0082 Another example
0083 ---------------
0084
0085 Intel NVMe drives contain two cores on the physical device.
0086 Each core of the drive has segregated access to its LBA range.
0087 The current LBA model has a RAID 0 128k chunk on each core, resulting
0088 in a 256k stripe across the two cores::
0089
0090 Core 0: Core 1:
0091 __________ __________
0092 | LBA 512| | LBA 768|
0093 | LBA 0 | | LBA 256|
0094 ---------- ----------
0095
0096 The purpose of this unstriping is to provide better QoS in noisy
0097 neighbor environments. When two partitions are created on the
0098 aggregate drive without this unstriping, reads on one partition
0099 can affect writes on another partition. This is because the partitions
0100 are striped across the two cores. When we unstripe this hardware RAID 0
0101 and make partitions on each new exposed device the two partitions are now
0102 physically separated.
0103
0104 With the dm-unstriped target we're able to segregate an fio script that
0105 has read and write jobs that are independent of each other. Compared to
0106 when we run the test on a combined drive with partitions, we were able
0107 to get a 92% reduction in read latency using this device mapper target.
0108
0109
0110 Example dmsetup usage
0111 =====================
0112
0113 unstriped ontop of Intel NVMe device that has 2 cores
0114 -----------------------------------------------------
0115
0116 ::
0117
0118 dmsetup create nvmset0 --table '0 512 unstriped 2 256 0 /dev/nvme0n1 0'
0119 dmsetup create nvmset1 --table '0 512 unstriped 2 256 1 /dev/nvme0n1 0'
0120
0121 There will now be two devices that expose Intel NVMe core 0 and 1
0122 respectively::
0123
0124 /dev/mapper/nvmset0
0125 /dev/mapper/nvmset1
0126
0127 unstriped ontop of striped with 4 drives using 128K chunk size
0128 --------------------------------------------------------------
0129
0130 ::
0131
0132 dmsetup create raid_disk0 --table '0 512 unstriped 4 256 0 /dev/mapper/striped 0'
0133 dmsetup create raid_disk1 --table '0 512 unstriped 4 256 1 /dev/mapper/striped 0'
0134 dmsetup create raid_disk2 --table '0 512 unstriped 4 256 2 /dev/mapper/striped 0'
0135 dmsetup create raid_disk3 --table '0 512 unstriped 4 256 3 /dev/mapper/striped 0'
