0001 Introduction
0002 ============
0003
0004 ATA over Ethernet is a network protocol that provides simple access to
0005 block storage on the LAN.
0006
0007 http://support.coraid.com/documents/AoEr11.txt
0008
0009 The EtherDrive (R) HOWTO for 2.6 and 3.x kernels is found at ...
0010
0011 http://support.coraid.com/support/linux/EtherDrive-2.6-HOWTO.html
0012
0013 It has many tips and hints! Please see, especially, recommended
0014 tunings for virtual memory:
0015
0016 http://support.coraid.com/support/linux/EtherDrive-2.6-HOWTO-5.html#ss5.19
0017
0018 The aoetools are userland programs that are designed to work with this
0019 driver. The aoetools are on sourceforge.
0020
0021 http://aoetools.sourceforge.net/
0022
0023 The scripts in this Documentation/admin-guide/aoe directory are intended to
0024 document the use of the driver and are not necessary if you install
0025 the aoetools.
0026
0027
0028 Creating Device Nodes
0029 =====================
0030
0031 Users of udev should find the block device nodes created
0032 automatically, but to create all the necessary device nodes, use the
0033 udev configuration rules provided in udev.txt (in this directory).
0034
0035 There is a udev-install.sh script that shows how to install these
0036 rules on your system.
0037
0038 There is also an autoload script that shows how to edit
0039 /etc/modprobe.d/aoe.conf to ensure that the aoe module is loaded when
0040 necessary. Preloading the aoe module is preferable to autoloading,
0041 however, because AoE discovery takes a few seconds. It can be
0042 confusing when an AoE device is not present the first time the a
0043 command is run but appears a second later.
0044
0045 Using Device Nodes
0046 ==================
0047
0048 "cat /dev/etherd/err" blocks, waiting for error diagnostic output,
0049 like any retransmitted packets.
0050
0051 "echo eth2 eth4 > /dev/etherd/interfaces" tells the aoe driver to
0052 limit ATA over Ethernet traffic to eth2 and eth4. AoE traffic from
0053 untrusted networks should be ignored as a matter of security. See
0054 also the aoe_iflist driver option described below.
0055
0056 "echo > /dev/etherd/discover" tells the driver to find out what AoE
0057 devices are available.
0058
0059 In the future these character devices may disappear and be replaced
0060 by sysfs counterparts. Using the commands in aoetools insulates
0061 users from these implementation details.
0062
0063 The block devices are named like this::
0064
0065 e{shelf}.{slot}
0066 e{shelf}.{slot}p{part}
0067
0068 ... so that "e0.2" is the third blade from the left (slot 2) in the
0069 first shelf (shelf address zero). That's the whole disk. The first
0070 partition on that disk would be "e0.2p1".
0071
0072 Using sysfs
0073 ===========
0074
0075 Each aoe block device in /sys/block has the extra attributes of
0076 state, mac, and netif. The state attribute is "up" when the device
0077 is ready for I/O and "down" if detected but unusable. The
0078 "down,closewait" state shows that the device is still open and
0079 cannot come up again until it has been closed.
0080
0081 The mac attribute is the ethernet address of the remote AoE device.
0082 The netif attribute is the network interface on the localhost
0083 through which we are communicating with the remote AoE device.
0084
0085 There is a script in this directory that formats this information in
0086 a convenient way. Users with aoetools should use the aoe-stat
0087 command::
0088
0089 root@makki root# sh Documentation/admin-guide/aoe/status.sh
0090 e10.0 eth3 up
0091 e10.1 eth3 up
0092 e10.2 eth3 up
0093 e10.3 eth3 up
0094 e10.4 eth3 up
0095 e10.5 eth3 up
0096 e10.6 eth3 up
0097 e10.7 eth3 up
0098 e10.8 eth3 up
0099 e10.9 eth3 up
0100 e4.0 eth1 up
0101 e4.1 eth1 up
0102 e4.2 eth1 up
0103 e4.3 eth1 up
0104 e4.4 eth1 up
0105 e4.5 eth1 up
0106 e4.6 eth1 up
0107 e4.7 eth1 up
0108 e4.8 eth1 up
0109 e4.9 eth1 up
0110
0111 Use /sys/module/aoe/parameters/aoe_iflist (or better, the driver
0112 option discussed below) instead of /dev/etherd/interfaces to limit
0113 AoE traffic to the network interfaces in the given
0114 whitespace-separated list. Unlike the old character device, the
0115 sysfs entry can be read from as well as written to.
0116
0117 It's helpful to trigger discovery after setting the list of allowed
0118 interfaces. The aoetools package provides an aoe-discover script
0119 for this purpose. You can also directly use the
0120 /dev/etherd/discover special file described above.
0121
0122 Driver Options
0123 ==============
0124
0125 There is a boot option for the built-in aoe driver and a
0126 corresponding module parameter, aoe_iflist. Without this option,
0127 all network interfaces may be used for ATA over Ethernet. Here is a
0128 usage example for the module parameter::
0129
0130 modprobe aoe_iflist="eth1 eth3"
0131
0132 The aoe_deadsecs module parameter determines the maximum number of
0133 seconds that the driver will wait for an AoE device to provide a
0134 response to an AoE command. After aoe_deadsecs seconds have
0135 elapsed, the AoE device will be marked as "down". A value of zero
0136 is supported for testing purposes and makes the aoe driver keep
0137 trying AoE commands forever.
0138
0139 The aoe_maxout module parameter has a default of 128. This is the
0140 maximum number of unresponded packets that will be sent to an AoE
0141 target at one time.
0142
0143 The aoe_dyndevs module parameter defaults to 1, meaning that the
0144 driver will assign a block device minor number to a discovered AoE
0145 target based on the order of its discovery. With dynamic minor
0146 device numbers in use, a greater range of AoE shelf and slot
0147 addresses can be supported. Users with udev will never have to
0148 think about minor numbers. Using aoe_dyndevs=0 allows device nodes
0149 to be pre-created using a static minor-number scheme with the
0150 aoe-mkshelf script in the aoetools.
