0001 Reexporting NFS filesystems
0002 ===========================
0003
0004 Overview
0005 --------
0006
0007 It is possible to reexport an NFS filesystem over NFS. However, this
0008 feature comes with a number of limitations. Before trying it, we
0009 recommend some careful research to determine whether it will work for
0010 your purposes.
0011
0012 A discussion of current known limitations follows.
0013
0014 "fsid=" required, crossmnt broken
0015 ---------------------------------
0016
0017 We require the "fsid=" export option on any reexport of an NFS
0018 filesystem. You can use "uuidgen -r" to generate a unique argument.
0019
0020 The "crossmnt" export does not propagate "fsid=", so it will not allow
0021 traversing into further nfs filesystems; if you wish to export nfs
0022 filesystems mounted under the exported filesystem, you'll need to export
0023 them explicitly, assigning each its own unique "fsid= option.
0024
0025 Reboot recovery
0026 ---------------
0027
0028 The NFS protocol's normal reboot recovery mechanisms don't work for the
0029 case when the reexport server reboots. Clients will lose any locks
0030 they held before the reboot, and further IO will result in errors.
0031 Closing and reopening files should clear the errors.
0032
0033 Filehandle limits
0034 -----------------
0035
0036 If the original server uses an X byte filehandle for a given object, the
0037 reexport server's filehandle for the reexported object will be X+22
0038 bytes, rounded up to the nearest multiple of four bytes.
0039
0040 The result must fit into the RFC-mandated filehandle size limits:
0041
0042 +-------+-----------+
0043 | NFSv2 | 32 bytes |
0044 +-------+-----------+
0045 | NFSv3 | 64 bytes |
0046 +-------+-----------+
0047 | NFSv4 | 128 bytes |
0048 +-------+-----------+
0049
0050 So, for example, you will only be able to reexport a filesystem over
0051 NFSv2 if the original server gives you filehandles that fit in 10
0052 bytes--which is unlikely.
0053
0054 In general there's no way to know the maximum filehandle size given out
0055 by an NFS server without asking the server vendor.
0056
0057 But the following table gives a few examples. The first column is the
0058 typical length of the filehandle from a Linux server exporting the given
0059 filesystem, the second is the length after that nfs export is reexported
0060 by another Linux host:
0061
0062 +--------+-------------------+----------------+
0063 | | filehandle length | after reexport |
0064 +========+===================+================+
0065 | ext4: | 28 bytes | 52 bytes |
0066 +--------+-------------------+----------------+
0067 | xfs: | 32 bytes | 56 bytes |
0068 +--------+-------------------+----------------+
0069 | btrfs: | 40 bytes | 64 bytes |
0070 +--------+-------------------+----------------+
0071
0072 All will therefore fit in an NFSv3 or NFSv4 filehandle after reexport,
0073 but none are reexportable over NFSv2.
0074
0075 Linux server filehandles are a bit more complicated than this, though;
0076 for example:
0077
0078 - The (non-default) "subtreecheck" export option generally
0079 requires another 4 to 8 bytes in the filehandle.
0080 - If you export a subdirectory of a filesystem (instead of
0081 exporting the filesystem root), that also usually adds 4 to 8
0082 bytes.
0083 - If you export over NFSv2, knfsd usually uses a shorter
0084 filesystem identifier that saves 8 bytes.
0085 - The root directory of an export uses a filehandle that is
0086 shorter.
0087
0088 As you can see, the 128-byte NFSv4 filehandle is large enough that
0089 you're unlikely to have trouble using NFSv4 to reexport any filesystem
0090 exported from a Linux server. In general, if the original server is
0091 something that also supports NFSv3, you're *probably* OK. Re-exporting
0092 over NFSv3 may be dicier, and reexporting over NFSv2 will probably
0093 never work.
0094
0095 For more details of Linux filehandle structure, the best reference is
0096 the source code and comments; see in particular:
0097
0098 - include/linux/exportfs.h:enum fid_type
0099 - include/uapi/linux/nfsd/nfsfh.h:struct nfs_fhbase_new
0100 - fs/nfsd/nfsfh.c:set_version_and_fsid_type
0101 - fs/nfs/export.c:nfs_encode_fh
0102
0103 Open DENY bits ignored
0104 ----------------------
0105
0106 NFS since NFSv4 supports ALLOW and DENY bits taken from Windows, which
0107 allow you, for example, to open a file in a mode which forbids other
0108 read opens or write opens. The Linux client doesn't use them, and the
0109 server's support has always been incomplete: they are enforced only
0110 against other NFS users, not against processes accessing the exported
0111 filesystem locally. A reexport server will also not pass them along to
0112 the original server, so they will not be enforced between clients of
0113 different reexport servers.
