0001 :orphan:
0002
0003 Making Filesystems Exportable
0004 =============================
0005
0006 Overview
0007 --------
0008
0009 All filesystem operations require a dentry (or two) as a starting
0010 point. Local applications have a reference-counted hold on suitable
0011 dentries via open file descriptors or cwd/root. However remote
0012 applications that access a filesystem via a remote filesystem protocol
0013 such as NFS may not be able to hold such a reference, and so need a
0014 different way to refer to a particular dentry. As the alternative
0015 form of reference needs to be stable across renames, truncates, and
0016 server-reboot (among other things, though these tend to be the most
0017 problematic), there is no simple answer like 'filename'.
0018
0019 The mechanism discussed here allows each filesystem implementation to
0020 specify how to generate an opaque (outside of the filesystem) byte
0021 string for any dentry, and how to find an appropriate dentry for any
0022 given opaque byte string.
0023 This byte string will be called a "filehandle fragment" as it
0024 corresponds to part of an NFS filehandle.
0025
0026 A filesystem which supports the mapping between filehandle fragments
0027 and dentries will be termed "exportable".
0028
0029
0030
0031 Dcache Issues
0032 -------------
0033
0034 The dcache normally contains a proper prefix of any given filesystem
0035 tree. This means that if any filesystem object is in the dcache, then
0036 all of the ancestors of that filesystem object are also in the dcache.
0037 As normal access is by filename this prefix is created naturally and
0038 maintained easily (by each object maintaining a reference count on
0039 its parent).
0040
0041 However when objects are included into the dcache by interpreting a
0042 filehandle fragment, there is no automatic creation of a path prefix
0043 for the object. This leads to two related but distinct features of
0044 the dcache that are not needed for normal filesystem access.
0045
0046 1. The dcache must sometimes contain objects that are not part of the
0047 proper prefix. i.e that are not connected to the root.
0048 2. The dcache must be prepared for a newly found (via ->lookup) directory
0049 to already have a (non-connected) dentry, and must be able to move
0050 that dentry into place (based on the parent and name in the
0051 ->lookup). This is particularly needed for directories as
0052 it is a dcache invariant that directories only have one dentry.
0053
0054 To implement these features, the dcache has:
0055
0056 a. A dentry flag DCACHE_DISCONNECTED which is set on
0057 any dentry that might not be part of the proper prefix.
0058 This is set when anonymous dentries are created, and cleared when a
0059 dentry is noticed to be a child of a dentry which is in the proper
0060 prefix. If the refcount on a dentry with this flag set
0061 becomes zero, the dentry is immediately discarded, rather than being
0062 kept in the dcache. If a dentry that is not already in the dcache
0063 is repeatedly accessed by filehandle (as NFSD might do), an new dentry
0064 will be a allocated for each access, and discarded at the end of
0065 the access.
0066
0067 Note that such a dentry can acquire children, name, ancestors, etc.
0068 without losing DCACHE_DISCONNECTED - that flag is only cleared when
0069 subtree is successfully reconnected to root. Until then dentries
0070 in such subtree are retained only as long as there are references;
0071 refcount reaching zero means immediate eviction, same as for unhashed
0072 dentries. That guarantees that we won't need to hunt them down upon
0073 umount.
0074
0075 b. A primitive for creation of secondary roots - d_obtain_root(inode).
0076 Those do _not_ bear DCACHE_DISCONNECTED. They are placed on the
0077 per-superblock list (->s_roots), so they can be located at umount
0078 time for eviction purposes.
0079
0080 c. Helper routines to allocate anonymous dentries, and to help attach
0081 loose directory dentries at lookup time. They are:
0082
0083 d_obtain_alias(inode) will return a dentry for the given inode.
0084 If the inode already has a dentry, one of those is returned.
0085
0086 If it doesn't, a new anonymous (IS_ROOT and
0087 DCACHE_DISCONNECTED) dentry is allocated and attached.
0088
0089 In the case of a directory, care is taken that only one dentry
0090 can ever be attached.
0091
0092 d_splice_alias(inode, dentry) will introduce a new dentry into the tree;
0093 either the passed-in dentry or a preexisting alias for the given inode
0094 (such as an anonymous one created by d_obtain_alias), if appropriate.
0095 It returns NULL when the passed-in dentry is used, following the calling
0096 convention of ->lookup.
0097
0098 Filesystem Issues
0099 -----------------
0100
0101 For a filesystem to be exportable it must:
0102
0103 1. provide the filehandle fragment routines described below.
0104 2. make sure that d_splice_alias is used rather than d_add
0105 when ->lookup finds an inode for a given parent and name.
0106
0107 If inode is NULL, d_splice_alias(inode, dentry) is equivalent to::
0108
0109 d_add(dentry, inode), NULL
0110
0111 Similarly, d_splice_alias(ERR_PTR(err), dentry) = ERR_PTR(err)
0112
0113 Typically the ->lookup routine will simply end with a::
0114
0115 return d_splice_alias(inode, dentry);
0116 }
0117
0118
0119
0120 A file system implementation declares that instances of the filesystem
0121 are exportable by setting the s_export_op field in the struct
0122 super_block. This field must point to a "struct export_operations"
0123 struct which has the following members:
0124
0125 encode_fh (optional)
0126 Takes a dentry and creates a filehandle fragment which can later be used
0127 to find or create a dentry for the same object. The default
0128 implementation creates a filehandle fragment that encodes a 32bit inode
0129 and generation number for the inode encoded, and if necessary the
0130 same information for the parent.
0131
0132 fh_to_dentry (mandatory)
0133 Given a filehandle fragment, this should find the implied object and
0134 create a dentry for it (possibly with d_obtain_alias).
0135
0136 fh_to_parent (optional but strongly recommended)
0137 Given a filehandle fragment, this should find the parent of the
0138 implied object and create a dentry for it (possibly with
0139 d_obtain_alias). May fail if the filehandle fragment is too small.
0140
0141 get_parent (optional but strongly recommended)
0142 When given a dentry for a directory, this should return a dentry for
0143 the parent. Quite possibly the parent dentry will have been allocated
0144 by d_alloc_anon. The default get_parent function just returns an error
0145 so any filehandle lookup that requires finding a parent will fail.
0146 ->lookup("..") is *not* used as a default as it can leave ".." entries
0147 in the dcache which are too messy to work with.
0148
0149 get_name (optional)
0150 When given a parent dentry and a child dentry, this should find a name
0151 in the directory identified by the parent dentry, which leads to the
0152 object identified by the child dentry. If no get_name function is
0153 supplied, a default implementation is provided which uses vfs_readdir
0154 to find potential names, and matches inode numbers to find the correct
0155 match.
0156
0157 flags
0158 Some filesystems may need to be handled differently than others. The
0159 export_operations struct also includes a flags field that allows the
0160 filesystem to communicate such information to nfsd. See the Export
0161 Operations Flags section below for more explanation.
0162
0163 A filehandle fragment consists of an array of 1 or more 4byte words,
0164 together with a one byte "type".
0165 The decode_fh routine should not depend on the stated size that is
0166 passed to it. This size may be larger than the original filehandle
0167 generated by encode_fh, in which case it will have been padded with
0168 nuls. Rather, the encode_fh routine should choose a "type" which
0169 indicates the decode_fh how much of the filehandle is valid, and how
0170 it should be interpreted.
0171
0172 Export Operations Flags
0173 -----------------------
0174 In addition to the operation vector pointers, struct export_operations also
0175 contains a "flags" field that allows the filesystem to communicate to nfsd
0176 that it may want to do things differently when dealing with it. The
0177 following flags are defined:
0178
0179 EXPORT_OP_NOWCC - disable NFSv3 WCC attributes on this filesystem
0180 RFC 1813 recommends that servers always send weak cache consistency
0181 (WCC) data to the client after each operation. The server should
0182 atomically collect attributes about the inode, do an operation on it,
0183 and then collect the attributes afterward. This allows the client to
0184 skip issuing GETATTRs in some situations but means that the server
0185 is calling vfs_getattr for almost all RPCs. On some filesystems
0186 (particularly those that are clustered or networked) this is expensive
0187 and atomicity is difficult to guarantee. This flag indicates to nfsd
0188 that it should skip providing WCC attributes to the client in NFSv3
0189 replies when doing operations on this filesystem. Consider enabling
0190 this on filesystems that have an expensive ->getattr inode operation,
0191 or when atomicity between pre and post operation attribute collection
0192 is impossible to guarantee.
0193
0194 EXPORT_OP_NOSUBTREECHK - disallow subtree checking on this fs
0195 Many NFS operations deal with filehandles, which the server must then
0196 vet to ensure that they live inside of an exported tree. When the
0197 export consists of an entire filesystem, this is trivial. nfsd can just
0198 ensure that the filehandle live on the filesystem. When only part of a
0199 filesystem is exported however, then nfsd must walk the ancestors of the
0200 inode to ensure that it's within an exported subtree. This is an
0201 expensive operation and not all filesystems can support it properly.
0202 This flag exempts the filesystem from subtree checking and causes
0203 exportfs to get back an error if it tries to enable subtree checking
0204 on it.
0205
0206 EXPORT_OP_CLOSE_BEFORE_UNLINK - always close cached files before unlinking
0207 On some exportable filesystems (such as NFS) unlinking a file that
0208 is still open can cause a fair bit of extra work. For instance,
0209 the NFS client will do a "sillyrename" to ensure that the file
0210 sticks around while it's still open. When reexporting, that open
0211 file is held by nfsd so we usually end up doing a sillyrename, and
0212 then immediately deleting the sillyrenamed file just afterward when
0213 the link count actually goes to zero. Sometimes this delete can race
0214 with other operations (for instance an rmdir of the parent directory).
0215 This flag causes nfsd to close any open files for this inode _before_
0216 calling into the vfs to do an unlink or a rename that would replace
0217 an existing file.