0001 .. SPDX-License-Identifier: GPL-2.0
0002
0003 ======
0004 NILFS2
0005 ======
0006
0007 NILFS2 is a log-structured file system (LFS) supporting continuous
0008 snapshotting. In addition to versioning capability of the entire file
0009 system, users can even restore files mistakenly overwritten or
0010 destroyed just a few seconds ago. Since NILFS2 can keep consistency
0011 like conventional LFS, it achieves quick recovery after system
0012 crashes.
0013
0014 NILFS2 creates a number of checkpoints every few seconds or per
0015 synchronous write basis (unless there is no change). Users can select
0016 significant versions among continuously created checkpoints, and can
0017 change them into snapshots which will be preserved until they are
0018 changed back to checkpoints.
0019
0020 There is no limit on the number of snapshots until the volume gets
0021 full. Each snapshot is mountable as a read-only file system
0022 concurrently with its writable mount, and this feature is convenient
0023 for online backup.
0024
0025 The userland tools are included in nilfs-utils package, which is
0026 available from the following download page. At least "mkfs.nilfs2",
0027 "mount.nilfs2", "umount.nilfs2", and "nilfs_cleanerd" (so called
0028 cleaner or garbage collector) are required. Details on the tools are
0029 described in the man pages included in the package.
0030
0031 :Project web page: https://nilfs.sourceforge.io/
0032 :Download page: https://nilfs.sourceforge.io/en/download.html
0033 :List info: http://vger.kernel.org/vger-lists.html#linux-nilfs
0034
0035 Caveats
0036 =======
0037
0038 Features which NILFS2 does not support yet:
0039
0040 - atime
0041 - extended attributes
0042 - POSIX ACLs
0043 - quotas
0044 - fsck
0045 - defragmentation
0046
0047 Mount options
0048 =============
0049
0050 NILFS2 supports the following mount options:
0051 (*) == default
0052
0053 ======================= =======================================================
0054 barrier(*) This enables/disables the use of write barriers. This
0055 nobarrier requires an IO stack which can support barriers, and
0056 if nilfs gets an error on a barrier write, it will
0057 disable again with a warning.
0058 errors=continue Keep going on a filesystem error.
0059 errors=remount-ro(*) Remount the filesystem read-only on an error.
0060 errors=panic Panic and halt the machine if an error occurs.
0061 cp=n Specify the checkpoint-number of the snapshot to be
0062 mounted. Checkpoints and snapshots are listed by lscp
0063 user command. Only the checkpoints marked as snapshot
0064 are mountable with this option. Snapshot is read-only,
0065 so a read-only mount option must be specified together.
0066 order=relaxed(*) Apply relaxed order semantics that allows modified data
0067 blocks to be written to disk without making a
0068 checkpoint if no metadata update is going. This mode
0069 is equivalent to the ordered data mode of the ext3
0070 filesystem except for the updates on data blocks still
0071 conserve atomicity. This will improve synchronous
0072 write performance for overwriting.
0073 order=strict Apply strict in-order semantics that preserves sequence
0074 of all file operations including overwriting of data
0075 blocks. That means, it is guaranteed that no
0076 overtaking of events occurs in the recovered file
0077 system after a crash.
0078 norecovery Disable recovery of the filesystem on mount.
0079 This disables every write access on the device for
0080 read-only mounts or snapshots. This option will fail
0081 for r/w mounts on an unclean volume.
0082 discard This enables/disables the use of discard/TRIM commands.
0083 nodiscard(*) The discard/TRIM commands are sent to the underlying
0084 block device when blocks are freed. This is useful
0085 for SSD devices and sparse/thinly-provisioned LUNs.
0086 ======================= =======================================================
0087
0088 Ioctls
0089 ======
0090
0091 There is some NILFS2 specific functionality which can be accessed by applications
0092 through the system call interfaces. The list of all NILFS2 specific ioctls are
0093 shown in the table below.
0094
0095 Table of NILFS2 specific ioctls:
0096
0097 ============================== ===============================================
0098 Ioctl Description
0099 ============================== ===============================================
0100 NILFS_IOCTL_CHANGE_CPMODE Change mode of given checkpoint between
0101 checkpoint and snapshot state. This ioctl is
0102 used in chcp and mkcp utilities.
0103
0104 NILFS_IOCTL_DELETE_CHECKPOINT Remove checkpoint from NILFS2 file system.
0105 This ioctl is used in rmcp utility.
0106
0107 NILFS_IOCTL_GET_CPINFO Return info about requested checkpoints. This
0108 ioctl is used in lscp utility and by
0109 nilfs_cleanerd daemon.
0110
0111 NILFS_IOCTL_GET_CPSTAT Return checkpoints statistics. This ioctl is
0112 used by lscp, rmcp utilities and by
0113 nilfs_cleanerd daemon.
0114
0115 NILFS_IOCTL_GET_SUINFO Return segment usage info about requested
0116 segments. This ioctl is used in lssu,
0117 nilfs_resize utilities and by nilfs_cleanerd
0118 daemon.
0119
0120 NILFS_IOCTL_SET_SUINFO Modify segment usage info of requested
0121 segments. This ioctl is used by
0122 nilfs_cleanerd daemon to skip unnecessary
0123 cleaning operation of segments and reduce
0124 performance penalty or wear of flash device
0125 due to redundant move of in-use blocks.
0126
0127 NILFS_IOCTL_GET_SUSTAT Return segment usage statistics. This ioctl
0128 is used in lssu, nilfs_resize utilities and
0129 by nilfs_cleanerd daemon.
0130
0131 NILFS_IOCTL_GET_VINFO Return information on virtual block addresses.
0132 This ioctl is used by nilfs_cleanerd daemon.
0133
0134 NILFS_IOCTL_GET_BDESCS Return information about descriptors of disk
0135 block numbers. This ioctl is used by
0136 nilfs_cleanerd daemon.
0137
0138 NILFS_IOCTL_CLEAN_SEGMENTS Do garbage collection operation in the
0139 environment of requested parameters from
0140 userspace. This ioctl is used by
0141 nilfs_cleanerd daemon.
0142
0143 NILFS_IOCTL_SYNC Make a checkpoint. This ioctl is used in
0144 mkcp utility.
0145
0146 NILFS_IOCTL_RESIZE Resize NILFS2 volume. This ioctl is used
0147 by nilfs_resize utility.
0148
0149 NILFS_IOCTL_SET_ALLOC_RANGE Define lower limit of segments in bytes and
0150 upper limit of segments in bytes. This ioctl
0151 is used by nilfs_resize utility.
0152 ============================== ===============================================
0153
0154 NILFS2 usage
0155 ============
0156
0157 To use nilfs2 as a local file system, simply::
0158
0159 # mkfs -t nilfs2 /dev/block_device
0160 # mount -t nilfs2 /dev/block_device /dir
0161
0162 This will also invoke the cleaner through the mount helper program
0163 (mount.nilfs2).
0164
0165 Checkpoints and snapshots are managed by the following commands.
0166 Their manpages are included in the nilfs-utils package above.
0167
0168 ==== ===========================================================
0169 lscp list checkpoints or snapshots.
0170 mkcp make a checkpoint or a snapshot.
0171 chcp change an existing checkpoint to a snapshot or vice versa.
0172 rmcp invalidate specified checkpoint(s).
0173 ==== ===========================================================
0174
0175 To mount a snapshot::
0176
0177 # mount -t nilfs2 -r -o cp=<cno> /dev/block_device /snap_dir
0178
0179 where <cno> is the checkpoint number of the snapshot.
0180
0181 To unmount the NILFS2 mount point or snapshot, simply::
0182
0183 # umount /dir
0184
0185 Then, the cleaner daemon is automatically shut down by the umount
0186 helper program (umount.nilfs2).
0187
0188 Disk format
0189 ===========
0190
0191 A nilfs2 volume is equally divided into a number of segments except
0192 for the super block (SB) and segment #0. A segment is the container
0193 of logs. Each log is composed of summary information blocks, payload
0194 blocks, and an optional super root block (SR)::
0195
0196 ______________________________________________________
0197 | |SB| | Segment | Segment | Segment | ... | Segment | |
0198 |_|__|_|____0____|____1____|____2____|_____|____N____|_|
0199 0 +1K +4K +8M +16M +24M +(8MB x N)
0200 . . (Typical offsets for 4KB-block)
0201 . .
0202 .______________________.
0203 | log | log |... | log |
0204 |__1__|__2__|____|__m__|
0205 . .
0206 . .
0207 . .
0208 .______________________________.
0209 | Summary | Payload blocks |SR|
0210 |_blocks__|_________________|__|
0211
0212 The payload blocks are organized per file, and each file consists of
0213 data blocks and B-tree node blocks::
0214
0215 |<--- File-A --->|<--- File-B --->|
0216 _______________________________________________________________
0217 | Data blocks | B-tree blocks | Data blocks | B-tree blocks | ...
0218 _|_____________|_______________|_____________|_______________|_
0219
0220
0221 Since only the modified blocks are written in the log, it may have
0222 files without data blocks or B-tree node blocks.
0223
0224 The organization of the blocks is recorded in the summary information
0225 blocks, which contains a header structure (nilfs_segment_summary), per
0226 file structures (nilfs_finfo), and per block structures (nilfs_binfo)::
0227
0228 _________________________________________________________________________
0229 | Summary | finfo | binfo | ... | binfo | finfo | binfo | ... | binfo |...
0230 |_blocks__|___A___|_(A,1)_|_____|(A,Na)_|___B___|_(B,1)_|_____|(B,Nb)_|___
0231
0232
0233 The logs include regular files, directory files, symbolic link files
0234 and several meta data files. The mata data files are the files used
0235 to maintain file system meta data. The current version of NILFS2 uses
0236 the following meta data files::
0237
0238 1) Inode file (ifile) -- Stores on-disk inodes
0239 2) Checkpoint file (cpfile) -- Stores checkpoints
0240 3) Segment usage file (sufile) -- Stores allocation state of segments
0241 4) Data address translation file -- Maps virtual block numbers to usual
0242 (DAT) block numbers. This file serves to
0243 make on-disk blocks relocatable.
0244
0245 The following figure shows a typical organization of the logs::
0246
0247 _________________________________________________________________________
0248 | Summary | regular file | file | ... | ifile | cpfile | sufile | DAT |SR|
0249 |_blocks__|_or_directory_|_______|_____|_______|________|________|_____|__|
0250
0251
0252 To stride over segment boundaries, this sequence of files may be split
0253 into multiple logs. The sequence of logs that should be treated as
0254 logically one log, is delimited with flags marked in the segment
0255 summary. The recovery code of nilfs2 looks this boundary information
0256 to ensure atomicity of updates.
0257
0258 The super root block is inserted for every checkpoints. It includes
0259 three special inodes, inodes for the DAT, cpfile, and sufile. Inodes
0260 of regular files, directories, symlinks and other special files, are
0261 included in the ifile. The inode of ifile itself is included in the
0262 corresponding checkpoint entry in the cpfile. Thus, the hierarchy
0263 among NILFS2 files can be depicted as follows::
0264
0265 Super block (SB)
0266 |
0267 v
0268 Super root block (the latest cno=xx)
0269 |-- DAT
0270 |-- sufile
0271 `-- cpfile
0272 |-- ifile (cno=c1)
0273 |-- ifile (cno=c2) ---- file (ino=i1)
0274 : : |-- file (ino=i2)
0275 `-- ifile (cno=xx) |-- file (ino=i3)
0276 : :
0277 `-- file (ino=yy)
0278 ( regular file, directory, or symlink )
0279
0280 For detail on the format of each file, please see nilfs2_ondisk.h
0281 located at include/uapi/linux directory.
0282
0283 There are no patents or other intellectual property that we protect
0284 with regard to the design of NILFS2. It is allowed to replicate the
0285 design in hopes that other operating systems could share (mount, read,
0286 write, etc.) data stored in this format.
