0001 .. SPDX-License-Identifier: GPL-2.0
0002
0003 .. _fsverity:
0004
0005 =======================================================
0006 fs-verity: read-only file-based authenticity protection
0007 =======================================================
0008
0009 Introduction
0010 ============
0011
0012 fs-verity (``fs/verity/``) is a support layer that filesystems can
0013 hook into to support transparent integrity and authenticity protection
0014 of read-only files. Currently, it is supported by the ext4, f2fs, and
0015 btrfs filesystems. Like fscrypt, not too much filesystem-specific
0016 code is needed to support fs-verity.
0017
0018 fs-verity is similar to `dm-verity
0019 <https://www.kernel.org/doc/Documentation/device-mapper/verity.txt>`_
0020 but works on files rather than block devices. On regular files on
0021 filesystems supporting fs-verity, userspace can execute an ioctl that
0022 causes the filesystem to build a Merkle tree for the file and persist
0023 it to a filesystem-specific location associated with the file.
0024
0025 After this, the file is made readonly, and all reads from the file are
0026 automatically verified against the file's Merkle tree. Reads of any
0027 corrupted data, including mmap reads, will fail.
0028
0029 Userspace can use another ioctl to retrieve the root hash (actually
0030 the "fs-verity file digest", which is a hash that includes the Merkle
0031 tree root hash) that fs-verity is enforcing for the file. This ioctl
0032 executes in constant time, regardless of the file size.
0033
0034 fs-verity is essentially a way to hash a file in constant time,
0035 subject to the caveat that reads which would violate the hash will
0036 fail at runtime.
0037
0038 Use cases
0039 =========
0040
0041 By itself, the base fs-verity feature only provides integrity
0042 protection, i.e. detection of accidental (non-malicious) corruption.
0043
0044 However, because fs-verity makes retrieving the file hash extremely
0045 efficient, it's primarily meant to be used as a tool to support
0046 authentication (detection of malicious modifications) or auditing
0047 (logging file hashes before use).
0048
0049 Trusted userspace code (e.g. operating system code running on a
0050 read-only partition that is itself authenticated by dm-verity) can
0051 authenticate the contents of an fs-verity file by using the
0052 `FS_IOC_MEASURE_VERITY`_ ioctl to retrieve its hash, then verifying a
0053 digital signature of it.
0054
0055 A standard file hash could be used instead of fs-verity. However,
0056 this is inefficient if the file is large and only a small portion may
0057 be accessed. This is often the case for Android application package
0058 (APK) files, for example. These typically contain many translations,
0059 classes, and other resources that are infrequently or even never
0060 accessed on a particular device. It would be slow and wasteful to
0061 read and hash the entire file before starting the application.
0062
0063 Unlike an ahead-of-time hash, fs-verity also re-verifies data each
0064 time it's paged in. This ensures that malicious disk firmware can't
0065 undetectably change the contents of the file at runtime.
0066
0067 fs-verity does not replace or obsolete dm-verity. dm-verity should
0068 still be used on read-only filesystems. fs-verity is for files that
0069 must live on a read-write filesystem because they are independently
0070 updated and potentially user-installed, so dm-verity cannot be used.
0071
0072 The base fs-verity feature is a hashing mechanism only; actually
0073 authenticating the files may be done by:
0074
0075 * Userspace-only
0076
0077 * Builtin signature verification + userspace policy
0078
0079 fs-verity optionally supports a simple signature verification
0080 mechanism where users can configure the kernel to require that
0081 all fs-verity files be signed by a key loaded into a keyring;
0082 see `Built-in signature verification`_.
0083
0084 * Integrity Measurement Architecture (IMA)
0085
0086 IMA supports including fs-verity file digests and signatures in the
0087 IMA measurement list and verifying fs-verity based file signatures
0088 stored as security.ima xattrs, based on policy.
0089
0090
0091 User API
0092 ========
0093
0094 FS_IOC_ENABLE_VERITY
0095 --------------------
0096
0097 The FS_IOC_ENABLE_VERITY ioctl enables fs-verity on a file. It takes
0098 in a pointer to a struct fsverity_enable_arg, defined as
0099 follows::
0100
0101 struct fsverity_enable_arg {
0102 __u32 version;
0103 __u32 hash_algorithm;
0104 __u32 block_size;
0105 __u32 salt_size;
0106 __u64 salt_ptr;
0107 __u32 sig_size;
0108 __u32 __reserved1;
0109 __u64 sig_ptr;
0110 __u64 __reserved2[11];
0111 };
0112
0113 This structure contains the parameters of the Merkle tree to build for
0114 the file, and optionally contains a signature. It must be initialized
0115 as follows:
0116
0117 - ``version`` must be 1.
0118 - ``hash_algorithm`` must be the identifier for the hash algorithm to
0119 use for the Merkle tree, such as FS_VERITY_HASH_ALG_SHA256. See
0120 ``include/uapi/linux/fsverity.h`` for the list of possible values.
0121 - ``block_size`` must be the Merkle tree block size. Currently, this
0122 must be equal to the system page size, which is usually 4096 bytes.
0123 Other sizes may be supported in the future. This value is not
0124 necessarily the same as the filesystem block size.
0125 - ``salt_size`` is the size of the salt in bytes, or 0 if no salt is
0126 provided. The salt is a value that is prepended to every hashed
0127 block; it can be used to personalize the hashing for a particular
0128 file or device. Currently the maximum salt size is 32 bytes.
0129 - ``salt_ptr`` is the pointer to the salt, or NULL if no salt is
0130 provided.
0131 - ``sig_size`` is the size of the signature in bytes, or 0 if no
0132 signature is provided. Currently the signature is (somewhat
0133 arbitrarily) limited to 16128 bytes. See `Built-in signature
0134 verification`_ for more information.
0135 - ``sig_ptr`` is the pointer to the signature, or NULL if no
0136 signature is provided.
0137 - All reserved fields must be zeroed.
0138
0139 FS_IOC_ENABLE_VERITY causes the filesystem to build a Merkle tree for
0140 the file and persist it to a filesystem-specific location associated
0141 with the file, then mark the file as a verity file. This ioctl may
0142 take a long time to execute on large files, and it is interruptible by
0143 fatal signals.
0144
0145 FS_IOC_ENABLE_VERITY checks for write access to the inode. However,
0146 it must be executed on an O_RDONLY file descriptor and no processes
0147 can have the file open for writing. Attempts to open the file for
0148 writing while this ioctl is executing will fail with ETXTBSY. (This
0149 is necessary to guarantee that no writable file descriptors will exist
0150 after verity is enabled, and to guarantee that the file's contents are
0151 stable while the Merkle tree is being built over it.)
0152
0153 On success, FS_IOC_ENABLE_VERITY returns 0, and the file becomes a
0154 verity file. On failure (including the case of interruption by a
0155 fatal signal), no changes are made to the file.
0156
0157 FS_IOC_ENABLE_VERITY can fail with the following errors:
0158
0159 - ``EACCES``: the process does not have write access to the file
0160 - ``EBADMSG``: the signature is malformed
0161 - ``EBUSY``: this ioctl is already running on the file
0162 - ``EEXIST``: the file already has verity enabled
0163 - ``EFAULT``: the caller provided inaccessible memory
0164 - ``EINTR``: the operation was interrupted by a fatal signal
0165 - ``EINVAL``: unsupported version, hash algorithm, or block size; or
0166 reserved bits are set; or the file descriptor refers to neither a
0167 regular file nor a directory.
0168 - ``EISDIR``: the file descriptor refers to a directory
0169 - ``EKEYREJECTED``: the signature doesn't match the file
0170 - ``EMSGSIZE``: the salt or signature is too long
0171 - ``ENOKEY``: the fs-verity keyring doesn't contain the certificate
0172 needed to verify the signature
0173 - ``ENOPKG``: fs-verity recognizes the hash algorithm, but it's not
0174 available in the kernel's crypto API as currently configured (e.g.
0175 for SHA-512, missing CONFIG_CRYPTO_SHA512).
0176 - ``ENOTTY``: this type of filesystem does not implement fs-verity
0177 - ``EOPNOTSUPP``: the kernel was not configured with fs-verity
0178 support; or the filesystem superblock has not had the 'verity'
0179 feature enabled on it; or the filesystem does not support fs-verity
0180 on this file. (See `Filesystem support`_.)
0181 - ``EPERM``: the file is append-only; or, a signature is required and
0182 one was not provided.
0183 - ``EROFS``: the filesystem is read-only
0184 - ``ETXTBSY``: someone has the file open for writing. This can be the
0185 caller's file descriptor, another open file descriptor, or the file
0186 reference held by a writable memory map.
0187
0188 FS_IOC_MEASURE_VERITY
0189 ---------------------
0190
0191 The FS_IOC_MEASURE_VERITY ioctl retrieves the digest of a verity file.
0192 The fs-verity file digest is a cryptographic digest that identifies
0193 the file contents that are being enforced on reads; it is computed via
0194 a Merkle tree and is different from a traditional full-file digest.
0195
0196 This ioctl takes in a pointer to a variable-length structure::
0197
0198 struct fsverity_digest {
0199 __u16 digest_algorithm;
0200 __u16 digest_size; /* input/output */
0201 __u8 digest[];
0202 };
0203
0204 ``digest_size`` is an input/output field. On input, it must be
0205 initialized to the number of bytes allocated for the variable-length
0206 ``digest`` field.
0207
0208 On success, 0 is returned and the kernel fills in the structure as
0209 follows:
0210
0211 - ``digest_algorithm`` will be the hash algorithm used for the file
0212 digest. It will match ``fsverity_enable_arg::hash_algorithm``.
0213 - ``digest_size`` will be the size of the digest in bytes, e.g. 32
0214 for SHA-256. (This can be redundant with ``digest_algorithm``.)
0215 - ``digest`` will be the actual bytes of the digest.
0216
0217 FS_IOC_MEASURE_VERITY is guaranteed to execute in constant time,
0218 regardless of the size of the file.
0219
0220 FS_IOC_MEASURE_VERITY can fail with the following errors:
0221
0222 - ``EFAULT``: the caller provided inaccessible memory
0223 - ``ENODATA``: the file is not a verity file
0224 - ``ENOTTY``: this type of filesystem does not implement fs-verity
0225 - ``EOPNOTSUPP``: the kernel was not configured with fs-verity
0226 support, or the filesystem superblock has not had the 'verity'
0227 feature enabled on it. (See `Filesystem support`_.)
0228 - ``EOVERFLOW``: the digest is longer than the specified
0229 ``digest_size`` bytes. Try providing a larger buffer.
0230
0231 FS_IOC_READ_VERITY_METADATA
0232 ---------------------------
0233
0234 The FS_IOC_READ_VERITY_METADATA ioctl reads verity metadata from a
0235 verity file. This ioctl is available since Linux v5.12.
0236
0237 This ioctl allows writing a server program that takes a verity file
0238 and serves it to a client program, such that the client can do its own
0239 fs-verity compatible verification of the file. This only makes sense
0240 if the client doesn't trust the server and if the server needs to
0241 provide the storage for the client.
0242
0243 This is a fairly specialized use case, and most fs-verity users won't
0244 need this ioctl.
0245
0246 This ioctl takes in a pointer to the following structure::
0247
0248 #define FS_VERITY_METADATA_TYPE_MERKLE_TREE 1
0249 #define FS_VERITY_METADATA_TYPE_DESCRIPTOR 2
0250 #define FS_VERITY_METADATA_TYPE_SIGNATURE 3
0251
0252 struct fsverity_read_metadata_arg {
0253 __u64 metadata_type;
0254 __u64 offset;
0255 __u64 length;
0256 __u64 buf_ptr;
0257 __u64 __reserved;
0258 };
0259
0260 ``metadata_type`` specifies the type of metadata to read:
0261
0262 - ``FS_VERITY_METADATA_TYPE_MERKLE_TREE`` reads the blocks of the
0263 Merkle tree. The blocks are returned in order from the root level
0264 to the leaf level. Within each level, the blocks are returned in
0265 the same order that their hashes are themselves hashed.
0266 See `Merkle tree`_ for more information.
0267
0268 - ``FS_VERITY_METADATA_TYPE_DESCRIPTOR`` reads the fs-verity
0269 descriptor. See `fs-verity descriptor`_.
0270
0271 - ``FS_VERITY_METADATA_TYPE_SIGNATURE`` reads the signature which was
0272 passed to FS_IOC_ENABLE_VERITY, if any. See `Built-in signature
0273 verification`_.
0274
0275 The semantics are similar to those of ``pread()``. ``offset``
0276 specifies the offset in bytes into the metadata item to read from, and
0277 ``length`` specifies the maximum number of bytes to read from the
0278 metadata item. ``buf_ptr`` is the pointer to the buffer to read into,
0279 cast to a 64-bit integer. ``__reserved`` must be 0. On success, the
0280 number of bytes read is returned. 0 is returned at the end of the
0281 metadata item. The returned length may be less than ``length``, for
0282 example if the ioctl is interrupted.
0283
0284 The metadata returned by FS_IOC_READ_VERITY_METADATA isn't guaranteed
0285 to be authenticated against the file digest that would be returned by
0286 `FS_IOC_MEASURE_VERITY`_, as the metadata is expected to be used to
0287 implement fs-verity compatible verification anyway (though absent a
0288 malicious disk, the metadata will indeed match). E.g. to implement
0289 this ioctl, the filesystem is allowed to just read the Merkle tree
0290 blocks from disk without actually verifying the path to the root node.
0291
0292 FS_IOC_READ_VERITY_METADATA can fail with the following errors:
0293
0294 - ``EFAULT``: the caller provided inaccessible memory
0295 - ``EINTR``: the ioctl was interrupted before any data was read
0296 - ``EINVAL``: reserved fields were set, or ``offset + length``
0297 overflowed
0298 - ``ENODATA``: the file is not a verity file, or
0299 FS_VERITY_METADATA_TYPE_SIGNATURE was requested but the file doesn't
0300 have a built-in signature
0301 - ``ENOTTY``: this type of filesystem does not implement fs-verity, or
0302 this ioctl is not yet implemented on it
0303 - ``EOPNOTSUPP``: the kernel was not configured with fs-verity
0304 support, or the filesystem superblock has not had the 'verity'
0305 feature enabled on it. (See `Filesystem support`_.)
0306
0307 FS_IOC_GETFLAGS
0308 ---------------
0309
0310 The existing ioctl FS_IOC_GETFLAGS (which isn't specific to fs-verity)
0311 can also be used to check whether a file has fs-verity enabled or not.
0312 To do so, check for FS_VERITY_FL (0x00100000) in the returned flags.
0313
0314 The verity flag is not settable via FS_IOC_SETFLAGS. You must use
0315 FS_IOC_ENABLE_VERITY instead, since parameters must be provided.
0316
0317 statx
0318 -----
0319
0320 Since Linux v5.5, the statx() system call sets STATX_ATTR_VERITY if
0321 the file has fs-verity enabled. This can perform better than
0322 FS_IOC_GETFLAGS and FS_IOC_MEASURE_VERITY because it doesn't require
0323 opening the file, and opening verity files can be expensive.
0324
0325 Accessing verity files
0326 ======================
0327
0328 Applications can transparently access a verity file just like a
0329 non-verity one, with the following exceptions:
0330
0331 - Verity files are readonly. They cannot be opened for writing or
0332 truncate()d, even if the file mode bits allow it. Attempts to do
0333 one of these things will fail with EPERM. However, changes to
0334 metadata such as owner, mode, timestamps, and xattrs are still
0335 allowed, since these are not measured by fs-verity. Verity files
0336 can also still be renamed, deleted, and linked to.
0337
0338 - Direct I/O is not supported on verity files. Attempts to use direct
0339 I/O on such files will fall back to buffered I/O.
0340
0341 - DAX (Direct Access) is not supported on verity files, because this
0342 would circumvent the data verification.
0343
0344 - Reads of data that doesn't match the verity Merkle tree will fail
0345 with EIO (for read()) or SIGBUS (for mmap() reads).
0346
0347 - If the sysctl "fs.verity.require_signatures" is set to 1 and the
0348 file is not signed by a key in the fs-verity keyring, then opening
0349 the file will fail. See `Built-in signature verification`_.
0350
0351 Direct access to the Merkle tree is not supported. Therefore, if a
0352 verity file is copied, or is backed up and restored, then it will lose
0353 its "verity"-ness. fs-verity is primarily meant for files like
0354 executables that are managed by a package manager.
0355
0356 File digest computation
0357 =======================
0358
0359 This section describes how fs-verity hashes the file contents using a
0360 Merkle tree to produce the digest which cryptographically identifies
0361 the file contents. This algorithm is the same for all filesystems
0362 that support fs-verity.
0363
0364 Userspace only needs to be aware of this algorithm if it needs to
0365 compute fs-verity file digests itself, e.g. in order to sign files.
0366
0367 .. _fsverity_merkle_tree:
0368
0369 Merkle tree
0370 -----------
0371
0372 The file contents is divided into blocks, where the block size is
0373 configurable but is usually 4096 bytes. The end of the last block is
0374 zero-padded if needed. Each block is then hashed, producing the first
0375 level of hashes. Then, the hashes in this first level are grouped
0376 into 'blocksize'-byte blocks (zero-padding the ends as needed) and
0377 these blocks are hashed, producing the second level of hashes. This
0378 proceeds up the tree until only a single block remains. The hash of
0379 this block is the "Merkle tree root hash".
0380
0381 If the file fits in one block and is nonempty, then the "Merkle tree
0382 root hash" is simply the hash of the single data block. If the file
0383 is empty, then the "Merkle tree root hash" is all zeroes.
0384
0385 The "blocks" here are not necessarily the same as "filesystem blocks".
0386
0387 If a salt was specified, then it's zero-padded to the closest multiple
0388 of the input size of the hash algorithm's compression function, e.g.
0389 64 bytes for SHA-256 or 128 bytes for SHA-512. The padded salt is
0390 prepended to every data or Merkle tree block that is hashed.
0391
0392 The purpose of the block padding is to cause every hash to be taken
0393 over the same amount of data, which simplifies the implementation and
0394 keeps open more possibilities for hardware acceleration. The purpose
0395 of the salt padding is to make the salting "free" when the salted hash
0396 state is precomputed, then imported for each hash.
0397
0398 Example: in the recommended configuration of SHA-256 and 4K blocks,
0399 128 hash values fit in each block. Thus, each level of the Merkle
0400 tree is approximately 128 times smaller than the previous, and for
0401 large files the Merkle tree's size converges to approximately 1/127 of
0402 the original file size. However, for small files, the padding is
0403 significant, making the space overhead proportionally more.
0404
0405 .. _fsverity_descriptor:
0406
0407 fs-verity descriptor
0408 --------------------
0409
0410 By itself, the Merkle tree root hash is ambiguous. For example, it
0411 can't a distinguish a large file from a small second file whose data
0412 is exactly the top-level hash block of the first file. Ambiguities
0413 also arise from the convention of padding to the next block boundary.
0414
0415 To solve this problem, the fs-verity file digest is actually computed
0416 as a hash of the following structure, which contains the Merkle tree
0417 root hash as well as other fields such as the file size::
0418
0419 struct fsverity_descriptor {
0420 __u8 version; /* must be 1 */
0421 __u8 hash_algorithm; /* Merkle tree hash algorithm */
0422 __u8 log_blocksize; /* log2 of size of data and tree blocks */
0423 __u8 salt_size; /* size of salt in bytes; 0 if none */
0424 __le32 __reserved_0x04; /* must be 0 */
0425 __le64 data_size; /* size of file the Merkle tree is built over */
0426 __u8 root_hash[64]; /* Merkle tree root hash */
0427 __u8 salt[32]; /* salt prepended to each hashed block */
0428 __u8 __reserved[144]; /* must be 0's */
0429 };
0430
0431 Built-in signature verification
0432 ===============================
0433
0434 With CONFIG_FS_VERITY_BUILTIN_SIGNATURES=y, fs-verity supports putting
0435 a portion of an authentication policy (see `Use cases`_) in the
0436 kernel. Specifically, it adds support for:
0437
0438 1. At fs-verity module initialization time, a keyring ".fs-verity" is
0439 created. The root user can add trusted X.509 certificates to this
0440 keyring using the add_key() system call, then (when done)
0441 optionally use keyctl_restrict_keyring() to prevent additional
0442 certificates from being added.
0443
0444 2. `FS_IOC_ENABLE_VERITY`_ accepts a pointer to a PKCS#7 formatted
0445 detached signature in DER format of the file's fs-verity digest.
0446 On success, this signature is persisted alongside the Merkle tree.
0447 Then, any time the file is opened, the kernel will verify the
0448 file's actual digest against this signature, using the certificates
0449 in the ".fs-verity" keyring.
0450
0451 3. A new sysctl "fs.verity.require_signatures" is made available.
0452 When set to 1, the kernel requires that all verity files have a
0453 correctly signed digest as described in (2).
0454
0455 fs-verity file digests must be signed in the following format, which
0456 is similar to the structure used by `FS_IOC_MEASURE_VERITY`_::
0457
0458 struct fsverity_formatted_digest {
0459 char magic[8]; /* must be "FSVerity" */
0460 __le16 digest_algorithm;
0461 __le16 digest_size;
0462 __u8 digest[];
0463 };
0464
0465 fs-verity's built-in signature verification support is meant as a
0466 relatively simple mechanism that can be used to provide some level of
0467 authenticity protection for verity files, as an alternative to doing
0468 the signature verification in userspace or using IMA-appraisal.
0469 However, with this mechanism, userspace programs still need to check
0470 that the verity bit is set, and there is no protection against verity
0471 files being swapped around.
0472
0473 Filesystem support
0474 ==================
0475
0476 fs-verity is supported by several filesystems, described below. The
0477 CONFIG_FS_VERITY kconfig option must be enabled to use fs-verity on
0478 any of these filesystems.
0479
0480 ``include/linux/fsverity.h`` declares the interface between the
0481 ``fs/verity/`` support layer and filesystems. Briefly, filesystems
0482 must provide an ``fsverity_operations`` structure that provides
0483 methods to read and write the verity metadata to a filesystem-specific
0484 location, including the Merkle tree blocks and
0485 ``fsverity_descriptor``. Filesystems must also call functions in
0486 ``fs/verity/`` at certain times, such as when a file is opened or when
0487 pages have been read into the pagecache. (See `Verifying data`_.)
0488
0489 ext4
0490 ----
0491
0492 ext4 supports fs-verity since Linux v5.4 and e2fsprogs v1.45.2.
0493
0494 To create verity files on an ext4 filesystem, the filesystem must have
0495 been formatted with ``-O verity`` or had ``tune2fs -O verity`` run on
0496 it. "verity" is an RO_COMPAT filesystem feature, so once set, old
0497 kernels will only be able to mount the filesystem readonly, and old
0498 versions of e2fsck will be unable to check the filesystem. Moreover,
0499 currently ext4 only supports mounting a filesystem with the "verity"
0500 feature when its block size is equal to PAGE_SIZE (often 4096 bytes).
0501
0502 ext4 sets the EXT4_VERITY_FL on-disk inode flag on verity files. It
0503 can only be set by `FS_IOC_ENABLE_VERITY`_, and it cannot be cleared.
0504
0505 ext4 also supports encryption, which can be used simultaneously with
0506 fs-verity. In this case, the plaintext data is verified rather than
0507 the ciphertext. This is necessary in order to make the fs-verity file
0508 digest meaningful, since every file is encrypted differently.
0509
0510 ext4 stores the verity metadata (Merkle tree and fsverity_descriptor)
0511 past the end of the file, starting at the first 64K boundary beyond
0512 i_size. This approach works because (a) verity files are readonly,
0513 and (b) pages fully beyond i_size aren't visible to userspace but can
0514 be read/written internally by ext4 with only some relatively small
0515 changes to ext4. This approach avoids having to depend on the
0516 EA_INODE feature and on rearchitecturing ext4's xattr support to
0517 support paging multi-gigabyte xattrs into memory, and to support
0518 encrypting xattrs. Note that the verity metadata *must* be encrypted
0519 when the file is, since it contains hashes of the plaintext data.
0520
0521 Currently, ext4 verity only supports the case where the Merkle tree
0522 block size, filesystem block size, and page size are all the same. It
0523 also only supports extent-based files.
0524
0525 f2fs
0526 ----
0527
0528 f2fs supports fs-verity since Linux v5.4 and f2fs-tools v1.11.0.
0529
0530 To create verity files on an f2fs filesystem, the filesystem must have
0531 been formatted with ``-O verity``.
0532
0533 f2fs sets the FADVISE_VERITY_BIT on-disk inode flag on verity files.
0534 It can only be set by `FS_IOC_ENABLE_VERITY`_, and it cannot be
0535 cleared.
0536
0537 Like ext4, f2fs stores the verity metadata (Merkle tree and
0538 fsverity_descriptor) past the end of the file, starting at the first
0539 64K boundary beyond i_size. See explanation for ext4 above.
0540 Moreover, f2fs supports at most 4096 bytes of xattr entries per inode
0541 which wouldn't be enough for even a single Merkle tree block.
0542
0543 Currently, f2fs verity only supports a Merkle tree block size of 4096.
0544 Also, f2fs doesn't support enabling verity on files that currently
0545 have atomic or volatile writes pending.
0546
0547 btrfs
0548 -----
0549
0550 btrfs supports fs-verity since Linux v5.15. Verity-enabled inodes are
0551 marked with a RO_COMPAT inode flag, and the verity metadata is stored
0552 in separate btree items.
0553
0554 Implementation details
0555 ======================
0556
0557 Verifying data
0558 --------------
0559
0560 fs-verity ensures that all reads of a verity file's data are verified,
0561 regardless of which syscall is used to do the read (e.g. mmap(),
0562 read(), pread()) and regardless of whether it's the first read or a
0563 later read (unless the later read can return cached data that was
0564 already verified). Below, we describe how filesystems implement this.
0565
0566 Pagecache
0567 ~~~~~~~~~
0568
0569 For filesystems using Linux's pagecache, the ``->read_folio()`` and
0570 ``->readahead()`` methods must be modified to verify pages before they
0571 are marked Uptodate. Merely hooking ``->read_iter()`` would be
0572 insufficient, since ``->read_iter()`` is not used for memory maps.
0573
0574 Therefore, fs/verity/ provides a function fsverity_verify_page() which
0575 verifies a page that has been read into the pagecache of a verity
0576 inode, but is still locked and not Uptodate, so it's not yet readable
0577 by userspace. As needed to do the verification,
0578 fsverity_verify_page() will call back into the filesystem to read
0579 Merkle tree pages via fsverity_operations::read_merkle_tree_page().
0580
0581 fsverity_verify_page() returns false if verification failed; in this
0582 case, the filesystem must not set the page Uptodate. Following this,
0583 as per the usual Linux pagecache behavior, attempts by userspace to
0584 read() from the part of the file containing the page will fail with
0585 EIO, and accesses to the page within a memory map will raise SIGBUS.
0586
0587 fsverity_verify_page() currently only supports the case where the
0588 Merkle tree block size is equal to PAGE_SIZE (often 4096 bytes).
0589
0590 In principle, fsverity_verify_page() verifies the entire path in the
0591 Merkle tree from the data page to the root hash. However, for
0592 efficiency the filesystem may cache the hash pages. Therefore,
0593 fsverity_verify_page() only ascends the tree reading hash pages until
0594 an already-verified hash page is seen, as indicated by the PageChecked
0595 bit being set. It then verifies the path to that page.
0596
0597 This optimization, which is also used by dm-verity, results in
0598 excellent sequential read performance. This is because usually (e.g.
0599 127 in 128 times for 4K blocks and SHA-256) the hash page from the
0600 bottom level of the tree will already be cached and checked from
0601 reading a previous data page. However, random reads perform worse.
0602
0603 Block device based filesystems
0604 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
0605
0606 Block device based filesystems (e.g. ext4 and f2fs) in Linux also use
0607 the pagecache, so the above subsection applies too. However, they
0608 also usually read many pages from a file at once, grouped into a
0609 structure called a "bio". To make it easier for these types of
0610 filesystems to support fs-verity, fs/verity/ also provides a function
0611 fsverity_verify_bio() which verifies all pages in a bio.
0612
0613 ext4 and f2fs also support encryption. If a verity file is also
0614 encrypted, the pages must be decrypted before being verified. To
0615 support this, these filesystems allocate a "post-read context" for
0616 each bio and store it in ``->bi_private``::
0617
0618 struct bio_post_read_ctx {
0619 struct bio *bio;
0620 struct work_struct work;
0621 unsigned int cur_step;
0622 unsigned int enabled_steps;
0623 };
0624
0625 ``enabled_steps`` is a bitmask that specifies whether decryption,
0626 verity, or both is enabled. After the bio completes, for each needed
0627 postprocessing step the filesystem enqueues the bio_post_read_ctx on a
0628 workqueue, and then the workqueue work does the decryption or
0629 verification. Finally, pages where no decryption or verity error
0630 occurred are marked Uptodate, and the pages are unlocked.
0631
0632 On many filesystems, files can contain holes. Normally,
0633 ``->readahead()`` simply zeroes holes and sets the corresponding pages
0634 Uptodate; no bios are issued. To prevent this case from bypassing
0635 fs-verity, these filesystems use fsverity_verify_page() to verify hole
0636 pages.
0637
0638 Filesystems also disable direct I/O on verity files, since otherwise
0639 direct I/O would bypass fs-verity.
0640
0641 Userspace utility
0642 =================
0643
0644 This document focuses on the kernel, but a userspace utility for
0645 fs-verity can be found at:
0646
0647 https://git.kernel.org/pub/scm/linux/kernel/git/ebiggers/fsverity-utils.git
0648
0649 See the README.md file in the fsverity-utils source tree for details,
0650 including examples of setting up fs-verity protected files.
0651
0652 Tests
0653 =====
0654
0655 To test fs-verity, use xfstests. For example, using `kvm-xfstests
0656 <https://github.com/tytso/xfstests-bld/blob/master/Documentation/kvm-quickstart.md>`_::
0657
0658 kvm-xfstests -c ext4,f2fs,btrfs -g verity
0659
0660 FAQ
0661 ===
0662
0663 This section answers frequently asked questions about fs-verity that
0664 weren't already directly answered in other parts of this document.
0665
0666 :Q: Why isn't fs-verity part of IMA?
0667 :A: fs-verity and IMA (Integrity Measurement Architecture) have
0668 different focuses. fs-verity is a filesystem-level mechanism for
0669 hashing individual files using a Merkle tree. In contrast, IMA
0670 specifies a system-wide policy that specifies which files are
0671 hashed and what to do with those hashes, such as log them,
0672 authenticate them, or add them to a measurement list.
0673
0674 IMA supports the fs-verity hashing mechanism as an alternative
0675 to full file hashes, for those who want the performance and
0676 security benefits of the Merkle tree based hash. However, it
0677 doesn't make sense to force all uses of fs-verity to be through
0678 IMA. fs-verity already meets many users' needs even as a
0679 standalone filesystem feature, and it's testable like other
0680 filesystem features e.g. with xfstests.
0681
0682 :Q: Isn't fs-verity useless because the attacker can just modify the
0683 hashes in the Merkle tree, which is stored on-disk?
0684 :A: To verify the authenticity of an fs-verity file you must verify
0685 the authenticity of the "fs-verity file digest", which
0686 incorporates the root hash of the Merkle tree. See `Use cases`_.
0687
0688 :Q: Isn't fs-verity useless because the attacker can just replace a
0689 verity file with a non-verity one?
0690 :A: See `Use cases`_. In the initial use case, it's really trusted
0691 userspace code that authenticates the files; fs-verity is just a
0692 tool to do this job efficiently and securely. The trusted
0693 userspace code will consider non-verity files to be inauthentic.
0694
0695 :Q: Why does the Merkle tree need to be stored on-disk? Couldn't you
0696 store just the root hash?
0697 :A: If the Merkle tree wasn't stored on-disk, then you'd have to
0698 compute the entire tree when the file is first accessed, even if
0699 just one byte is being read. This is a fundamental consequence of
0700 how Merkle tree hashing works. To verify a leaf node, you need to
0701 verify the whole path to the root hash, including the root node
0702 (the thing which the root hash is a hash of). But if the root
0703 node isn't stored on-disk, you have to compute it by hashing its
0704 children, and so on until you've actually hashed the entire file.
0705
0706 That defeats most of the point of doing a Merkle tree-based hash,
0707 since if you have to hash the whole file ahead of time anyway,
0708 then you could simply do sha256(file) instead. That would be much
0709 simpler, and a bit faster too.
0710
0711 It's true that an in-memory Merkle tree could still provide the
0712 advantage of verification on every read rather than just on the
0713 first read. However, it would be inefficient because every time a
0714 hash page gets evicted (you can't pin the entire Merkle tree into
0715 memory, since it may be very large), in order to restore it you
0716 again need to hash everything below it in the tree. This again
0717 defeats most of the point of doing a Merkle tree-based hash, since
0718 a single block read could trigger re-hashing gigabytes of data.
0719
0720 :Q: But couldn't you store just the leaf nodes and compute the rest?
0721 :A: See previous answer; this really just moves up one level, since
0722 one could alternatively interpret the data blocks as being the
0723 leaf nodes of the Merkle tree. It's true that the tree can be
0724 computed much faster if the leaf level is stored rather than just
0725 the data, but that's only because each level is less than 1% the
0726 size of the level below (assuming the recommended settings of
0727 SHA-256 and 4K blocks). For the exact same reason, by storing
0728 "just the leaf nodes" you'd already be storing over 99% of the
0729 tree, so you might as well simply store the whole tree.
0730
0731 :Q: Can the Merkle tree be built ahead of time, e.g. distributed as
0732 part of a package that is installed to many computers?
0733 :A: This isn't currently supported. It was part of the original
0734 design, but was removed to simplify the kernel UAPI and because it
0735 wasn't a critical use case. Files are usually installed once and
0736 used many times, and cryptographic hashing is somewhat fast on
0737 most modern processors.
0738
0739 :Q: Why doesn't fs-verity support writes?
0740 :A: Write support would be very difficult and would require a
0741 completely different design, so it's well outside the scope of
0742 fs-verity. Write support would require:
0743
0744 - A way to maintain consistency between the data and hashes,
0745 including all levels of hashes, since corruption after a crash
0746 (especially of potentially the entire file!) is unacceptable.
0747 The main options for solving this are data journalling,
0748 copy-on-write, and log-structured volume. But it's very hard to
0749 retrofit existing filesystems with new consistency mechanisms.
0750 Data journalling is available on ext4, but is very slow.
0751
0752 - Rebuilding the Merkle tree after every write, which would be
0753 extremely inefficient. Alternatively, a different authenticated
0754 dictionary structure such as an "authenticated skiplist" could
0755 be used. However, this would be far more complex.
0756
0757 Compare it to dm-verity vs. dm-integrity. dm-verity is very
0758 simple: the kernel just verifies read-only data against a
0759 read-only Merkle tree. In contrast, dm-integrity supports writes
0760 but is slow, is much more complex, and doesn't actually support
0761 full-device authentication since it authenticates each sector
0762 independently, i.e. there is no "root hash". It doesn't really
0763 make sense for the same device-mapper target to support these two
0764 very different cases; the same applies to fs-verity.
0765
0766 :Q: Since verity files are immutable, why isn't the immutable bit set?
0767 :A: The existing "immutable" bit (FS_IMMUTABLE_FL) already has a
0768 specific set of semantics which not only make the file contents
0769 read-only, but also prevent the file from being deleted, renamed,
0770 linked to, or having its owner or mode changed. These extra
0771 properties are unwanted for fs-verity, so reusing the immutable
0772 bit isn't appropriate.
0773
0774 :Q: Why does the API use ioctls instead of setxattr() and getxattr()?
0775 :A: Abusing the xattr interface for basically arbitrary syscalls is
0776 heavily frowned upon by most of the Linux filesystem developers.
0777 An xattr should really just be an xattr on-disk, not an API to
0778 e.g. magically trigger construction of a Merkle tree.
0779
0780 :Q: Does fs-verity support remote filesystems?
0781 :A: So far all filesystems that have implemented fs-verity support are
0782 local filesystems, but in principle any filesystem that can store
0783 per-file verity metadata can support fs-verity, regardless of
0784 whether it's local or remote. Some filesystems may have fewer
0785 options of where to store the verity metadata; one possibility is
0786 to store it past the end of the file and "hide" it from userspace
0787 by manipulating i_size. The data verification functions provided
0788 by ``fs/verity/`` also assume that the filesystem uses the Linux
0789 pagecache, but both local and remote filesystems normally do so.
0790
0791 :Q: Why is anything filesystem-specific at all? Shouldn't fs-verity
0792 be implemented entirely at the VFS level?
0793 :A: There are many reasons why this is not possible or would be very
0794 difficult, including the following:
0795
0796 - To prevent bypassing verification, pages must not be marked
0797 Uptodate until they've been verified. Currently, each
0798 filesystem is responsible for marking pages Uptodate via
0799 ``->readahead()``. Therefore, currently it's not possible for
0800 the VFS to do the verification on its own. Changing this would
0801 require significant changes to the VFS and all filesystems.
0802
0803 - It would require defining a filesystem-independent way to store
0804 the verity metadata. Extended attributes don't work for this
0805 because (a) the Merkle tree may be gigabytes, but many
0806 filesystems assume that all xattrs fit into a single 4K
0807 filesystem block, and (b) ext4 and f2fs encryption doesn't
0808 encrypt xattrs, yet the Merkle tree *must* be encrypted when the
0809 file contents are, because it stores hashes of the plaintext
0810 file contents.
0811
0812 So the verity metadata would have to be stored in an actual
0813 file. Using a separate file would be very ugly, since the
0814 metadata is fundamentally part of the file to be protected, and
0815 it could cause problems where users could delete the real file
0816 but not the metadata file or vice versa. On the other hand,
0817 having it be in the same file would break applications unless
0818 filesystems' notion of i_size were divorced from the VFS's,
0819 which would be complex and require changes to all filesystems.
0820
0821 - It's desirable that FS_IOC_ENABLE_VERITY uses the filesystem's
0822 transaction mechanism so that either the file ends up with
0823 verity enabled, or no changes were made. Allowing intermediate
0824 states to occur after a crash may cause problems.