0001 .. SPDX-License-Identifier: GPL-2.0
0002
0003 ==================
0004 KUnit Architecture
0005 ==================
0006
0007 The KUnit architecture can be divided into two parts:
0008
0009 - Kernel testing library
0010 - kunit_tool (Command line test harness)
0011
0012 In-Kernel Testing Framework
0013 ===========================
0014
0015 The kernel testing library supports KUnit tests written in C using
0016 KUnit. KUnit tests are kernel code. KUnit does several things:
0017
0018 - Organizes tests
0019 - Reports test results
0020 - Provides test utilities
0021
0022 Test Cases
0023 ----------
0024
0025 The fundamental unit in KUnit is the test case. The KUnit test cases are
0026 grouped into KUnit suites. A KUnit test case is a function with type
0027 signature ``void (*)(struct kunit *test)``.
0028 These test case functions are wrapped in a struct called
0029 struct kunit_case.
0030
0031 .. note:
0032 ``generate_params`` is optional for non-parameterized tests.
0033
0034 Each KUnit test case gets a ``struct kunit`` context
0035 object passed to it that tracks a running test. The KUnit assertion
0036 macros and other KUnit utilities use the ``struct kunit`` context
0037 object. As an exception, there are two fields:
0038
0039 - ``->priv``: The setup functions can use it to store arbitrary test
0040 user data.
0041
0042 - ``->param_value``: It contains the parameter value which can be
0043 retrieved in the parameterized tests.
0044
0045 Test Suites
0046 -----------
0047
0048 A KUnit suite includes a collection of test cases. The KUnit suites
0049 are represented by the ``struct kunit_suite``. For example:
0050
0051 .. code-block:: c
0052
0053 static struct kunit_case example_test_cases[] = {
0054 KUNIT_CASE(example_test_foo),
0055 KUNIT_CASE(example_test_bar),
0056 KUNIT_CASE(example_test_baz),
0057 {}
0058 };
0059
0060 static struct kunit_suite example_test_suite = {
0061 .name = "example",
0062 .init = example_test_init,
0063 .exit = example_test_exit,
0064 .test_cases = example_test_cases,
0065 };
0066 kunit_test_suite(example_test_suite);
0067
0068 In the above example, the test suite ``example_test_suite``, runs the
0069 test cases ``example_test_foo``, ``example_test_bar``, and
0070 ``example_test_baz``. Before running the test, the ``example_test_init``
0071 is called and after running the test, ``example_test_exit`` is called.
0072 The ``kunit_test_suite(example_test_suite)`` registers the test suite
0073 with the KUnit test framework.
0074
0075 Executor
0076 --------
0077
0078 The KUnit executor can list and run built-in KUnit tests on boot.
0079 The Test suites are stored in a linker section
0080 called ``.kunit_test_suites``. For code, see:
0081 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/include/asm-generic/vmlinux.lds.h?h=v5.15#n945.
0082 The linker section consists of an array of pointers to
0083 ``struct kunit_suite``, and is populated by the ``kunit_test_suites()``
0084 macro. To run all tests compiled into the kernel, the KUnit executor
0085 iterates over the linker section array.
0086
0087 .. kernel-figure:: kunit_suitememorydiagram.svg
0088 :alt: KUnit Suite Memory
0089
0090 KUnit Suite Memory Diagram
0091
0092 On the kernel boot, the KUnit executor uses the start and end addresses
0093 of this section to iterate over and run all tests. For code, see:
0094 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/lib/kunit/executor.c
0095
0096 When built as a module, the ``kunit_test_suites()`` macro defines a
0097 ``module_init()`` function, which runs all the tests in the compilation
0098 unit instead of utilizing the executor.
0099
0100 In KUnit tests, some error classes do not affect other tests
0101 or parts of the kernel, each KUnit case executes in a separate thread
0102 context. For code, see:
0103 https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/lib/kunit/try-catch.c?h=v5.15#n58
0104
0105 Assertion Macros
0106 ----------------
0107
0108 KUnit tests verify state using expectations/assertions.
0109 All expectations/assertions are formatted as:
0110 ``KUNIT_{EXPECT|ASSERT}_<op>[_MSG](kunit, property[, message])``
0111
0112 - ``{EXPECT|ASSERT}`` determines whether the check is an assertion or an
0113 expectation.
0114
0115 - For an expectation, if the check fails, marks the test as failed
0116 and logs the failure.
0117
0118 - An assertion, on failure, causes the test case to terminate
0119 immediately.
0120
0121 - Assertions call function:
0122 ``void __noreturn kunit_abort(struct kunit *)``.
0123
0124 - ``kunit_abort`` calls function:
0125 ``void __noreturn kunit_try_catch_throw(struct kunit_try_catch *try_catch)``.
0126
0127 - ``kunit_try_catch_throw`` calls function:
0128 ``void kthread_complete_and_exit(struct completion *, long) __noreturn;``
0129 and terminates the special thread context.
0130
0131 - ``<op>`` denotes a check with options: ``TRUE`` (supplied property
0132 has the boolean value “true”), ``EQ`` (two supplied properties are
0133 equal), ``NOT_ERR_OR_NULL`` (supplied pointer is not null and does not
0134 contain an “err” value).
0135
0136 - ``[_MSG]`` prints a custom message on failure.
0137
0138 Test Result Reporting
0139 ---------------------
0140 KUnit prints test results in KTAP format. KTAP is based on TAP14, see:
0141 https://github.com/isaacs/testanything.github.io/blob/tap14/tap-version-14-specification.md.
0142 KTAP (yet to be standardized format) works with KUnit and Kselftest.
0143 The KUnit executor prints KTAP results to dmesg, and debugfs
0144 (if configured).
0145
0146 Parameterized Tests
0147 -------------------
0148
0149 Each KUnit parameterized test is associated with a collection of
0150 parameters. The test is invoked multiple times, once for each parameter
0151 value and the parameter is stored in the ``param_value`` field.
0152 The test case includes a KUNIT_CASE_PARAM() macro that accepts a
0153 generator function.
0154 The generator function is passed the previous parameter and returns the next
0155 parameter. It also provides a macro to generate common-case generators based on
0156 arrays.
0157
0158 kunit_tool (Command Line Test Harness)
0159 ======================================
0160
0161 kunit_tool is a Python script ``(tools/testing/kunit/kunit.py)``
0162 that can be used to configure, build, exec, parse and run (runs other
0163 commands in order) test results. You can either run KUnit tests using
0164 kunit_tool or can include KUnit in kernel and parse manually.
0165
0166 - ``configure`` command generates the kernel ``.config`` from a
0167 ``.kunitconfig`` file (and any architecture-specific options).
0168 For some architectures, additional config options are specified in the
0169 ``qemu_config`` Python script
0170 (For example: ``tools/testing/kunit/qemu_configs/powerpc.py``).
0171 It parses both the existing ``.config`` and the ``.kunitconfig`` files
0172 and ensures that ``.config`` is a superset of ``.kunitconfig``.
0173 If this is not the case, it will combine the two and run
0174 ``make olddefconfig`` to regenerate the ``.config`` file. It then
0175 verifies that ``.config`` is now a superset. This checks if all
0176 Kconfig dependencies are correctly specified in ``.kunitconfig``.
0177 ``kunit_config.py`` includes the parsing Kconfigs code. The code which
0178 runs ``make olddefconfig`` is a part of ``kunit_kernel.py``. You can
0179 invoke this command via: ``./tools/testing/kunit/kunit.py config`` and
0180 generate a ``.config`` file.
0181 - ``build`` runs ``make`` on the kernel tree with required options
0182 (depends on the architecture and some options, for example: build_dir)
0183 and reports any errors.
0184 To build a KUnit kernel from the current ``.config``, you can use the
0185 ``build`` argument: ``./tools/testing/kunit/kunit.py build``.
0186 - ``exec`` command executes kernel results either directly (using
0187 User-mode Linux configuration), or via an emulator such
0188 as QEMU. It reads results from the log via standard
0189 output (stdout), and passes them to ``parse`` to be parsed.
0190 If you already have built a kernel with built-in KUnit tests,
0191 you can run the kernel and display the test results with the ``exec``
0192 argument: ``./tools/testing/kunit/kunit.py exec``.
0193 - ``parse`` extracts the KTAP output from a kernel log, parses
0194 the test results, and prints a summary. For failed tests, any
0195 diagnostic output will be included.
