0001 .. SPDX-License-Identifier: GPL-2.0
0002
0003 ==========================
0004 Frequently Asked Questions
0005 ==========================
0006
0007 How is this different from Autotest, kselftest, and so on?
0008 ==========================================================
0009 KUnit is a unit testing framework. Autotest, kselftest (and some others) are
0010 not.
0011
0012 A `unit test <https://martinfowler.com/bliki/UnitTest.html>`_ is supposed to
0013 test a single unit of code in isolation and hence the name *unit test*. A unit
0014 test should be the finest granularity of testing and should allow all possible
0015 code paths to be tested in the code under test. This is only possible if the
0016 code under test is small and does not have any external dependencies outside of
0017 the test's control like hardware.
0018
0019 There are no testing frameworks currently available for the kernel that do not
0020 require installing the kernel on a test machine or in a virtual machine. All
0021 testing frameworks require tests to be written in userspace and run on the
0022 kernel under test. This is true for Autotest, kselftest, and some others,
0023 disqualifying any of them from being considered unit testing frameworks.
0024
0025 Does KUnit support running on architectures other than UML?
0026 ===========================================================
0027
0028 Yes, mostly.
0029
0030 For the most part, the KUnit core framework (what we use to write the tests)
0031 can compile to any architecture. It compiles like just another part of the
0032 kernel and runs when the kernel boots, or when built as a module, when the
0033 module is loaded. However, there is infrastructure, like the KUnit Wrapper
0034 (``tools/testing/kunit/kunit.py``) that does not support other architectures.
0035
0036 In short, yes, you can run KUnit on other architectures, but it might require
0037 more work than using KUnit on UML.
0038
0039 For more information, see :ref:`kunit-on-non-uml`.
0040
0041 What is the difference between a unit test and other kinds of tests?
0042 ====================================================================
0043 Most existing tests for the Linux kernel would be categorized as an integration
0044 test, or an end-to-end test.
0045
0046 - A unit test is supposed to test a single unit of code in isolation. A unit
0047 test should be the finest granularity of testing and, as such, allows all
0048 possible code paths to be tested in the code under test. This is only possible
0049 if the code under test is small and does not have any external dependencies
0050 outside of the test's control like hardware.
0051 - An integration test tests the interaction between a minimal set of components,
0052 usually just two or three. For example, someone might write an integration
0053 test to test the interaction between a driver and a piece of hardware, or to
0054 test the interaction between the userspace libraries the kernel provides and
0055 the kernel itself. However, one of these tests would probably not test the
0056 entire kernel along with hardware interactions and interactions with the
0057 userspace.
0058 - An end-to-end test usually tests the entire system from the perspective of the
0059 code under test. For example, someone might write an end-to-end test for the
0060 kernel by installing a production configuration of the kernel on production
0061 hardware with a production userspace and then trying to exercise some behavior
0062 that depends on interactions between the hardware, the kernel, and userspace.
0063
0064 KUnit is not working, what should I do?
0065 =======================================
0066
0067 Unfortunately, there are a number of things which can break, but here are some
0068 things to try.
0069
0070 1. Run ``./tools/testing/kunit/kunit.py run`` with the ``--raw_output``
0071 parameter. This might show details or error messages hidden by the kunit_tool
0072 parser.
0073 2. Instead of running ``kunit.py run``, try running ``kunit.py config``,
0074 ``kunit.py build``, and ``kunit.py exec`` independently. This can help track
0075 down where an issue is occurring. (If you think the parser is at fault, you
0076 can run it manually against ``stdin`` or a file with ``kunit.py parse``.)
0077 3. Running the UML kernel directly can often reveal issues or error messages,
0078 ``kunit_tool`` ignores. This should be as simple as running ``./vmlinux``
0079 after building the UML kernel (for example, by using ``kunit.py build``).
0080 Note that UML has some unusual requirements (such as the host having a tmpfs
0081 filesystem mounted), and has had issues in the past when built statically and
0082 the host has KASLR enabled. (On older host kernels, you may need to run
0083 ``setarch `uname -m` -R ./vmlinux`` to disable KASLR.)
0084 4. Make sure the kernel .config has ``CONFIG_KUNIT=y`` and at least one test
0085 (e.g. ``CONFIG_KUNIT_EXAMPLE_TEST=y``). kunit_tool will keep its .config
0086 around, so you can see what config was used after running ``kunit.py run``.
0087 It also preserves any config changes you might make, so you can
0088 enable/disable things with ``make ARCH=um menuconfig`` or similar, and then
0089 re-run kunit_tool.
0090 5. Try to run ``make ARCH=um defconfig`` before running ``kunit.py run``. This
0091 may help clean up any residual config items which could be causing problems.
0092 6. Finally, try running KUnit outside UML. KUnit and KUnit tests can be
0093 built into any kernel, or can be built as a module and loaded at runtime.
0094 Doing so should allow you to determine if UML is causing the issue you're
0095 seeing. When tests are built-in, they will execute when the kernel boots, and
0096 modules will automatically execute associated tests when loaded. Test results
0097 can be collected from ``/sys/kernel/debug/kunit/<test suite>/results``, and
0098 can be parsed with ``kunit.py parse``. For more details, see "KUnit on
0099 non-UML architectures" in Documentation/dev-tools/kunit/usage.rst.
0100
0101 If none of the above tricks help, you are always welcome to email any issues to
0102 kunit-dev@googlegroups.com.
