0001 ==================
0002 BPF Selftest Notes
0003 ==================
0004 General instructions on running selftests can be found in
0005 `Documentation/bpf/bpf_devel_QA.rst`__.
0006
0007 __ /Documentation/bpf/bpf_devel_QA.rst#q-how-to-run-bpf-selftests
0008
0009 =========================
0010 Running Selftests in a VM
0011 =========================
0012
0013 It's now possible to run the selftests using ``tools/testing/selftests/bpf/vmtest.sh``.
0014 The script tries to ensure that the tests are run with the same environment as they
0015 would be run post-submit in the CI used by the Maintainers.
0016
0017 This script downloads a suitable Kconfig and VM userspace image from the system used by
0018 the CI. It builds the kernel (without overwriting your existing Kconfig), recompiles the
0019 bpf selftests, runs them (by default ``tools/testing/selftests/bpf/test_progs``) and
0020 saves the resulting output (by default in ``~/.bpf_selftests``).
0021
0022 Script dependencies:
0023 - clang (preferably built from sources, https://github.com/llvm/llvm-project);
0024 - pahole (preferably built from sources, https://git.kernel.org/pub/scm/devel/pahole/pahole.git/);
0025 - qemu;
0026 - docutils (for ``rst2man``);
0027 - libcap-devel.
0028
0029 For more information on about using the script, run:
0030
0031 .. code-block:: console
0032
0033 $ tools/testing/selftests/bpf/vmtest.sh -h
0034
0035 In case of linker errors when running selftests, try using static linking:
0036
0037 .. code-block:: console
0038
0039 $ LDLIBS=-static vmtest.sh
0040
0041 .. note:: Some distros may not support static linking.
0042
0043 .. note:: The script uses pahole and clang based on host environment setting.
0044 If you want to change pahole and llvm, you can change `PATH` environment
0045 variable in the beginning of script.
0046
0047 .. note:: The script currently only supports x86_64 and s390x architectures.
0048
0049 Additional information about selftest failures are
0050 documented here.
0051
0052 profiler[23] test failures with clang/llvm <12.0.0
0053 ==================================================
0054
0055 With clang/llvm <12.0.0, the profiler[23] test may fail.
0056 The symptom looks like
0057
0058 .. code-block:: c
0059
0060 // r9 is a pointer to map_value
0061 // r7 is a scalar
0062 17: bf 96 00 00 00 00 00 00 r6 = r9
0063 18: 0f 76 00 00 00 00 00 00 r6 += r7
0064 math between map_value pointer and register with unbounded min value is not allowed
0065
0066 // the instructions below will not be seen in the verifier log
0067 19: a5 07 01 00 01 01 00 00 if r7 < 257 goto +1
0068 20: bf 96 00 00 00 00 00 00 r6 = r9
0069 // r6 is used here
0070
0071 The verifier will reject such code with above error.
0072 At insn 18 the r7 is indeed unbounded. The later insn 19 checks the bounds and
0073 the insn 20 undoes map_value addition. It is currently impossible for the
0074 verifier to understand such speculative pointer arithmetic.
0075 Hence `this patch`__ addresses it on the compiler side. It was committed on llvm 12.
0076
0077 __ https://reviews.llvm.org/D85570
0078
0079 The corresponding C code
0080
0081 .. code-block:: c
0082
0083 for (int i = 0; i < MAX_CGROUPS_PATH_DEPTH; i++) {
0084 filepart_length = bpf_probe_read_str(payload, ...);
0085 if (filepart_length <= MAX_PATH) {
0086 barrier_var(filepart_length); // workaround
0087 payload += filepart_length;
0088 }
0089 }
0090
0091 bpf_iter test failures with clang/llvm 10.0.0
0092 =============================================
0093
0094 With clang/llvm 10.0.0, the following two bpf_iter tests failed:
0095 * ``bpf_iter/ipv6_route``
0096 * ``bpf_iter/netlink``
0097
0098 The symptom for ``bpf_iter/ipv6_route`` looks like
0099
0100 .. code-block:: c
0101
0102 2: (79) r8 = *(u64 *)(r1 +8)
0103 ...
0104 14: (bf) r2 = r8
0105 15: (0f) r2 += r1
0106 ; BPF_SEQ_PRINTF(seq, "%pi6 %02x ", &rt->fib6_dst.addr, rt->fib6_dst.plen);
0107 16: (7b) *(u64 *)(r8 +64) = r2
0108 only read is supported
0109
0110 The symptom for ``bpf_iter/netlink`` looks like
0111
0112 .. code-block:: c
0113
0114 ; struct netlink_sock *nlk = ctx->sk;
0115 2: (79) r7 = *(u64 *)(r1 +8)
0116 ...
0117 15: (bf) r2 = r7
0118 16: (0f) r2 += r1
0119 ; BPF_SEQ_PRINTF(seq, "%pK %-3d ", s, s->sk_protocol);
0120 17: (7b) *(u64 *)(r7 +0) = r2
0121 only read is supported
0122
0123 This is due to a llvm BPF backend bug. `The fix`__
0124 has been pushed to llvm 10.x release branch and will be
0125 available in 10.0.1. The patch is available in llvm 11.0.0 trunk.
0126
0127 __ https://reviews.llvm.org/D78466
0128
0129 bpf_verif_scale/loop6.o test failure with Clang 12
0130 ==================================================
0131
0132 With Clang 12, the following bpf_verif_scale test failed:
0133 * ``bpf_verif_scale/loop6.o``
0134
0135 The verifier output looks like
0136
0137 .. code-block:: c
0138
0139 R1 type=ctx expected=fp
0140 The sequence of 8193 jumps is too complex.
0141
0142 The reason is compiler generating the following code
0143
0144 .. code-block:: c
0145
0146 ; for (i = 0; (i < VIRTIO_MAX_SGS) && (i < num); i++) {
0147 14: 16 05 40 00 00 00 00 00 if w5 == 0 goto +64 <LBB0_6>
0148 15: bc 51 00 00 00 00 00 00 w1 = w5
0149 16: 04 01 00 00 ff ff ff ff w1 += -1
0150 17: 67 05 00 00 20 00 00 00 r5 <<= 32
0151 18: 77 05 00 00 20 00 00 00 r5 >>= 32
0152 19: a6 01 01 00 05 00 00 00 if w1 < 5 goto +1 <LBB0_4>
0153 20: b7 05 00 00 06 00 00 00 r5 = 6
0154 00000000000000a8 <LBB0_4>:
0155 21: b7 02 00 00 00 00 00 00 r2 = 0
0156 22: b7 01 00 00 00 00 00 00 r1 = 0
0157 ; for (i = 0; (i < VIRTIO_MAX_SGS) && (i < num); i++) {
0158 23: 7b 1a e0 ff 00 00 00 00 *(u64 *)(r10 - 32) = r1
0159 24: 7b 5a c0 ff 00 00 00 00 *(u64 *)(r10 - 64) = r5
0160
0161 Note that insn #15 has w1 = w5 and w1 is refined later but
0162 r5(w5) is eventually saved on stack at insn #24 for later use.
0163 This cause later verifier failure. The bug has been `fixed`__ in
0164 Clang 13.
0165
0166 __ https://reviews.llvm.org/D97479
0167
0168 BPF CO-RE-based tests and Clang version
0169 =======================================
0170
0171 A set of selftests use BPF target-specific built-ins, which might require
0172 bleeding-edge Clang versions (Clang 12 nightly at this time).
0173
0174 Few sub-tests of core_reloc test suit (part of test_progs test runner) require
0175 the following built-ins, listed with corresponding Clang diffs introducing
0176 them to Clang/LLVM. These sub-tests are going to be skipped if Clang is too
0177 old to support them, they shouldn't cause build failures or runtime test
0178 failures:
0179
0180 - __builtin_btf_type_id() [0_, 1_, 2_];
0181 - __builtin_preserve_type_info(), __builtin_preserve_enum_value() [3_, 4_].
0182
0183 .. _0: https://reviews.llvm.org/D74572
0184 .. _1: https://reviews.llvm.org/D74668
0185 .. _2: https://reviews.llvm.org/D85174
0186 .. _3: https://reviews.llvm.org/D83878
0187 .. _4: https://reviews.llvm.org/D83242
0188
0189 Floating-point tests and Clang version
0190 ======================================
0191
0192 Certain selftests, e.g. core_reloc, require support for the floating-point
0193 types, which was introduced in `Clang 13`__. The older Clang versions will
0194 either crash when compiling these tests, or generate an incorrect BTF.
0195
0196 __ https://reviews.llvm.org/D83289
0197
0198 Kernel function call test and Clang version
0199 ===========================================
0200
0201 Some selftests (e.g. kfunc_call and bpf_tcp_ca) require a LLVM support
0202 to generate extern function in BTF. It was introduced in `Clang 13`__.
0203
0204 Without it, the error from compiling bpf selftests looks like:
0205
0206 .. code-block:: console
0207
0208 libbpf: failed to find BTF for extern 'tcp_slow_start' [25] section: -2
0209
0210 __ https://reviews.llvm.org/D93563
0211
0212 btf_tag test and Clang version
0213 ==============================
0214
0215 The btf_tag selftest requires LLVM support to recognize the btf_decl_tag and
0216 btf_type_tag attributes. They are introduced in `Clang 14` [0_, 1_].
0217 The subtests ``btf_type_tag_user_{mod1, mod2, vmlinux}`` also requires
0218 pahole version ``1.23``.
0219
0220 Without them, the btf_tag selftest will be skipped and you will observe:
0221
0222 .. code-block:: console
0223
0224 #<test_num> btf_tag:SKIP
0225
0226 .. _0: https://reviews.llvm.org/D111588
0227 .. _1: https://reviews.llvm.org/D111199
0228
0229 Clang dependencies for static linking tests
0230 ===========================================
0231
0232 linked_vars, linked_maps, and linked_funcs tests depend on `Clang fix`__ to
0233 generate valid BTF information for weak variables. Please make sure you use
0234 Clang that contains the fix.
0235
0236 __ https://reviews.llvm.org/D100362
0237
0238 Clang relocation changes
0239 ========================
0240
0241 Clang 13 patch `clang reloc patch`_ made some changes on relocations such
0242 that existing relocation types are broken into more types and
0243 each new type corresponds to only one way to resolve relocation.
0244 See `kernel llvm reloc`_ for more explanation and some examples.
0245 Using clang 13 to compile old libbpf which has static linker support,
0246 there will be a compilation failure::
0247
0248 libbpf: ELF relo #0 in section #6 has unexpected type 2 in .../bpf_tcp_nogpl.o
0249
0250 Here, ``type 2`` refers to new relocation type ``R_BPF_64_ABS64``.
0251 To fix this issue, user newer libbpf.
0252
0253 .. Links
0254 .. _clang reloc patch: https://reviews.llvm.org/D102712
0255 .. _kernel llvm reloc: /Documentation/bpf/llvm_reloc.rst
0256
0257 Clang dependencies for the u32 spill test (xdpwall)
0258 ===================================================
0259 The xdpwall selftest requires a change in `Clang 14`__.
0260
0261 Without it, the xdpwall selftest will fail and the error message
0262 from running test_progs will look like:
0263
0264 .. code-block:: console
0265
0266 test_xdpwall:FAIL:Does LLVM have https://reviews.llvm.org/D109073? unexpected error: -4007
0267
0268 __ https://reviews.llvm.org/D109073