0001 ===================
0002 Testing BPF on s390
0003 ===================
0004
0005 1. Introduction
0006 ***************
0007
0008 IBM Z are mainframe computers, which are descendants of IBM System/360 from
0009 year 1964. They are supported by the Linux kernel under the name "s390". This
0010 document describes how to test BPF in an s390 QEMU guest.
0011
0012 2. One-time setup
0013 *****************
0014
0015 The following is required to build and run the test suite:
0016
0017 * s390 GCC
0018 * s390 development headers and libraries
0019 * Clang with BPF support
0020 * QEMU with s390 support
0021 * Disk image with s390 rootfs
0022
0023 Debian supports installing compiler and libraries for s390 out of the box.
0024 Users of other distros may use debootstrap in order to set up a Debian chroot::
0025
0026 sudo debootstrap \
0027 --variant=minbase \
0028 --include=sudo \
0029 testing \
0030 ./s390-toolchain
0031 sudo mount --rbind /dev ./s390-toolchain/dev
0032 sudo mount --rbind /proc ./s390-toolchain/proc
0033 sudo mount --rbind /sys ./s390-toolchain/sys
0034 sudo chroot ./s390-toolchain
0035
0036 Once on Debian, the build prerequisites can be installed as follows::
0037
0038 sudo dpkg --add-architecture s390x
0039 sudo apt-get update
0040 sudo apt-get install \
0041 bc \
0042 bison \
0043 cmake \
0044 debootstrap \
0045 dwarves \
0046 flex \
0047 g++ \
0048 gcc \
0049 g++-s390x-linux-gnu \
0050 gcc-s390x-linux-gnu \
0051 gdb-multiarch \
0052 git \
0053 make \
0054 python3 \
0055 qemu-system-misc \
0056 qemu-utils \
0057 rsync \
0058 libcap-dev:s390x \
0059 libelf-dev:s390x \
0060 libncurses-dev
0061
0062 Latest Clang targeting BPF can be installed as follows::
0063
0064 git clone https://github.com/llvm/llvm-project.git
0065 ln -s ../../clang llvm-project/llvm/tools/
0066 mkdir llvm-project-build
0067 cd llvm-project-build
0068 cmake \
0069 -DLLVM_TARGETS_TO_BUILD=BPF \
0070 -DCMAKE_BUILD_TYPE=Release \
0071 -DCMAKE_INSTALL_PREFIX=/opt/clang-bpf \
0072 ../llvm-project/llvm
0073 make
0074 sudo make install
0075 export PATH=/opt/clang-bpf/bin:$PATH
0076
0077 The disk image can be prepared using a loopback mount and debootstrap::
0078
0079 qemu-img create -f raw ./s390.img 1G
0080 sudo losetup -f ./s390.img
0081 sudo mkfs.ext4 /dev/loopX
0082 mkdir ./s390.rootfs
0083 sudo mount /dev/loopX ./s390.rootfs
0084 sudo debootstrap \
0085 --foreign \
0086 --arch=s390x \
0087 --variant=minbase \
0088 --include=" \
0089 iproute2, \
0090 iputils-ping, \
0091 isc-dhcp-client, \
0092 kmod, \
0093 libcap2, \
0094 libelf1, \
0095 netcat, \
0096 procps" \
0097 testing \
0098 ./s390.rootfs
0099 sudo umount ./s390.rootfs
0100 sudo losetup -d /dev/loopX
0101
0102 3. Compilation
0103 **************
0104
0105 In addition to the usual Kconfig options required to run the BPF test suite, it
0106 is also helpful to select::
0107
0108 CONFIG_NET_9P=y
0109 CONFIG_9P_FS=y
0110 CONFIG_NET_9P_VIRTIO=y
0111 CONFIG_VIRTIO_PCI=y
0112
0113 as that would enable a very easy way to share files with the s390 virtual
0114 machine.
0115
0116 Compiling kernel, modules and testsuite, as well as preparing gdb scripts to
0117 simplify debugging, can be done using the following commands::
0118
0119 make ARCH=s390 CROSS_COMPILE=s390x-linux-gnu- menuconfig
0120 make ARCH=s390 CROSS_COMPILE=s390x-linux-gnu- bzImage modules scripts_gdb
0121 make ARCH=s390 CROSS_COMPILE=s390x-linux-gnu- \
0122 -C tools/testing/selftests \
0123 TARGETS=bpf \
0124 INSTALL_PATH=$PWD/tools/testing/selftests/kselftest_install \
0125 install
0126
0127 4. Running the test suite
0128 *************************
0129
0130 The virtual machine can be started as follows::
0131
0132 qemu-system-s390x \
0133 -cpu max,zpci=on \
0134 -smp 2 \
0135 -m 4G \
0136 -kernel linux/arch/s390/boot/compressed/vmlinux \
0137 -drive file=./s390.img,if=virtio,format=raw \
0138 -nographic \
0139 -append 'root=/dev/vda rw console=ttyS1' \
0140 -virtfs local,path=./linux,security_model=none,mount_tag=linux \
0141 -object rng-random,filename=/dev/urandom,id=rng0 \
0142 -device virtio-rng-ccw,rng=rng0 \
0143 -netdev user,id=net0 \
0144 -device virtio-net-ccw,netdev=net0
0145
0146 When using this on a real IBM Z, ``-enable-kvm`` may be added for better
0147 performance. When starting the virtual machine for the first time, disk image
0148 setup must be finalized using the following command::
0149
0150 /debootstrap/debootstrap --second-stage
0151
0152 Directory with the code built on the host as well as ``/proc`` and ``/sys``
0153 need to be mounted as follows::
0154
0155 mkdir -p /linux
0156 mount -t 9p linux /linux
0157 mount -t proc proc /proc
0158 mount -t sysfs sys /sys
0159
0160 After that, the test suite can be run using the following commands::
0161
0162 cd /linux/tools/testing/selftests/kselftest_install
0163 ./run_kselftest.sh
0164
0165 As usual, tests can be also run individually::
0166
0167 cd /linux/tools/testing/selftests/bpf
0168 ./test_verifier
0169
0170 5. Debugging
0171 ************
0172
0173 It is possible to debug the s390 kernel using QEMU GDB stub, which is activated
0174 by passing ``-s`` to QEMU.
0175
0176 It is preferable to turn KASLR off, so that gdb would know where to find the
0177 kernel image in memory, by building the kernel with::
0178
0179 RANDOMIZE_BASE=n
0180
0181 GDB can then be attached using the following command::
0182
0183 gdb-multiarch -ex 'target remote localhost:1234' ./vmlinux
0184
0185 6. Network
0186 **********
0187
0188 In case one needs to use the network in the virtual machine in order to e.g.
0189 install additional packages, it can be configured using::
0190
0191 dhclient eth0
0192
0193 7. Links
0194 ********
0195
0196 This document is a compilation of techniques, whose more comprehensive
0197 descriptions can be found by following these links:
0198
0199 - `Debootstrap <https://wiki.debian.org/EmDebian/CrossDebootstrap>`_
0200 - `Multiarch <https://wiki.debian.org/Multiarch/HOWTO>`_
0201 - `Building LLVM <https://llvm.org/docs/CMake.html>`_
0202 - `Cross-compiling the kernel <https://wiki.gentoo.org/wiki/Embedded_Handbook/General/Cross-compiling_the_kernel>`_
0203 - `QEMU s390x Guest Support <https://wiki.qemu.org/Documentation/Platforms/S390X>`_
0204 - `Plan 9 folder sharing over Virtio <https://wiki.qemu.org/Documentation/9psetup>`_
0205 - `Using GDB with QEMU <https://wiki.osdev.org/Kernel_Debugging#Use_GDB_with_QEMU>`_
