0001 ==================================
0002 Kernel Lock Torture Test Operation
0003 ==================================
0004
0005 CONFIG_LOCK_TORTURE_TEST
0006 ========================
0007
0008 The CONFIG LOCK_TORTURE_TEST config option provides a kernel module
0009 that runs torture tests on core kernel locking primitives. The kernel
0010 module, 'locktorture', may be built after the fact on the running
0011 kernel to be tested, if desired. The tests periodically output status
0012 messages via printk(), which can be examined via the dmesg (perhaps
0013 grepping for "torture"). The test is started when the module is loaded,
0014 and stops when the module is unloaded. This program is based on how RCU
0015 is tortured, via rcutorture.
0016
0017 This torture test consists of creating a number of kernel threads which
0018 acquire the lock and hold it for specific amount of time, thus simulating
0019 different critical region behaviors. The amount of contention on the lock
0020 can be simulated by either enlarging this critical region hold time and/or
0021 creating more kthreads.
0022
0023
0024 Module Parameters
0025 =================
0026
0027 This module has the following parameters:
0028
0029
0030 Locktorture-specific
0031 --------------------
0032
0033 nwriters_stress
0034 Number of kernel threads that will stress exclusive lock
0035 ownership (writers). The default value is twice the number
0036 of online CPUs.
0037
0038 nreaders_stress
0039 Number of kernel threads that will stress shared lock
0040 ownership (readers). The default is the same amount of writer
0041 locks. If the user did not specify nwriters_stress, then
0042 both readers and writers be the amount of online CPUs.
0043
0044 torture_type
0045 Type of lock to torture. By default, only spinlocks will
0046 be tortured. This module can torture the following locks,
0047 with string values as follows:
0048
0049 - "lock_busted":
0050 Simulates a buggy lock implementation.
0051
0052 - "spin_lock":
0053 spin_lock() and spin_unlock() pairs.
0054
0055 - "spin_lock_irq":
0056 spin_lock_irq() and spin_unlock_irq() pairs.
0057
0058 - "rw_lock":
0059 read/write lock() and unlock() rwlock pairs.
0060
0061 - "rw_lock_irq":
0062 read/write lock_irq() and unlock_irq()
0063 rwlock pairs.
0064
0065 - "mutex_lock":
0066 mutex_lock() and mutex_unlock() pairs.
0067
0068 - "rtmutex_lock":
0069 rtmutex_lock() and rtmutex_unlock() pairs.
0070 Kernel must have CONFIG_RT_MUTEX=y.
0071
0072 - "rwsem_lock":
0073 read/write down() and up() semaphore pairs.
0074
0075
0076 Torture-framework (RCU + locking)
0077 ---------------------------------
0078
0079 shutdown_secs
0080 The number of seconds to run the test before terminating
0081 the test and powering off the system. The default is
0082 zero, which disables test termination and system shutdown.
0083 This capability is useful for automated testing.
0084
0085 onoff_interval
0086 The number of seconds between each attempt to execute a
0087 randomly selected CPU-hotplug operation. Defaults
0088 to zero, which disables CPU hotplugging. In
0089 CONFIG_HOTPLUG_CPU=n kernels, locktorture will silently
0090 refuse to do any CPU-hotplug operations regardless of
0091 what value is specified for onoff_interval.
0092
0093 onoff_holdoff
0094 The number of seconds to wait until starting CPU-hotplug
0095 operations. This would normally only be used when
0096 locktorture was built into the kernel and started
0097 automatically at boot time, in which case it is useful
0098 in order to avoid confusing boot-time code with CPUs
0099 coming and going. This parameter is only useful if
0100 CONFIG_HOTPLUG_CPU is enabled.
0101
0102 stat_interval
0103 Number of seconds between statistics-related printk()s.
0104 By default, locktorture will report stats every 60 seconds.
0105 Setting the interval to zero causes the statistics to
0106 be printed -only- when the module is unloaded.
0107
0108 stutter
0109 The length of time to run the test before pausing for this
0110 same period of time. Defaults to "stutter=5", so as
0111 to run and pause for (roughly) five-second intervals.
0112 Specifying "stutter=0" causes the test to run continuously
0113 without pausing.
0114
0115 shuffle_interval
0116 The number of seconds to keep the test threads affinitied
0117 to a particular subset of the CPUs, defaults to 3 seconds.
0118 Used in conjunction with test_no_idle_hz.
0119
0120 verbose
0121 Enable verbose debugging printing, via printk(). Enabled
0122 by default. This extra information is mostly related to
0123 high-level errors and reports from the main 'torture'
0124 framework.
0125
0126
0127 Statistics
0128 ==========
0129
0130 Statistics are printed in the following format::
0131
0132 spin_lock-torture: Writes: Total: 93746064 Max/Min: 0/0 Fail: 0
0133 (A) (B) (C) (D) (E)
0134
0135 (A): Lock type that is being tortured -- torture_type parameter.
0136
0137 (B): Number of writer lock acquisitions. If dealing with a read/write
0138 primitive a second "Reads" statistics line is printed.
0139
0140 (C): Number of times the lock was acquired.
0141
0142 (D): Min and max number of times threads failed to acquire the lock.
0143
0144 (E): true/false values if there were errors acquiring the lock. This should
0145 -only- be positive if there is a bug in the locking primitive's
0146 implementation. Otherwise a lock should never fail (i.e., spin_lock()).
0147 Of course, the same applies for (C), above. A dummy example of this is
0148 the "lock_busted" type.
0149
0150 Usage
0151 =====
0152
0153 The following script may be used to torture locks::
0154
0155 #!/bin/sh
0156
0157 modprobe locktorture
0158 sleep 3600
0159 rmmod locktorture
0160 dmesg | grep torture:
0161
0162 The output can be manually inspected for the error flag of "!!!".
0163 One could of course create a more elaborate script that automatically
0164 checked for such errors. The "rmmod" command forces a "SUCCESS",
0165 "FAILURE", or "RCU_HOTPLUG" indication to be printk()ed. The first
0166 two are self-explanatory, while the last indicates that while there
0167 were no locking failures, CPU-hotplug problems were detected.
0168
0169 Also see: Documentation/RCU/torture.rst
