Back to home page

OSCL-LXR

 
 

    


0001 perf-probe(1)
0002 =============
0003 
0004 NAME
0005 ----
0006 perf-probe - Define new dynamic tracepoints
0007 
0008 SYNOPSIS
0009 --------
0010 [verse]
0011 'perf probe' [options] --add='PROBE' [...]
0012 or
0013 'perf probe' [options] PROBE
0014 or
0015 'perf probe' [options] --del='[GROUP:]EVENT' [...]
0016 or
0017 'perf probe' --list[=[GROUP:]EVENT]
0018 or
0019 'perf probe' [options] --line='LINE'
0020 or
0021 'perf probe' [options] --vars='PROBEPOINT'
0022 or
0023 'perf probe' [options] --funcs
0024 or
0025 'perf probe' [options] --definition='PROBE' [...]
0026 
0027 DESCRIPTION
0028 -----------
0029 This command defines dynamic tracepoint events, by symbol and registers
0030 without debuginfo, or by C expressions (C line numbers, C function names,
0031 and C local variables) with debuginfo.
0032 
0033 
0034 OPTIONS
0035 -------
0036 -k::
0037 --vmlinux=PATH::
0038         Specify vmlinux path which has debuginfo (Dwarf binary).
0039         Only when using this with --definition, you can give an offline
0040         vmlinux file.
0041 
0042 -m::
0043 --module=MODNAME|PATH::
0044         Specify module name in which perf-probe searches probe points
0045         or lines. If a path of module file is passed, perf-probe
0046         treat it as an offline module (this means you can add a probe on
0047         a module which has not been loaded yet).
0048 
0049 -s::
0050 --source=PATH::
0051         Specify path to kernel source.
0052 
0053 -v::
0054 --verbose::
0055         Be more verbose (show parsed arguments, etc).
0056         Can not use with -q.
0057 
0058 -q::
0059 --quiet::
0060         Be quiet (do not show any messages including errors).
0061         Can not use with -v.
0062 
0063 -a::
0064 --add=::
0065         Define a probe event (see PROBE SYNTAX for detail).
0066 
0067 -d::
0068 --del=::
0069         Delete probe events. This accepts glob wildcards('*', '?') and character
0070         classes(e.g. [a-z], [!A-Z]).
0071 
0072 -l::
0073 --list[=[GROUP:]EVENT]::
0074         List up current probe events. This can also accept filtering patterns of
0075         event names.
0076         When this is used with --cache, perf shows all cached probes instead of
0077         the live probes.
0078 
0079 -L::
0080 --line=::
0081         Show source code lines which can be probed. This needs an argument
0082         which specifies a range of the source code. (see LINE SYNTAX for detail)
0083 
0084 -V::
0085 --vars=::
0086         Show available local variables at given probe point. The argument
0087         syntax is same as PROBE SYNTAX, but NO ARGs.
0088 
0089 --externs::
0090         (Only for --vars) Show external defined variables in addition to local
0091         variables.
0092 
0093 --no-inlines::
0094         (Only for --add) Search only for non-inlined functions. The functions
0095         which do not have instances are ignored.
0096 
0097 -F::
0098 --funcs[=FILTER]::
0099         Show available functions in given module or kernel. With -x/--exec,
0100         can also list functions in a user space executable / shared library.
0101         This also can accept a FILTER rule argument.
0102 
0103 -D::
0104 --definition=::
0105         Show trace-event definition converted from given probe-event instead
0106         of write it into tracing/[k,u]probe_events.
0107 
0108 --filter=FILTER::
0109         (Only for --vars and --funcs) Set filter. FILTER is a combination of glob
0110         pattern, see FILTER PATTERN for detail.
0111         Default FILTER is "!__k???tab_* & !__crc_*" for --vars, and "!_*"
0112         for --funcs.
0113         If several filters are specified, only the last filter is used.
0114 
0115 -f::
0116 --force::
0117         Forcibly add events with existing name.
0118 
0119 -n::
0120 --dry-run::
0121         Dry run. With this option, --add and --del doesn't execute actual
0122         adding and removal operations.
0123 
0124 --cache::
0125         (With --add) Cache the probes. Any events which successfully added
0126         are also stored in the cache file.
0127         (With --list) Show cached probes.
0128         (With --del) Remove cached probes.
0129 
0130 --max-probes=NUM::
0131         Set the maximum number of probe points for an event. Default is 128.
0132 
0133 --target-ns=PID:
0134         Obtain mount namespace information from the target pid.  This is
0135         used when creating a uprobe for a process that resides in a
0136         different mount namespace from the perf(1) utility.
0137 
0138 -x::
0139 --exec=PATH::
0140         Specify path to the executable or shared library file for user
0141         space tracing. Can also be used with --funcs option.
0142 
0143 --demangle::
0144         Demangle application symbols. --no-demangle is also available
0145         for disabling demangling.
0146 
0147 --demangle-kernel::
0148         Demangle kernel symbols. --no-demangle-kernel is also available
0149         for disabling kernel demangling.
0150 
0151 In absence of -m/-x options, perf probe checks if the first argument after
0152 the options is an absolute path name. If its an absolute path, perf probe
0153 uses it as a target module/target user space binary to probe.
0154 
0155 PROBE SYNTAX
0156 ------------
0157 Probe points are defined by following syntax.
0158 
0159     1) Define event based on function name
0160      [[GROUP:]EVENT=]FUNC[@SRC][:RLN|+OFFS|%return|;PTN] [ARG ...]
0161 
0162     2) Define event based on source file with line number
0163      [[GROUP:]EVENT=]SRC:ALN [ARG ...]
0164 
0165     3) Define event based on source file with lazy pattern
0166      [[GROUP:]EVENT=]SRC;PTN [ARG ...]
0167 
0168     4) Pre-defined SDT events or cached event with name
0169      %[sdt_PROVIDER:]SDTEVENT
0170      or,
0171      sdt_PROVIDER:SDTEVENT
0172 
0173 'EVENT' specifies the name of new event, if omitted, it will be set the name of the probed function, and for return probes, a "\_\_return" suffix is automatically added to the function name. You can also specify a group name by 'GROUP', if omitted, set 'probe' is used for kprobe and 'probe_<bin>' is used for uprobe.
0174 Note that using existing group name can conflict with other events. Especially, using the group name reserved for kernel modules can hide embedded events in the
0175 modules.
0176 'FUNC' specifies a probed function name, and it may have one of the following options; '+OFFS' is the offset from function entry address in bytes, ':RLN' is the relative-line number from function entry line, and '%return' means that it probes function return. And ';PTN' means lazy matching pattern (see LAZY MATCHING). Note that ';PTN' must be the end of the probe point definition.  In addition, '@SRC' specifies a source file which has that function.
0177 It is also possible to specify a probe point by the source line number or lazy matching by using 'SRC:ALN' or 'SRC;PTN' syntax, where 'SRC' is the source file path, ':ALN' is the line number and ';PTN' is the lazy matching pattern.
0178 'ARG' specifies the arguments of this probe point, (see PROBE ARGUMENT).
0179 'SDTEVENT' and 'PROVIDER' is the pre-defined event name which is defined by user SDT (Statically Defined Tracing) or the pre-cached probes with event name.
0180 Note that before using the SDT event, the target binary (on which SDT events are defined) must be scanned by linkperf:perf-buildid-cache[1] to make SDT events as cached events.
0181 
0182 For details of the SDT, see below.
0183 https://sourceware.org/gdb/onlinedocs/gdb/Static-Probe-Points.html
0184 
0185 ESCAPED CHARACTER
0186 -----------------
0187 
0188 In the probe syntax, '=', '@', '+', ':' and ';' are treated as a special character. You can use a backslash ('\') to escape the special characters.
0189 This is useful if you need to probe on a specific versioned symbols, like @GLIBC_... suffixes, or also you need to specify a source file which includes the special characters.
0190 Note that usually single backslash is consumed by shell, so you might need to pass double backslash (\\) or wrapping with single quotes (\'AAA\@BBB').
0191 See EXAMPLES how it is used.
0192 
0193 PROBE ARGUMENT
0194 --------------
0195 Each probe argument follows below syntax.
0196 
0197  [NAME=]LOCALVAR|$retval|%REG|@SYMBOL[:TYPE][@user]
0198 
0199 'NAME' specifies the name of this argument (optional). You can use the name of local variable, local data structure member (e.g. var->field, var.field2), local array with fixed index (e.g. array[1], var->array[0], var->pointer[2]), or kprobe-tracer argument format (e.g. $retval, %ax, etc). Note that the name of this argument will be set as the last member name if you specify a local data structure member (e.g. field2 for 'var->field1.field2'.)
0200 '$vars' and '$params' special arguments are also available for NAME, '$vars' is expanded to the local variables (including function parameters) which can access at given probe point. '$params' is expanded to only the function parameters.
0201 'TYPE' casts the type of this argument (optional). If omitted, perf probe automatically set the type based on debuginfo (*). Currently, basic types (u8/u16/u32/u64/s8/s16/s32/s64), hexadecimal integers (x/x8/x16/x32/x64), signedness casting (u/s), "string" and bitfield are supported. (see TYPES for detail)
0202 On x86 systems %REG is always the short form of the register: for example %AX. %RAX or %EAX is not valid.
0203 "@user" is a special attribute which means the LOCALVAR will be treated as a user-space memory. This is only valid for kprobe event.
0204 
0205 TYPES
0206 -----
0207 Basic types (u8/u16/u32/u64/s8/s16/s32/s64) and hexadecimal integers (x8/x16/x32/x64) are integer types. Prefix 's' and 'u' means those types are signed and unsigned respectively, and 'x' means that is shown in hexadecimal format. Traced arguments are shown in decimal (sNN/uNN) or hex (xNN). You can also use 's' or 'u' to specify only signedness and leave its size auto-detected by perf probe. Moreover, you can use 'x' to explicitly specify to be shown in hexadecimal (the size is also auto-detected).
0208 String type is a special type, which fetches a "null-terminated" string from kernel space. This means it will fail and store NULL if the string container has been paged out. You can specify 'string' type only for the local variable or structure member which is an array of or a pointer to 'char' or 'unsigned char' type.
0209 Bitfield is another special type, which takes 3 parameters, bit-width, bit-offset, and container-size (usually 32). The syntax is;
0210 
0211  b<bit-width>@<bit-offset>/<container-size>
0212 
0213 LINE SYNTAX
0214 -----------
0215 Line range is described by following syntax.
0216 
0217  "FUNC[@SRC][:RLN[+NUM|-RLN2]]|SRC[:ALN[+NUM|-ALN2]]"
0218 
0219 FUNC specifies the function name of showing lines. 'RLN' is the start line
0220 number from function entry line, and 'RLN2' is the end line number. As same as
0221 probe syntax, 'SRC' means the source file path, 'ALN' is start line number,
0222 and 'ALN2' is end line number in the file. It is also possible to specify how
0223 many lines to show by using 'NUM'. Moreover, 'FUNC@SRC' combination is good
0224 for searching a specific function when several functions share same name.
0225 So, "source.c:100-120" shows lines between 100th to l20th in source.c file. And "func:10+20" shows 20 lines from 10th line of func function.
0226 
0227 LAZY MATCHING
0228 -------------
0229 The lazy line matching is similar to glob matching but ignoring spaces in both of pattern and target. So this accepts wildcards('*', '?') and character classes(e.g. [a-z], [!A-Z]).
0230 
0231 e.g.
0232  'a=*' can matches 'a=b', 'a = b', 'a == b' and so on.
0233 
0234 This provides some sort of flexibility and robustness to probe point definitions against minor code changes. For example, actual 10th line of schedule() can be moved easily by modifying schedule(), but the same line matching 'rq=cpu_rq*' may still exist in the function.)
0235 
0236 FILTER PATTERN
0237 --------------
0238 The filter pattern is a glob matching pattern(s) to filter variables.
0239 In addition, you can use "!" for specifying filter-out rule. You also can give several rules combined with "&" or "|", and fold those rules as one rule by using "(" ")".
0240 
0241 e.g.
0242  With --filter "foo* | bar*", perf probe -V shows variables which start with "foo" or "bar".
0243  With --filter "!foo* & *bar", perf probe -V shows variables which don't start with "foo" and end with "bar", like "fizzbar". But "foobar" is filtered out.
0244 
0245 EXAMPLES
0246 --------
0247 Display which lines in schedule() can be probed:
0248 
0249  ./perf probe --line schedule
0250 
0251 Add a probe on schedule() function 12th line with recording cpu local variable:
0252 
0253  ./perf probe schedule:12 cpu
0254  or
0255  ./perf probe --add='schedule:12 cpu'
0256 
0257 Add one or more probes which has the name start with "schedule".
0258 
0259  ./perf probe schedule*
0260  or
0261  ./perf probe --add='schedule*'
0262 
0263 Add probes on lines in schedule() function which calls update_rq_clock().
0264 
0265  ./perf probe 'schedule;update_rq_clock*'
0266  or
0267  ./perf probe --add='schedule;update_rq_clock*'
0268 
0269 Delete all probes on schedule().
0270 
0271  ./perf probe --del='schedule*'
0272 
0273 Add probes at zfree() function on /bin/zsh
0274 
0275  ./perf probe -x /bin/zsh zfree or ./perf probe /bin/zsh zfree
0276 
0277 Add probes at malloc() function on libc
0278 
0279  ./perf probe -x /lib/libc.so.6 malloc or ./perf probe /lib/libc.so.6 malloc
0280 
0281 Add a uprobe to a target process running in a different mount namespace
0282 
0283  ./perf probe --target-ns <target pid> -x /lib64/libc.so.6 malloc
0284 
0285 Add a USDT probe to a target process running in a different mount namespace
0286 
0287  ./perf probe --target-ns <target pid> -x /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.121-0.b13.el7_3.x86_64/jre/lib/amd64/server/libjvm.so %sdt_hotspot:thread__sleep__end
0288 
0289 Add a probe on specific versioned symbol by backslash escape
0290 
0291  ./perf probe -x /lib64/libc-2.25.so 'malloc_get_state\@GLIBC_2.2.5'
0292 
0293 Add a probe in a source file using special characters by backslash escape
0294 
0295  ./perf probe -x /opt/test/a.out 'foo\+bar.c:4'
0296 
0297 
0298 PERMISSIONS AND SYSCTL
0299 ----------------------
0300 Since perf probe depends on ftrace (tracefs) and kallsyms (/proc/kallsyms), you have to care about the permission and some sysctl knobs.
0301 
0302  - Since tracefs and kallsyms requires root or privileged user to access it, the following perf probe commands also require it; --add, --del, --list (except for --cache option)
0303 
0304  - The system admin can remount the tracefs with 755 (`sudo mount -o remount,mode=755 /sys/kernel/tracing/`) to allow unprivileged user to run the perf probe --list command.
0305 
0306  - /proc/sys/kernel/kptr_restrict = 2 (restrict all users) also prevents perf probe to retrieve the important information from kallsyms. You also need to set to 1 (restrict non CAP_SYSLOG users) for the above commands. Since the user-space probe doesn't need to access kallsyms, this is only for probing the kernel function (kprobes).
0307 
0308  - Since the perf probe commands read the vmlinux (for kernel) and/or the debuginfo file (including user-space application), you need to ensure that you can read those files.
0309 
0310 
0311 SEE ALSO
0312 --------
0313 linkperf:perf-trace[1], linkperf:perf-record[1], linkperf:perf-buildid-cache[1]