0001 ============================
0002 Kernel NFS Server Statistics
0003 ============================
0004
0005 :Authors: Greg Banks <gnb@sgi.com> - 26 Mar 2009
0006
0007 This document describes the format and semantics of the statistics
0008 which the kernel NFS server makes available to userspace. These
0009 statistics are available in several text form pseudo files, each of
0010 which is described separately below.
0011
0012 In most cases you don't need to know these formats, as the nfsstat(8)
0013 program from the nfs-utils distribution provides a helpful command-line
0014 interface for extracting and printing them.
0015
0016 All the files described here are formatted as a sequence of text lines,
0017 separated by newline '\n' characters. Lines beginning with a hash
0018 '#' character are comments intended for humans and should be ignored
0019 by parsing routines. All other lines contain a sequence of fields
0020 separated by whitespace.
0021
0022 /proc/fs/nfsd/pool_stats
0023 ========================
0024
0025 This file is available in kernels from 2.6.30 onwards, if the
0026 /proc/fs/nfsd filesystem is mounted (it almost always should be).
0027
0028 The first line is a comment which describes the fields present in
0029 all the other lines. The other lines present the following data as
0030 a sequence of unsigned decimal numeric fields. One line is shown
0031 for each NFS thread pool.
0032
0033 All counters are 64 bits wide and wrap naturally. There is no way
0034 to zero these counters, instead applications should do their own
0035 rate conversion.
0036
0037 pool
0038 The id number of the NFS thread pool to which this line applies.
0039 This number does not change.
0040
0041 Thread pool ids are a contiguous set of small integers starting
0042 at zero. The maximum value depends on the thread pool mode, but
0043 currently cannot be larger than the number of CPUs in the system.
0044 Note that in the default case there will be a single thread pool
0045 which contains all the nfsd threads and all the CPUs in the system,
0046 and thus this file will have a single line with a pool id of "0".
0047
0048 packets-arrived
0049 Counts how many NFS packets have arrived. More precisely, this
0050 is the number of times that the network stack has notified the
0051 sunrpc server layer that new data may be available on a transport
0052 (e.g. an NFS or UDP socket or an NFS/RDMA endpoint).
0053
0054 Depending on the NFS workload patterns and various network stack
0055 effects (such as Large Receive Offload) which can combine packets
0056 on the wire, this may be either more or less than the number
0057 of NFS calls received (which statistic is available elsewhere).
0058 However this is a more accurate and less workload-dependent measure
0059 of how much CPU load is being placed on the sunrpc server layer
0060 due to NFS network traffic.
0061
0062 sockets-enqueued
0063 Counts how many times an NFS transport is enqueued to wait for
0064 an nfsd thread to service it, i.e. no nfsd thread was considered
0065 available.
0066
0067 The circumstance this statistic tracks indicates that there was NFS
0068 network-facing work to be done but it couldn't be done immediately,
0069 thus introducing a small delay in servicing NFS calls. The ideal
0070 rate of change for this counter is zero; significantly non-zero
0071 values may indicate a performance limitation.
0072
0073 This can happen because there are too few nfsd threads in the thread
0074 pool for the NFS workload (the workload is thread-limited), in which
0075 case configuring more nfsd threads will probably improve the
0076 performance of the NFS workload.
0077
0078 threads-woken
0079 Counts how many times an idle nfsd thread is woken to try to
0080 receive some data from an NFS transport.
0081
0082 This statistic tracks the circumstance where incoming
0083 network-facing NFS work is being handled quickly, which is a good
0084 thing. The ideal rate of change for this counter will be close
0085 to but less than the rate of change of the packets-arrived counter.
0086
0087 threads-timedout
0088 Counts how many times an nfsd thread triggered an idle timeout,
0089 i.e. was not woken to handle any incoming network packets for
0090 some time.
0091
0092 This statistic counts a circumstance where there are more nfsd
0093 threads configured than can be used by the NFS workload. This is
0094 a clue that the number of nfsd threads can be reduced without
0095 affecting performance. Unfortunately, it's only a clue and not
0096 a strong indication, for a couple of reasons:
0097
0098 - Currently the rate at which the counter is incremented is quite
0099 slow; the idle timeout is 60 minutes. Unless the NFS workload
0100 remains constant for hours at a time, this counter is unlikely
0101 to be providing information that is still useful.
0102
0103 - It is usually a wise policy to provide some slack,
0104 i.e. configure a few more nfsds than are currently needed,
0105 to allow for future spikes in load.
0106
0107
0108 Note that incoming packets on NFS transports will be dealt with in
0109 one of three ways. An nfsd thread can be woken (threads-woken counts
0110 this case), or the transport can be enqueued for later attention
0111 (sockets-enqueued counts this case), or the packet can be temporarily
0112 deferred because the transport is currently being used by an nfsd
0113 thread. This last case is not very interesting and is not explicitly
0114 counted, but can be inferred from the other counters thus::
0115
0116 packets-deferred = packets-arrived - ( sockets-enqueued + threads-woken )
0117
0118
0119 More
0120 ====
0121
0122 Descriptions of the other statistics file should go here.
