0001 What: /sys/class/rtrs-client
0002 Date: Feb 2020
0003 KernelVersion: 5.7
0004 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
0005 Description: When a user of RTRS API creates a new session, a directory entry with
0006 the name of that session is created under /sys/class/rtrs-client/<session-name>/
0007
0008 What: /sys/class/rtrs-client/<session-name>/add_path
0009 Date: Feb 2020
0010 KernelVersion: 5.7
0011 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
0012 Description: RW, adds a new path (connection) to an existing session. Expected format is the
0013 following::
0014
0015 <[source addr,]destination addr>
0016 *addr ::= [ ip:<ipv4|ipv6> | gid:<gid> ]
0017
0018 What: /sys/class/rtrs-client/<session-name>/max_reconnect_attempts
0019 Date: Feb 2020
0020 KernelVersion: 5.7
0021 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
0022 Description: Maximum number reconnect attempts the client should make before giving up
0023 after connection breaks unexpectedly.
0024
0025 What: /sys/class/rtrs-client/<session-name>/mp_policy
0026 Date: Feb 2020
0027 KernelVersion: 5.7
0028 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
0029 Description: Multipath policy specifies which path should be selected on each IO:
0030
0031 round-robin (0):
0032 select path in per CPU round-robin manner.
0033
0034 min-inflight (1):
0035 select path with minimum inflights.
0036
0037 min-latency (2):
0038 select path with minimum latency.
0039
0040 What: /sys/class/rtrs-client/<session-name>/paths/
0041 Date: Feb 2020
0042 KernelVersion: 5.7
0043 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
0044 Description: Each path belonging to a given session is listed here by its source and
0045 destination address. When a new path is added to a session by writing to
0046 the "add_path" entry, a directory <src@dst> is created.
0047
0048 What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/state
0049 Date: Feb 2020
0050 KernelVersion: 5.7
0051 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
0052 Description: RO, Contains "connected" if the session is connected to the peer and fully
0053 functional. Otherwise the file contains "disconnected"
0054
0055 What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/reconnect
0056 Date: Feb 2020
0057 KernelVersion: 5.7
0058 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
0059 Description: Write "1" to the file in order to reconnect the path.
0060 Operation is blocking and returns 0 if reconnect was successful.
0061
0062 What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/disconnect
0063 Date: Feb 2020
0064 KernelVersion: 5.7
0065 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
0066 Description: Write "1" to the file in order to disconnect the path.
0067 Operation blocks until RTRS path is disconnected.
0068
0069 What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/remove_path
0070 Date: Feb 2020
0071 KernelVersion: 5.7
0072 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
0073 Description: Write "1" to the file in order to disconnected and remove the path
0074 from the session. Operation blocks until the path is disconnected
0075 and removed from the session.
0076
0077 What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/hca_name
0078 Date: Feb 2020
0079 KernelVersion: 5.7
0080 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
0081 Description: RO, Contains the name of HCA the connection established on.
0082
0083 What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/hca_port
0084 Date: Feb 2020
0085 KernelVersion: 5.7
0086 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
0087 Description: RO, Contains the port number of active port traffic is going through.
0088
0089 What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/src_addr
0090 Date: Feb 2020
0091 KernelVersion: 5.7
0092 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
0093 Description: RO, Contains the source address of the path
0094
0095 What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/dst_addr
0096 Date: Feb 2020
0097 KernelVersion: 5.7
0098 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
0099 Description: RO, Contains the destination address of the path
0100
0101 What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/cur_latency
0102 Date: Feb 2020
0103 KernelVersion: 5.7
0104 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
0105 Description: RO, Contains the latency time calculated by the heart-beat messages.
0106 Whenever the client sends heart-beat message, it checks the time gap
0107 between sending the heart-beat message and receiving the ACK.
0108 This value can be changed regularly.
0109
0110 What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/reset_all
0111 Date: Feb 2020
0112 KernelVersion: 5.7
0113 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
0114 Description: RW, Read will return usage help, write 0 will clear all the statistics.
0115
0116 What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/cpu_migration
0117 Date: Feb 2020
0118 KernelVersion: 5.7
0119 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
0120 Description: RTRS expects that each HCA IRQ is pinned to a separate CPU. If it's
0121 not the case, the processing of an I/O response could be processed on a
0122 different CPU than where it was originally submitted. This file shows
0123 how many interrupts where generated on a non expected CPU.
0124
0125 "from:"
0126 is the CPU on which the IRQ was expected, but not generated.
0127 "to:"
0128 is the CPU on which the IRQ was generated, but not expected.
0129
0130 What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/reconnects
0131 Date: Feb 2020
0132 KernelVersion: 5.7
0133 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
0134 Description: Contains 2 unsigned int values, the first one records number of successful
0135 reconnects in the path lifetime, the second one records number of failed
0136 reconnects in the path lifetime.
0137
0138 What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/rdma
0139 Date: Feb 2020
0140 KernelVersion: 5.7
0141 Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com>
0142 Description: Contains statistics regarding rdma operations and inflight operations.
0143 The output consists of 6 values::
0144
0145 <read-count> <read-total-size> <write-count> \
0146 <write-total-size> <inflights> <failovered>
