Jack Wang | 745b6a3 | 2020-05-11 15:51:20 +0200 | [diff] [blame] | 1 | What: /sys/class/rtrs-client |
| 2 | Date: Feb 2020 |
| 3 | KernelVersion: 5.7 |
| 4 | Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> |
| 5 | Description: When a user of RTRS API creates a new session, a directory entry with |
| 6 | the name of that session is created under /sys/class/rtrs-client/<session-name>/ |
| 7 | |
| 8 | What: /sys/class/rtrs-client/<session-name>/add_path |
| 9 | Date: Feb 2020 |
| 10 | KernelVersion: 5.7 |
| 11 | Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> |
| 12 | Description: RW, adds a new path (connection) to an existing session. Expected format is the |
Mauro Carvalho Chehab | 002a9c2 | 2020-10-30 08:40:47 +0100 | [diff] [blame] | 13 | following:: |
Jack Wang | 745b6a3 | 2020-05-11 15:51:20 +0200 | [diff] [blame] | 14 | |
Mauro Carvalho Chehab | 002a9c2 | 2020-10-30 08:40:47 +0100 | [diff] [blame] | 15 | <[source addr,]destination addr> |
| 16 | *addr ::= [ ip:<ipv4|ipv6> | gid:<gid> ] |
Jack Wang | 745b6a3 | 2020-05-11 15:51:20 +0200 | [diff] [blame] | 17 | |
| 18 | What: /sys/class/rtrs-client/<session-name>/max_reconnect_attempts |
| 19 | Date: Feb 2020 |
| 20 | KernelVersion: 5.7 |
| 21 | Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> |
| 22 | Description: Maximum number reconnect attempts the client should make before giving up |
| 23 | after connection breaks unexpectedly. |
| 24 | |
| 25 | What: /sys/class/rtrs-client/<session-name>/mp_policy |
| 26 | Date: Feb 2020 |
| 27 | KernelVersion: 5.7 |
| 28 | Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> |
| 29 | Description: Multipath policy specifies which path should be selected on each IO: |
| 30 | |
| 31 | round-robin (0): |
Mauro Carvalho Chehab | 002a9c2 | 2020-10-30 08:40:47 +0100 | [diff] [blame] | 32 | select path in per CPU round-robin manner. |
Jack Wang | 745b6a3 | 2020-05-11 15:51:20 +0200 | [diff] [blame] | 33 | |
| 34 | min-inflight (1): |
Mauro Carvalho Chehab | 002a9c2 | 2020-10-30 08:40:47 +0100 | [diff] [blame] | 35 | select path with minimum inflights. |
Jack Wang | 745b6a3 | 2020-05-11 15:51:20 +0200 | [diff] [blame] | 36 | |
| 37 | What: /sys/class/rtrs-client/<session-name>/paths/ |
| 38 | Date: Feb 2020 |
| 39 | KernelVersion: 5.7 |
| 40 | Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> |
| 41 | Description: Each path belonging to a given session is listed here by its source and |
| 42 | destination address. When a new path is added to a session by writing to |
| 43 | the "add_path" entry, a directory <src@dst> is created. |
| 44 | |
| 45 | What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/state |
| 46 | Date: Feb 2020 |
| 47 | KernelVersion: 5.7 |
| 48 | Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> |
| 49 | Description: RO, Contains "connected" if the session is connected to the peer and fully |
| 50 | functional. Otherwise the file contains "disconnected" |
| 51 | |
| 52 | What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/reconnect |
| 53 | Date: Feb 2020 |
| 54 | KernelVersion: 5.7 |
| 55 | Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> |
| 56 | Description: Write "1" to the file in order to reconnect the path. |
| 57 | Operation is blocking and returns 0 if reconnect was successful. |
| 58 | |
| 59 | What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/disconnect |
| 60 | Date: Feb 2020 |
| 61 | KernelVersion: 5.7 |
| 62 | Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> |
| 63 | Description: Write "1" to the file in order to disconnect the path. |
| 64 | Operation blocks until RTRS path is disconnected. |
| 65 | |
| 66 | What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/remove_path |
| 67 | Date: Feb 2020 |
| 68 | KernelVersion: 5.7 |
| 69 | Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> |
| 70 | Description: Write "1" to the file in order to disconnected and remove the path |
| 71 | from the session. Operation blocks until the path is disconnected |
| 72 | and removed from the session. |
| 73 | |
| 74 | What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/hca_name |
| 75 | Date: Feb 2020 |
| 76 | KernelVersion: 5.7 |
| 77 | Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> |
| 78 | Description: RO, Contains the the name of HCA the connection established on. |
| 79 | |
| 80 | What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/hca_port |
| 81 | Date: Feb 2020 |
| 82 | KernelVersion: 5.7 |
| 83 | Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> |
| 84 | Description: RO, Contains the port number of active port traffic is going through. |
| 85 | |
| 86 | What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/src_addr |
| 87 | Date: Feb 2020 |
| 88 | KernelVersion: 5.7 |
| 89 | Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> |
| 90 | Description: RO, Contains the source address of the path |
| 91 | |
| 92 | What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/dst_addr |
| 93 | Date: Feb 2020 |
| 94 | KernelVersion: 5.7 |
| 95 | Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> |
| 96 | Description: RO, Contains the destination address of the path |
| 97 | |
| 98 | What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/reset_all |
| 99 | Date: Feb 2020 |
| 100 | KernelVersion: 5.7 |
| 101 | Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> |
| 102 | Description: RW, Read will return usage help, write 0 will clear all the statistics. |
| 103 | |
| 104 | What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/cpu_migration |
| 105 | Date: Feb 2020 |
| 106 | KernelVersion: 5.7 |
| 107 | Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> |
| 108 | Description: RTRS expects that each HCA IRQ is pinned to a separate CPU. If it's |
| 109 | not the case, the processing of an I/O response could be processed on a |
| 110 | different CPU than where it was originally submitted. This file shows |
| 111 | how many interrupts where generated on a non expected CPU. |
Mauro Carvalho Chehab | 002a9c2 | 2020-10-30 08:40:47 +0100 | [diff] [blame] | 112 | |
| 113 | "from:" |
| 114 | is the CPU on which the IRQ was expected, but not generated. |
| 115 | "to:" |
| 116 | is the CPU on which the IRQ was generated, but not expected. |
Jack Wang | 745b6a3 | 2020-05-11 15:51:20 +0200 | [diff] [blame] | 117 | |
| 118 | What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/reconnects |
| 119 | Date: Feb 2020 |
| 120 | KernelVersion: 5.7 |
| 121 | Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> |
| 122 | Description: Contains 2 unsigned int values, the first one records number of successful |
| 123 | reconnects in the path lifetime, the second one records number of failed |
| 124 | reconnects in the path lifetime. |
| 125 | |
| 126 | What: /sys/class/rtrs-client/<session-name>/paths/<src@dst>/stats/rdma |
| 127 | Date: Feb 2020 |
| 128 | KernelVersion: 5.7 |
| 129 | Contact: Jack Wang <jinpu.wang@cloud.ionos.com> Danil Kipnis <danil.kipnis@cloud.ionos.com> |
| 130 | Description: Contains statistics regarding rdma operations and inflight operations. |
Mauro Carvalho Chehab | 002a9c2 | 2020-10-30 08:40:47 +0100 | [diff] [blame] | 131 | The output consists of 6 values:: |
Jack Wang | 745b6a3 | 2020-05-11 15:51:20 +0200 | [diff] [blame] | 132 | |
Mauro Carvalho Chehab | 002a9c2 | 2020-10-30 08:40:47 +0100 | [diff] [blame] | 133 | <read-count> <read-total-size> <write-count> \ |
| 134 | <write-total-size> <inflights> <failovered> |