Mauro Carvalho Chehab | e4a5c33 | 2017-05-16 10:04:34 -0300 | [diff] [blame] | 1 | =============================== |
Linus Torvalds | 1da177e | 2005-04-16 15:20:36 -0700 | [diff] [blame] | 2 | Numa policy hit/miss statistics |
Mauro Carvalho Chehab | e4a5c33 | 2017-05-16 10:04:34 -0300 | [diff] [blame] | 3 | =============================== |
Linus Torvalds | 1da177e | 2005-04-16 15:20:36 -0700 | [diff] [blame] | 4 | |
| 5 | /sys/devices/system/node/node*/numastat |
| 6 | |
| 7 | All units are pages. Hugepages have separate counters. |
| 8 | |
Vlastimil Babka | 77691ee | 2020-05-07 14:02:17 +0200 | [diff] [blame] | 9 | The numa_hit, numa_miss and numa_foreign counters reflect how well processes |
| 10 | are able to allocate memory from nodes they prefer. If they succeed, numa_hit |
| 11 | is incremented on the preferred node, otherwise numa_foreign is incremented on |
| 12 | the preferred node and numa_miss on the node where allocation succeeded. |
| 13 | |
| 14 | Usually preferred node is the one local to the CPU where the process executes, |
| 15 | but restrictions such as mempolicies can change that, so there are also two |
| 16 | counters based on CPU local node. local_node is similar to numa_hit and is |
| 17 | incremented on allocation from a node by CPU on the same node. other_node is |
| 18 | similar to numa_miss and is incremented on the node where allocation succeeds |
| 19 | from a CPU from a different node. Note there is no counter analogical to |
| 20 | numa_foreign. |
| 21 | |
| 22 | In more detail: |
| 23 | |
Mauro Carvalho Chehab | e4a5c33 | 2017-05-16 10:04:34 -0300 | [diff] [blame] | 24 | =============== ============================================================ |
Geunsik Lim | 36519c9 | 2012-02-25 11:12:23 +0900 | [diff] [blame] | 25 | numa_hit A process wanted to allocate memory from this node, |
| 26 | and succeeded. |
| 27 | |
| 28 | numa_miss A process wanted to allocate memory from another node, |
| 29 | but ended up with memory from this node. |
| 30 | |
| 31 | numa_foreign A process wanted to allocate on this node, |
Vlastimil Babka | 77691ee | 2020-05-07 14:02:17 +0200 | [diff] [blame] | 32 | but ended up with memory from another node. |
Geunsik Lim | 36519c9 | 2012-02-25 11:12:23 +0900 | [diff] [blame] | 33 | |
Vlastimil Babka | 77691ee | 2020-05-07 14:02:17 +0200 | [diff] [blame] | 34 | local_node A process ran on this node's CPU, |
| 35 | and got memory from this node. |
Geunsik Lim | 36519c9 | 2012-02-25 11:12:23 +0900 | [diff] [blame] | 36 | |
Vlastimil Babka | 77691ee | 2020-05-07 14:02:17 +0200 | [diff] [blame] | 37 | other_node A process ran on a different node's CPU |
| 38 | and got memory from this node. |
Geunsik Lim | 36519c9 | 2012-02-25 11:12:23 +0900 | [diff] [blame] | 39 | |
| 40 | interleave_hit Interleaving wanted to allocate from this node |
| 41 | and succeeded. |
Mauro Carvalho Chehab | e4a5c33 | 2017-05-16 10:04:34 -0300 | [diff] [blame] | 42 | =============== ============================================================ |
Linus Torvalds | 1da177e | 2005-04-16 15:20:36 -0700 | [diff] [blame] | 43 | |
| 44 | For easier reading you can use the numastat utility from the numactl package |
Geunsik Lim | 36519c9 | 2012-02-25 11:12:23 +0900 | [diff] [blame] | 45 | (http://oss.sgi.com/projects/libnuma/). Note that it only works |
Linus Torvalds | 1da177e | 2005-04-16 15:20:36 -0700 | [diff] [blame] | 46 | well right now on machines with a small number of CPUs. |
| 47 | |
Vlastimil Babka | 77691ee | 2020-05-07 14:02:17 +0200 | [diff] [blame] | 48 | Note that on systems with memoryless nodes (where a node has CPUs but no |
| 49 | memory) the numa_hit, numa_miss and numa_foreign statistics can be skewed |
| 50 | heavily. In the current kernel implementation, if a process prefers a |
| 51 | memoryless node (i.e. because it is running on one of its local CPU), the |
| 52 | implementation actually treats one of the nearest nodes with memory as the |
| 53 | preferred node. As a result, such allocation will not increase the numa_foreign |
| 54 | counter on the memoryless node, and will skew the numa_hit, numa_miss and |
| 55 | numa_foreign statistics of the nearest node. |