Logan Gunthorpe | 44fc691 | 2017-03-02 16:24:32 -0700 | [diff] [blame] | 1 | ======================== |
| 2 | Linux Switchtec Support |
| 3 | ======================== |
| 4 | |
| 5 | Microsemi's "Switchtec" line of PCI switch devices is already |
| 6 | supported by the kernel with standard PCI switch drivers. However, the |
| 7 | Switchtec device advertises a special management endpoint which |
| 8 | enables some additional functionality. This includes: |
| 9 | |
| 10 | * Packet and Byte Counters |
| 11 | * Firmware Upgrades |
| 12 | * Event and Error logs |
| 13 | * Querying port link status |
| 14 | * Custom user firmware commands |
| 15 | |
| 16 | The switchtec kernel module implements this functionality. |
| 17 | |
| 18 | |
| 19 | Interface |
| 20 | ========= |
| 21 | |
| 22 | The primary means of communicating with the Switchtec management firmware is |
| 23 | through the Memory-mapped Remote Procedure Call (MRPC) interface. |
| 24 | Commands are submitted to the interface with a 4-byte command |
| 25 | identifier and up to 1KB of command specific data. The firmware will |
Wesley Yung | fb513f6 | 2018-10-10 22:42:55 +0800 | [diff] [blame] | 26 | respond with a 4-byte return code and up to 1KB of command-specific |
Logan Gunthorpe | 44fc691 | 2017-03-02 16:24:32 -0700 | [diff] [blame] | 27 | data. The interface only processes a single command at a time. |
| 28 | |
| 29 | |
| 30 | Userspace Interface |
| 31 | =================== |
| 32 | |
| 33 | The MRPC interface will be exposed to userspace through a simple char |
| 34 | device: /dev/switchtec#, one for each management endpoint in the system. |
| 35 | |
| 36 | The char device has the following semantics: |
| 37 | |
| 38 | * A write must consist of at least 4 bytes and no more than 1028 bytes. |
Wesley Yung | fb513f6 | 2018-10-10 22:42:55 +0800 | [diff] [blame] | 39 | The first 4 bytes will be interpreted as the Command ID and the |
| 40 | remainder will be used as the input data. A write will send the |
Logan Gunthorpe | 44fc691 | 2017-03-02 16:24:32 -0700 | [diff] [blame] | 41 | command to the firmware to begin processing. |
| 42 | |
| 43 | * Each write must be followed by exactly one read. Any double write will |
| 44 | produce an error and any read that doesn't follow a write will |
| 45 | produce an error. |
| 46 | |
| 47 | * A read will block until the firmware completes the command and return |
Wesley Yung | fb513f6 | 2018-10-10 22:42:55 +0800 | [diff] [blame] | 48 | the 4-byte Command Return Value plus up to 1024 bytes of output |
| 49 | data. (The length will be specified by the size parameter of the read |
| 50 | call -- reading less than 4 bytes will produce an error.) |
Logan Gunthorpe | 44fc691 | 2017-03-02 16:24:32 -0700 | [diff] [blame] | 51 | |
| 52 | * The poll call will also be supported for userspace applications that |
| 53 | need to do other things while waiting for the command to complete. |
Logan Gunthorpe | 52eabba | 2017-03-02 16:24:34 -0700 | [diff] [blame] | 54 | |
| 55 | The following IOCTLs are also supported by the device: |
| 56 | |
| 57 | * SWITCHTEC_IOCTL_FLASH_INFO - Retrieve firmware length and number |
| 58 | of partitions in the device. |
| 59 | |
| 60 | * SWITCHTEC_IOCTL_FLASH_PART_INFO - Retrieve address and lengeth for |
| 61 | any specified partition in flash. |
| 62 | |
| 63 | * SWITCHTEC_IOCTL_EVENT_SUMMARY - Read a structure of bitmaps |
| 64 | indicating all uncleared events. |
| 65 | |
| 66 | * SWITCHTEC_IOCTL_EVENT_CTL - Get the current count, clear and set flags |
| 67 | for any event. This ioctl takes in a switchtec_ioctl_event_ctl struct |
| 68 | with the event_id, index and flags set (index being the partition or PFF |
| 69 | number for non-global events). It returns whether the event has |
| 70 | occurred, the number of times and any event specific data. The flags |
| 71 | can be used to clear the count or enable and disable actions to |
| 72 | happen when the event occurs. |
| 73 | By using the SWITCHTEC_IOCTL_EVENT_FLAG_EN_POLL flag, |
| 74 | you can set an event to trigger a poll command to return with |
| 75 | POLLPRI. In this way, userspace can wait for events to occur. |
| 76 | |
| 77 | * SWITCHTEC_IOCTL_PFF_TO_PORT and SWITCHTEC_IOCTL_PORT_TO_PFF convert |
| 78 | between PCI Function Framework number (used by the event system) |
| 79 | and Switchtec Logic Port ID and Partition number (which is more |
| 80 | user friendly). |
Logan Gunthorpe | d045024 | 2017-08-03 12:19:54 -0600 | [diff] [blame] | 81 | |
| 82 | |
| 83 | Non-Transparent Bridge (NTB) Driver |
| 84 | =================================== |
| 85 | |
Wesley Yung | fb513f6 | 2018-10-10 22:42:55 +0800 | [diff] [blame] | 86 | An NTB hardware driver is provided for the Switchtec hardware in |
| 87 | ntb_hw_switchtec. Currently, it only supports switches configured with |
| 88 | exactly 2 NT partitions and zero or more non-NT partitions. It also requires |
| 89 | the following configuration settings: |
Logan Gunthorpe | d045024 | 2017-08-03 12:19:54 -0600 | [diff] [blame] | 90 | |
Wesley Yung | fb513f6 | 2018-10-10 22:42:55 +0800 | [diff] [blame] | 91 | * Both NT partitions must be able to access each other's GAS spaces. |
Logan Gunthorpe | d045024 | 2017-08-03 12:19:54 -0600 | [diff] [blame] | 92 | Thus, the bits in the GAS Access Vector under Management Settings |
| 93 | must be set to support this. |
Wesley Yung | fb513f6 | 2018-10-10 22:42:55 +0800 | [diff] [blame] | 94 | * Kernel configuration MUST include support for NTB (CONFIG_NTB needs |
| 95 | to be set) |
| 96 | |
| 97 | NT EP BAR 2 will be dynamically configured as a Direct Window, and |
| 98 | the configuration file does not need to configure it explicitly. |
| 99 | |
Mauro Carvalho Chehab | baa293e | 2019-06-27 15:39:22 -0300 | [diff] [blame] | 100 | Please refer to Documentation/driver-api/ntb.rst in Linux source tree for an overall |
Wesley Yung | fb513f6 | 2018-10-10 22:42:55 +0800 | [diff] [blame] | 101 | understanding of the Linux NTB stack. ntb_hw_switchtec works as an NTB |
| 102 | Hardware Driver in this stack. |