Mauro Carvalho Chehab | d2ba7ca | 2020-03-02 09:15:45 +0100 | [diff] [blame] | 1 | .. SPDX-License-Identifier: GPL-2.0 |
| 2 | |
| 3 | ================================= |
Karen Xie | c367346 | 2008-12-09 14:15:32 -0800 | [diff] [blame] | 4 | Chelsio S3 iSCSI Driver for Linux |
Mauro Carvalho Chehab | d2ba7ca | 2020-03-02 09:15:45 +0100 | [diff] [blame] | 5 | ================================= |
Karen Xie | c367346 | 2008-12-09 14:15:32 -0800 | [diff] [blame] | 6 | |
| 7 | Introduction |
| 8 | ============ |
| 9 | |
| 10 | The Chelsio T3 ASIC based Adapters (S310, S320, S302, S304, Mezz cards, etc. |
Karen Xie | 802d527 | 2009-02-13 21:39:04 -0800 | [diff] [blame] | 11 | series of products) support iSCSI acceleration and iSCSI Direct Data Placement |
Karen Xie | c367346 | 2008-12-09 14:15:32 -0800 | [diff] [blame] | 12 | (DDP) where the hardware handles the expensive byte touching operations, such |
| 13 | as CRC computation and verification, and direct DMA to the final host memory |
| 14 | destination: |
| 15 | |
| 16 | - iSCSI PDU digest generation and verification |
| 17 | |
| 18 | On transmitting, Chelsio S3 h/w computes and inserts the Header and |
| 19 | Data digest into the PDUs. |
| 20 | On receiving, Chelsio S3 h/w computes and verifies the Header and |
| 21 | Data digest of the PDUs. |
| 22 | |
| 23 | - Direct Data Placement (DDP) |
| 24 | |
| 25 | S3 h/w can directly place the iSCSI Data-In or Data-Out PDU's |
| 26 | payload into pre-posted final destination host-memory buffers based |
| 27 | on the Initiator Task Tag (ITT) in Data-In or Target Task Tag (TTT) |
| 28 | in Data-Out PDUs. |
| 29 | |
| 30 | - PDU Transmit and Recovery |
| 31 | |
| 32 | On transmitting, S3 h/w accepts the complete PDU (header + data) |
| 33 | from the host driver, computes and inserts the digests, decomposes |
| 34 | the PDU into multiple TCP segments if necessary, and transmit all |
| 35 | the TCP segments onto the wire. It handles TCP retransmission if |
| 36 | needed. |
| 37 | |
Karen Xie | 802d527 | 2009-02-13 21:39:04 -0800 | [diff] [blame] | 38 | On receiving, S3 h/w recovers the iSCSI PDU by reassembling TCP |
Karen Xie | c367346 | 2008-12-09 14:15:32 -0800 | [diff] [blame] | 39 | segments, separating the header and data, calculating and verifying |
Karen Xie | 802d527 | 2009-02-13 21:39:04 -0800 | [diff] [blame] | 40 | the digests, then forwarding the header to the host. The payload data, |
Karen Xie | c367346 | 2008-12-09 14:15:32 -0800 | [diff] [blame] | 41 | if possible, will be directly placed into the pre-posted host DDP |
| 42 | buffer. Otherwise, the payload data will be sent to the host too. |
| 43 | |
| 44 | The cxgb3i driver interfaces with open-iscsi initiator and provides the iSCSI |
| 45 | acceleration through Chelsio hardware wherever applicable. |
| 46 | |
| 47 | Using the cxgb3i Driver |
| 48 | ======================= |
| 49 | |
| 50 | The following steps need to be taken to accelerates the open-iscsi initiator: |
| 51 | |
| 52 | 1. Load the cxgb3i driver: "modprobe cxgb3i" |
| 53 | |
| 54 | The cxgb3i module registers a new transport class "cxgb3i" with open-iscsi. |
| 55 | |
Mauro Carvalho Chehab | d2ba7ca | 2020-03-02 09:15:45 +0100 | [diff] [blame] | 56 | * in the case of recompiling the kernel, the cxgb3i selection is located at:: |
| 57 | |
Karen Xie | c367346 | 2008-12-09 14:15:32 -0800 | [diff] [blame] | 58 | Device Drivers |
| 59 | SCSI device support ---> |
| 60 | [*] SCSI low-level drivers ---> |
| 61 | <M> Chelsio S3xx iSCSI support |
| 62 | |
| 63 | 2. Create an interface file located under /etc/iscsi/ifaces/ for the new |
| 64 | transport class "cxgb3i". |
| 65 | |
Mauro Carvalho Chehab | d2ba7ca | 2020-03-02 09:15:45 +0100 | [diff] [blame] | 66 | The content of the file should be in the following format:: |
| 67 | |
Karen Xie | c367346 | 2008-12-09 14:15:32 -0800 | [diff] [blame] | 68 | iface.transport_name = cxgb3i |
| 69 | iface.net_ifacename = <ethX> |
| 70 | iface.ipaddress = <iscsi ip address> |
| 71 | |
| 72 | * if iface.ipaddress is specified, <iscsi ip address> needs to be either the |
Mauro Carvalho Chehab | d2ba7ca | 2020-03-02 09:15:45 +0100 | [diff] [blame] | 73 | same as the ethX's ip address or an address on the same subnet. Make |
| 74 | sure the ip address is unique in the network. |
Karen Xie | c367346 | 2008-12-09 14:15:32 -0800 | [diff] [blame] | 75 | |
| 76 | 3. edit /etc/iscsi/iscsid.conf |
Karen Xie | 802d527 | 2009-02-13 21:39:04 -0800 | [diff] [blame] | 77 | The default setting for MaxRecvDataSegmentLength (131072) is too big; |
Mauro Carvalho Chehab | d2ba7ca | 2020-03-02 09:15:45 +0100 | [diff] [blame] | 78 | replace with a value no bigger than 15360 (for example 8192):: |
Karen Xie | c367346 | 2008-12-09 14:15:32 -0800 | [diff] [blame] | 79 | |
| 80 | node.conn[0].iscsi.MaxRecvDataSegmentLength = 8192 |
| 81 | |
| 82 | * The login would fail for a normal session if MaxRecvDataSegmentLength is |
Mauro Carvalho Chehab | d2ba7ca | 2020-03-02 09:15:45 +0100 | [diff] [blame] | 83 | too big. A error message in the format of |
| 84 | "cxgb3i: ERR! MaxRecvSegmentLength <X> too big. Need to be <= <Y>." |
| 85 | would be logged to dmesg. |
Karen Xie | c367346 | 2008-12-09 14:15:32 -0800 | [diff] [blame] | 86 | |
| 87 | 4. To direct open-iscsi traffic to go through cxgb3i's accelerated path, |
| 88 | "-I <iface file name>" option needs to be specified with most of the |
| 89 | iscsiadm command. <iface file name> is the transport interface file created |
| 90 | in step 2. |