– 25GbE and 10GbE Remote Direct Memory Access (RDMA) over Converged Ethernet (RoCE) Express2.1 

– 25GbE and 10GbE RoCE Express2 

– Coupling Express Long Reach (CE LR) (available also on z14 M0x, z14 ZR1, z13, and z13s) 

## 4.1.2 PCIe Generation 3 

– 10 GbE RoCE Express 

– Crypto Express7S (single/dual ports) 

– Crypto Express6s 

– Crypto Express5s 

The PCIe Generation 3 uses 128b/130b encoding for data transmission. This configuration reduces the encoding overhead to about 1.54% versus the PCIe Generation 2 overhead of 20% that uses 8b/10b encoding. 

The PCIe standard uses a low-voltage differential serial bus. Two wires are used for signal transmission, and a total of four wires (two for transmit and two for receive) form a lane of a PCIe link, which is full-duplex. Multiple lanes can be aggregated into a larger link width. PCIe supports link widths of 1, 2, 4, 8, 12, 16, and 32 lanes (x1, x2, x4, x8, x12, x16, and x32). 

The data transmission rate of a PCIe link is determined by the link width (numbers of lanes), the signaling rate of each lane, and the signal encoding rule. The signaling rate of one PCIe Generation 3 lane is eight gigatransfers per second (GTps), which means that nearly 8 gigabits are transmitted per second (Gbps). 

Note:  I/O infrastructure for z15 is implemented as PCIe Generation 3. The PU chip PCIe interface is PCIe Generation 4 (x16 @32 GBps), but the CPC I/O Fanout infrastructure uses externally PCIe Generation 3 connectivity. 

A PCIe Gen3 x16 link features the following data transmission rates: 

GLYPH<SM590000> The maximum theoretical data transmission rate per lane: 

8 Gbps * 128/130 bit (encoding) = 7.87 Gbps=984.6 MBps 

GLYPH<SM590000> The maximum theoretical data transmission rate per link: 

984.6 MBps * 16 (lanes) = 15.75 GBps 

Considering that the PCIe link is full-duplex mode, the data throughput rate of a PCIe Gen3 x16 link is 31.5 GBps (15.75 GBps in both directions). 

Link performance:  The link speeds do not represent the performance of the link. The performance depends on many factors, including latency through the adapters, cable lengths, and the type of workload. 

PCIe Gen3 x16 links are used in z15 servers for driving the PCIe+ I/O drawers, and for coupling links for CPC to CPC communications. 

Note:  Unless specified otherwise, PCIe  refers to PCIe Generation 3 in remaining sections of this chapter. 

