Parallel Sysplex

Overview

The IBM Integrated Coupling Adapter (ICA SR) is the latest short reach coupling link. It was announced on January 14, 2015 in US Hardware Announcement 115-001, “The IBM z13”.

Parallel Sysplex Infiniband (PSIFB) coupling links continue to be supported. They were announced in February 26, 2008 in US Hardware Announcement 108-154, “IBM System z10 Enterprise Class – The forward-thinking mainframe for the twenty-first century” as well as in the May 6, 2008 US Hardware Announcement 108-269, “IBM System z10 Enterprise Class: Helping to meet global 24x7 demands for information services with improvements for Internet access and coupling,” IFB3 protocol is described in the July 12, 2011 hardware announcement 111-121, “zEnterprise 196 enhancements deliver faster access to data.

There are several ways of classifying IFB links. It can be Double Data Rate (DDR) or Single Data Rate (SDR). The IFB links can also be characterized by the amount of lanes of communication within each physical link. For example, 12x links provide 12 lanes of communication within each physical link. 1x links provide a single lane of communication within each physical link.


Value

Reasons for migrating to IBM Integrated Coupling Adapter (ICA SR) links

Other z13 Coupling Connectivity Enhancements

PSIFB Links

External ICA SR and PSIFB coupling links support time synchronization signals for Server Time Protocol (STP).


Performance

Specifically, a comparison of the effect of the links relative performance can be seen in the table below.

This table assumes:

  1. The CF activity rate to the coupling facility is 9 operations per million instructions, or 9 operations/second per MIPS of the server. This is typical for a high end data sharing environment. If the activity rate is less, then the values shown in the table should be proportionally less.
  2. Heuristic Synchronous to Asynchronous conversion is not taken into account. This would limit the coupling overhead to about 18%.
  3. Shown is data for a non-System-Managed duplexing environment.
Host z114 z196 zBC12 zEC12 z13
CF        
z114 ISC-3 17% 21% 19% 24% N/A
z114 1x IFB 14% 17% 17% 21% 22%
z114 12x IFB 12% 15% 15% 17% 19%
z114 12x IFB3 10% 12% 12% 13% 14%
z196 ISC-3 17% 21% 19% 24% N/A
z196 1x IFB 13% 16% 16% 18% 21%
z196 12x IFB 11% 14% 14% 15% 17%
z196 12x IFB3 9% 11% 10% 12% 13%
zBC12 ISC-3 17% 21% 19% 24% N/A
zBC12 1x IFB 14% 18% 17% 20% 22%
zBC12 12x IFB 12% 15% 14% 17% 18%
zBC12 12x IFB3 10% 11% 11% 12% 14%
zEC12 ISC-3 17% 21% 19% 24% N/A
zEC12 1x IFB 13% 16% 16% 18% 20%
zEC12 12x IFB 11% 13% 13% 15% 17%
zEC12 12x IFB3 9% 10% 10% 11% 12%
z13 1x IFB 14% 17% 16% 19% 20%
z13 12x IFB 12% 14% 14% 16% 17%
z13 12x IFB3 9% 11% 10% 12% 12%
z13 ICA SR N/A N/A N/A N/A 11%

Configurations

Coupling Link types

Coupling links provide a path to transmit/receive Coupling Facility (CF) data as well as Server Time Protocol (STP) timekeeping messages. The CF data may be exchanged between z/OS and the CF or between CFs. For an IBM z13, the coupling link options are:

Integrated Coupling Adapter (ICA SR)

12x InfiniBand

1x InfiniBand

Internal coupling links (ICs) can be used for internal communication between Coupling Facilities (CFs) defined in LPARs and z/OS images on the same server.

ISC-3 links are not supported on IBM z13. The zEC12 and zBC12 are the last servers to support ISC-3.

Note 1: The ICA SR and InfiniBand (PSIFB) link data rates do not represent the performance of the link. The actual performance is dependent upon many factors including latency through the adapters, cable lengths, and the type of workload. Systems Lab Services can assist your migration to ICA SR and PSIFB coupling links by providing services to assess the impact of the migration or to assist with the implementation of the migration.

Note 2: The ICA SR fanout requires new cabling. For distances up to 100 m, clients can choose the OM3 fiber type. For distances up to 150 m, clients must choose the OM4 fiber type. Refer to IBM z Systems Planning for Fiber Optic Links (FICON/FCP, Coupling Links, and Open System Adapters), GA23-1407, and to IBM z Systems Maintenance for Fiber Optic Links (FICON/FCP, Coupling Links, and Open System Adapters), SY27-7694, which can be found in the Library section of Resource Link® at http://www.ibm.com/servers/resourcelink/svc03100.nsf?OpenDatabase

Note 3: The list of qualified DWDM vendors is available on Resource Link, located under the “Hardware products for server” on the Library page.)

Note 4: RPQs for fiber distances > 100 km:

  ICA SR 1x IFB 12x IFB
z13 Y Y Y
zEnterprise N Y Y
System z10 N Y Y


 

Number of Links and Fanouts

  Model Max # of ICA SR Fanouts Max # of IFB (HCA-O) Fanouts ICA SR Links
(2 Ports / Fanouts)
1x IFB Links (HCA3-O LR)
(4 Ports / Fanout)
12x IFB Links (HCA3-O)
(2 Ports / Fanout)
ISC-3 Links IC Links
z13 N30
(1 drawer)
10
 
4
 
20
 
16
 
8 n/a
 
32
 
N63
(2 drawer)
16 8
 
32 32
 
16
 
n/a
 
32
 
N96
(3 drawer)
16 12 32 48
 
24
 
n/a
 
32
 
NC9
(4 drawer)
16 16 32 64
 
32
 
n/a
 
32
 
NE1
(4 drawer)
16 16 32 64
 
32
 
n/a 32
 
zEC12 H20
(1 book)
n/a 8 n/a 32 16 48 32
H43
(2 book)
n/a 16 n/a 64 32 48 32
H66
(3 book)
n/a 16 n/a 64 32 48 32
H89
(4 book)
n/a 16 n/a 64 32 48 32
HA1
(4 book)
n/a 16 n/a 64 32 48 32
zBC12 H06
(1 drawer)
n/a 4 n/a 16 8 32 32
H13
(2 drawer)
n/a 8 n/a 32 16 32 32
z196 M15 n/a 8 n/a 32 16 48 32
M32 n/a 16 n/a 64 32 48 32
M49 n/a 16 n/a 64 32 48 32
M66 n/a 16 n/a 64 32 48 32
M80 n/a 16 n/a 64 32 48 32
z114 M05 n/a 4 n/a 16 8 48 32
M10 n/a 8 n/a 32 16 48 32

Maximum 16 ICA SR fanouts per server. Each fanout supports 2 ICA SR ports.

Maximum 16 HCA-O fanouts per server.
Each HCA-O fanout can support either:

ISC-3 Links are not supported on IBM z13.


More information

More information regarding Parallel Sysplex coupling links can be obtained from:

Redbooks

White Paper

Contact IBM

Browse z Systems


Hardware

Infrastructure matters. Businesses turn to the IBM mainframe for unmatched security, operational efficiency, speed, seamless scale, and lower cost per transaction. The IBM z System, the world’s premier data and transaction engine, is enabled for mobile, integrates transactions and analytics, and delivers efficient and trusted clouds.

Operating Systems

IBM z Systems supports multiple operation systems:

Software

IBM z Systems combine leading-edge innovation and unparalleled capabilities to power cloud, business analytics, mobile applications, and the most important workloads.

Server Time Protocol

Time Synchronization for the Next Generation