Parallel Sysplex InfiniBand (PSIFB) links are the latest Coupling Facility links. They were announced in February 26, 2008 in US Hardware Announcement 108-154, “IBM System z10 Enterprise Class – The forward-thinking mainframe for the twenty-first century” as well as in the May 6, 2008 US Hardware Announcement 108-269, “IBM System z10 Enterprise Class: Helping to meet global 24x7 demands for information services with improvements for Internet access and coupling,” IFB3 protocol is described in the July 12, 2011 hardware announcement 111-121, “zEnterprise 196 enhancements deliver faster access to data.”
There are several ways of classifying IFB links. It can be Double Data Rate (DDR) or Single Data Rate (SDR). The IFB links can also be characterized by the amount of lanes of communication within each physical link. For example, 12x links provide 12 lanes of communication within each physical link. 1x links provide a single lane of communication within each physical link.
There are many reasons for migrating to PSIFB links.
Like other coupling links, external InfiniBand coupling links support time synchronization signals for Server Time Protocol (STP).
Specifically, a comparison of the effect of the links relative performance can be seen in the table below.
This table assumes:
- The CF activity rate to the coupling facility is 9 operations per million instructions, or 9 operations/second per MIPS of the server. This is typical for a high end data sharing environment. If the activity rate is less, then the values shown in the table should be proportionally less.
- Heuristic Synchronous to Asynchronous conversion is not taken into account. This would limit the coupling overhead to about 18%.
- Shown is data for a non-System-Managed duplexing environment.
|Host||z10 BC||z10 EC||z114||z196||zBC12||zEC12|
|z10 BC ISC3||16||18||17||21||19||24|
|z10 BC 1x IFB||13||14||14||17||18||19|
|z10 BC 12x IFB||12||13||13||15||15||17|
|z10 BC ICB4||10||11||-||-||-||-|
|z10 EC ISC3||16||17||17||21||19||24|
|z10 EC 1x IFB||13||14||14||17||17||19|
|z10 EC 12x IFB||11||12||12||14||14||16|
|z10 EC ICB4||10||10||-||-||-||-|
|z114 1x IFB||13||14||14||17||17||19|
|z114 12x IFB||12||13||12||15||15||17|
|z114 12x IFB3||-||-||10||12||12||13|
|z196 1x IFB||13||14||13||16||16||18|
|z196 12x IFB||11||12||11||14||14||15|
|z196 12x IFB3||-||-||9||11||10||12|
|zBC12 1x IFB||14||15||14||18||17||20|
|zBC12 12x IFB||13||13||12||15||14||17|
|zBC12 12x IFB3||-||-||10||11||11||12|
|zEC12 1x IFB||13||13||13||16||16||18|
|zEC12 12x IFB||11||11||11||13||13||15|
|zEC12 12x IFB3||-||-||9||10||10||11|
Coupling Link types
Coupling links provide a path to transmit/receive Coupling Facility (CF) data as well as Server Time Protocol (STP) timekeeping messages. The CF data may be exchanged between z/OS and the CF or between CFs. For a zEnterprise (z114, z196), the coupling options are:
Note 1: The InfiniBand link data rates do not represent the performance of the link. The actual performance is dependent upon many factors including latency through the adapters, cable lengths, and the type of workload. Systems Lab Services can assist your migration to PSIFB coupling links by providing services to assess the impact of the migration or to assist with the implementation of the migration.
Note 2: The list of qualified DWDM vendors is available on Resource Link, located under the “Hardware products for server” on the Library page.)
The support types of IFB links for the System z servers are:
|Link||1x IFB||12x IFB|
Number of links
|#fanouts||1x IFBs (4 ports / fanout) (HCA3-O LR)
Max 12 fanouts *
|12x IFB(2 ports / fanout)
Max 16 fanouts *
|ISC-3 (spread over 2 fanouts)||IC|
* Maximum 16 HCA-O fanouts per server.
Each fanout can support:
In addition, up to 48 ISC-3 links can be configured spread over two fanouts.
The maximum number of possible external CF links on a z114 M10 is:
48 ISC-3 + 24 1x IFB3s (6 fanouts) = 72
The maximum number of possible external CF links on an M80 is:
48 ISC-3 + 48 1x IFB3s (12 fanouts) + eight 12x IFB3s (4 fanouts) = 104 links
More information regarding Parallel Sysplex using InfiniBand can be obtained from: