A new approach to high performance computing

Once completed, the MareNostrum supercomputer will be put to work unraveling some of science's knottiest problems, in partnership with businesses and research institutions around the world. Its to-do list includes issues that are familiar in the supercomputing world, such as protein folding, in silico drug screening and enzymatic reactions.

But the thinking behind MareNostrum's construction represents a new way of looking at these and other compute-intensive areas. Today's typical high performance computing installation runs a large, parallel RISC-based UNIX system, with performance, not reliability, of utmost importance. MareNostrum, however, is a small-footprint Linux cluster made up entirely of off-the-shelf components. With the extreme density of IBM eServer BladeCenter JS20 servers, diskless nodes and an open system environment, MareNostrum offers superior price/performance; greater reliability, availability and serviceability; and significant cost efficiencies—factors that are endearing Linux-based cluster servers to more and more businesses all the time.

Here's what distinguishes the high performance computing strategy behind MareNostrum:

Servers
There are 2282 IBM eServer BladeCenter JS20 servers housed in 163 BladeCenters chassis. Each server Blade has two Power PC 970 processors running at 2.20GHz, providing superior performance for several varieties of Linux. Ultra slim and powerful, the BladeCenter technology offers the highest commercially available computer density in the industry. And a smaller footprint without sacrifices in performance. The BladeCenter technology allows for 84 dual processor servers in a single 42 U rack. This results in over 1.4 Teraflops of compute power in a single rack.

Hot-swappable JS20 servers also allows administrators to change servers without disrupting applications, maximizing availability. Its shared-resources architecture helps to minimize power consumption and heat output, as well.

Storage
MareNostrum's storage subsystem consists of 20 storage server nodes with 7 terabytes of capacity each, or 140 terabytes of total capacity. Its backbone is the IBM TotalStorage DS4100 storage server, which, like the BladeCenter JS20, uses redundant, hot-swappable components for high availability. IBM TotalStorage DS4100 technology enables tremendous scalability and a wide range of RAID data protection options.

Switching
Four switch frames with Myrinet, including 10 CLOS 256+256 switches and 2 Spine 1280's and densely bundled Myrinet cabling enables faster parallel processing with less switching hardware. The redundant, hot-swappable power supply ensures greater availability. The complete switch with 12 chassis provides for 2560 uniform ports. This uniformity simplifies the programming model so researches can focus on their programs and not the system interconnect architecture.

Operating system
The Linux 2.6 kernel offers an array of enterprise and performance features that exploit the Power Architecture. The virtualization capabilities of Linux on POWER allow for more flexible partitioning, better balancing of workloads and superior scalability should workloads increase.

Diskless Image Management (DIM)
DIM is a prototype utility for managing the Linux distribution for the compute nodes on the storage servers so that the compute node does not have to manage the root file system. All the files for operation are obtained through the cluster network. Because of this, blades can operate immediately without Linux installation. This is on-demand operation. The blades do have a disk drive but that is reserved for future application use such as checkpointing. DIM also supports the network boot environment in a highly distributed fashion.