German Researchers Use IBM Big Data Solution To Manage World’s Largest Trove Of Climate Data

Storage software and IT services will support climate, environmental research on +500 petabytes of information, used by IPCC, Max-Planck, other leading researchers

ARMONK, N.Y. and HAMBURG, GERMANY - 08 May 2015: The German Climate Computing Center is using technology and services from IBM (NYSE: IBM) to manage the world’s largest climate simulation data archive, used by leading climate researchers worldwide.

The archive currently consists of more than 40 petabytes of data and is projected to grow by roughly 75 petabytes annually over the next five years. For comparison, a petabyte of data is equal to roughly 210,000 DVDs. As climate simulations are carried out on increasingly powerful supercomputers, massive amounts of data are produced that must be effectively stored and analyzed.

To efficiently manage and provide quick access to this amount of data for analysis and research through its Hierarchical Storage Management system, the Center has selected the High Performance Storage System software solution developed by IBM and the U.S. Department of Energy. Supported by IBM services, this solution is capable of handling more than 500 petabytes of data.

Organizations including the Intergovernmental Panel on Climate Change, the Max-Planck Institute for Meteorology, the University of Hamburg, the Helmholtz-Zentrum Geesthacht Center for Materials and Coastal Research, the Alfred Wegener Institute Helmholtz Center for Polar and Marine Research, the West African Science Service Center on Climate Change and Adapted Land Use, The Coordinated Energy and Water Cycle Observation Project, the Program For Earth System Modeling, and the Center itself use data in the archive for a range of research and reporting activities, including:

  • IPCC research and reporting on the increase in the earth’s temperature and its impact on the climate;
  • Simulations to help predict the geographical and environmental impact of oil spills;
  • The impact of climate change on West Africa, the Mediterranean, Central Europe, Indonesia and other key global regions to determine how to adapt to it;
  • Analysis for airplane traffic routes and altitudes to reduce the emission of carbon dioxide and other gases that may impact climate;
  • Developing 3D models and visualizations for creating smarter cities and more energy-efficient buildings;
  • High-definition modeling for the formation of clouds and precipitation to improve the accuracy of weather and climate prediction.

“Addressing the challenges of climate change and its impact on the environment and daily life for everyone on the planet requires gathering, storing and analyzing massive amounts of data,” said Professor Thomas Ludwig, director of the German Climate Computing Center and professor of computer science at the University of Hamburg. “We rely on the technology and the expertise of IBM to help us provide this essential service to the world’s foremost climate and environmental research institutions.”

The upgrade of the Center’s Hierarchical Storage Management system includes a solution consisting of the latest x86 servers and Red Hat Linux operating system running the HPSS software. This solution manages all simulation data and serves as the input and output interface between the Center’s high performance computing systems and the tape storage library where the climate data is stored.

“At IBM, one of the values we try to live up to daily is providing innovation that matters for our company and the world,” said Markus Koerner, vice president, IBM Global Technology Services. “I can think nothing that matters more than helping to effectively predict and mitigate the impact of climate change, and we believe the technology and services we provide for this Big Data challenge will significantly help advance the science of climate change.”

The solution uses a disk cache of five petabytes of storage capacity and is capable of providing data access speeds of up to 12 gigabytes per second, which will be upgraded to 18 gigabytes per second later this year. IBM Global Technology Services supports this solution with data center managed services on site, project management services, and hardware vendor management services, and the HPSS solution incorporates IBM DB2 scalable relational database software.

HPSS technology is the result of a 20-year collaborative effort between IBM and the Lawrence Livermore, Los Alamos, Sandia, Lawrence Berkeley and Oak Ridge National Laboratories of the U.S. Department of Energy. It is used for very large scientific data archives at research facilities and universities, and for weather forecast systems.

About the German Climate Computing Center

For further information about the Center, go to:

About IBM

For more information about IBM Global Technology Services, please visit:

Related resources


Shown here is one of the disc enclosures In the High Performance Storage System IBM is providing to help the German Climate Computing Center manage the world's largest archive of climate data. The solution provides 5 petabytes of disk cache and access speeds of up to 12 gigabytes per second to more than 40 petabytes of climate data for leading researchers studying weather, the environment and the effects of climate change. (Credit: German Climate Computing Center)

This image shows a simulation of the earth's temperature in the year 2090, based on research conducted on data at the German Climate Computing Center. Such simulations and other research on the more than 40 petabytes of data at the center require a solution and services from IBM to provide quick, efficient access to such a large archive of information. (Credit: German Climate Computing Center)

The German Climate Computing Center in Hamburg hosts the largest archive of climate data in the world -- currently 40 petabytes of information, and projected to grow by as much as 75 petabytes annually over the next five years, requiring a High Performance Storage System from IBM. For comparison, one petabyte equals roughly 210,000 DVDs. The solution uses software developed by IBM and the U.S. Department of Energy, is supported by IBM services and incorporates IBM DB2 database software. (Credit: German Climate Computing Center)

Related XML feeds
Topics XML feeds
Energy & Utilities
News about IBM solutions for the energy and utility industries
News about IBM solutions for federal, state, and local governments
IBM Analytics
News about IBM solutions that turn information into actionable insights.
IBM Storage
Storage software, tape and disk innovations
Services and solutions