InfoSphere DataStage

Integrate all types of data on distributed and mainframe platforms

IBM® InfoSphere® DataStage® integrates data across multiple systems using a high performance parallel framework, and it supports extended metadata management and enterprise connectivity. The scalable platform provides more flexible integration of all types of data, including big data at rest (Hadoop-based) or in motion (stream-based), on distributed and mainframe platforms.

InfoSphere DataStage provides these features and benefits:

Powerful, scalable ETL platform—supports the collection, integration and transformation of large volumes of data, with data structures ranging from simple to complex.
Support for big data and Hadoop—enables you to directly access big data on a distributed file system.
Near real-time data integration—as well as connectivity between data sources and applications.
Workload and business rules management—helps you optimize hardware utilization and prioritize mission-critical tasks.
Ease of use—helps improve speed, flexibility and effectiveness to build, deploy, update and manage your data integration infrastructure.

Powerful, scalable ETL platform

Manages data arriving in near real-time as well as data received on a periodic or scheduled basis.
Provides high-performance processing of very large data volumes.
Leverages the parallel processing capabilities of multiprocessor hardware platforms to help you manage growing data volumes and shrinking batch windows.
Supports heterogeneous data sources and targets in a single job including text files, XML, ERP systems, most databases (including partitioned databases), web services, and business intelligence tools.

Support for big data and Hadoop

Includes support for IBM InfoSphere BigInsights, Cloudera, Apache and Hortonworks Hadoop Distributed File System (HDFS).
Offers Balanced Optimization for Hadoop capabilities to push processing to the data and improve efficiency.
Supports big-data governance including features such as impact analysis and data lineage.

Workload and business rules management

Helps enable policy-driven control of system resources and prioritization of different classes of workloads.
Helps you optimize hardware utilization and prioritize tasks, control job activities where resources exceed specified thresholds, and assess and reassign the priority of jobs as they are submitted into the queue.
Integrates with IBM Operational Decision Management (formerly ILOG JRules), allowing you to implement decision logic within IBM InfoSphere Information Server.

Near real-time data integration

Captures messages from Message Oriented Middleware (MOM) queues using Java Message Services (JMS) or WebSphere MQ adapters, allowing you to combine data into conforming operational and historical analysis perspectives.
Provides a service-oriented architecture (SOA) for publishing data integration logic as shared services that can be reused over the enterprise.
Can simultaneously support high-speed, high reliability requirements of transactional processing and the large volume bulk data requirements of batch processing.

Ease of use

Includes an operations console and interactive debugger for parallel jobs to help you enhance productivity and accelerate problem resolution.
Helps reduce the development and maintenance cycle for data integration projects by simplifying administration and maximizing development resources.
Offers operational intelligence capabilities, smart management of metadata and metadata imports, and parallel debugging capabilities to help enhance productivity when working with partitioned data.

IBM Software

See what smarter software can do for you