The Marriage of Mario (NHPPS) and Luigi (OGCE)

Valdes, Francisco

Pipeline systems, like the plumbing characters of video game fame, have their strengths and weaknesses. This is generally due to the different niches they occupy in the landscape of pipeline systems. This paper describes a marriage between two pipeline systems, NHPPS and OGCE, which capitalizes on combining their strengths.

People frequently discuss interchange of applications elements between pipeline systems. In other words, extracting some pipeline pieces (modules and algorithms) orchestrated by one pipeline system for orchestration by another. However, pipeline systems themselves are applications in their own right. This paper describes an interesting architectural marriage where one pipeline system orchestrates another pipeline system to take advantage of the strengths and niches of both. We point out this concept in general terms since there may be other pipeline marriages possible, but describe a specific example that forms the basis of a major new pipeline application for the WIYN One-Degree Imager (ODI).

The NOAO High Performance Pipeline System (NHPPS), in use for many years at NOAO, is good at orchestrating legacy software, IRAF specifically, into multi-process workflows that effectively make use of a dedicated cluster of multi-core machines without needing to write explicit threaded applications. The Open Grid Computing Environment (OGCE) is designed to wrap and manage applications into Teragrid workflows. For the ODI project we will wrap single-node NHPPS pipelines into OGCE services for orchestration into distributed Teragrid workflows. The many advantages of this married architecture will be described.

Return to oral presentation list