Recording the history of Herschel Data Processing in the data products.

De Jong, Jeroen

For the scientific assessment of any products generated by a Data Processing (DP) system it is important to know how the product has been generated. Therefore, such a DP system should somehow record the history of commands used to generate the product. We present how the history is recorded during Herschel data processing, and how users can inspect this history record in the Herschel Interactive Processing Environment (HIPE). The Herschel DP software records after the execution of each (pipeline) task all information needed to redo the task. This includes the input parameters and their values that were used during processing, and in case an input parameter is a product, the complete history of that product is included. In this way one creates in the final product the complete chain of tasks needed to reproduce that product from the raw data. The user can even retrieve this chain as a Python script. Furthermore, software build numbers and human readable names of calibration files are stored as well. With this the users can easily see with which versions their data has been processed. When the product is saved in a FITS file then all the history information is stored in two binary FITS tables (one for the tasks and one for the parameters). Additionally, the corresponding Python script is saved in another table, just for convenience in case users want to inspect the file with other software.

Return to poster list