OPUS is the data processing software system developed at STScI (Rose et al. 1995, Rose 1998, Swade & Rose 1998). In addition to performing data processing for HST, OPUS is currently being used by the FUSE, AXAF, SIRTF, and INTEGRAL missions.
The current data processing ground system configuration for HST utilizes the capabilities of OPUS with regards to constructing a complex data processing system using a number of OPUS pipelines. OPUS pipelines can be run stand-alone, in series, or in parallel. A processing step in one pipeline can bridge to another pipeline. This paper presents concepts on how OPUS can be used to incorporate calibration reference file processing into the pipeline production of science data. Two types of calibration reference data will be addressed.
Generation of routine calibration reference files can be incorporated into an OPUS pipeline. This provides a more timely mechanism for getting better calibrated data to users, as well as saving the manual effort involved in generating standard calibration reference files. The OPUS pipeline would perform the following steps to create and make available a CDBS reference file:
In addition, the newly created calibration reference file is sent to the relevant science instrument team for evaluation of its quality and to be sure no problems exist with the file. The calibration reference file is installed in the CDBS system, after which it will be available for BESTREF.
An example of a particularly labor intensive task is the generation of dark calibration images for the HST WFPC2. This process could not only be automated with an OPUS pipeline, but quality of calibration performed in the pipeline could also be improved.
Currently, the WFPC2 instrument executes five long dark exposures each week. These five exposures are combined to generate a weekly dark CDBS reference file. This file is inspected and placed in the science data processing pipeline. About once a year, each weekly dark taken over that period is combined to form a high quality super-dark. The super-dark is made available to data users through the archive, but is not used in the science data processing pipeline.
An OPUS pipeline could perform the following processing for WFPC2 darks:
Contemporaneous reference files can be handled by OPUS in much the same manner as described above. In this case, there is no point in recording the existence of the file in a database, since it is only applicable to a small number of exposures. A better mechanism is to associate the calibration file directly to the science data through a keyword in the science data header. To generate and immediately use a contemporaneous reference file, an OPUS pipeline would perform the following steps:
The second generation of NICMOS data will utilize a contemporaneous reference file in attempt to calibrate the effect of SAA persistence (Najita, Dickinson, & Holfeltz 1998). Internal dark exposures will be taken immediately after a SAA passage when science exposures are scheduled to follow within 50 minutes. These post-SAA darks will be used to generate a cosmic ray map that can be applied to an image in order to reduce noise from cosmic ray persistence induced by SAA charged particle hits on the detector.
However, generation of the cosmic ray map cannot be automated at this time because it is not possible to remove the DC pedestal prevalent in NICMOS data within a pipeline environment. When, or if, automation of DC pedestal removal is possible, an OPUS data processing pipeline would be capable of utilizing the above scheme to remove SAA persistence.
Both types of calibration reference file pipelines described above can be simultaneously integrated into the current OPUS science data processing system for HST. More timely, or contemporaneous, generation of calibration reference files would improve the quality of data produced by the HST science data processing pipeline.
The OPUS baseline system is currently distributed on CD-ROM to help other institutions with their data processing pipeline management (Swade & Rose 1999). The OPUS CD-ROM comes complete with the Process Manager, the Observation Manager, a set of sample applications, and all the resource files required to get a sample pipeline running. In addition, the OPUS CD-ROM and the sample pipeline are fully documented in the OPUS Frequently Asked Questions (FAQ). The FAQ is available on the OPUS CD-ROM or at www.stsci.edu/software/OPUS/opusfaq.html.
Najita, J., Dickinson, M., & Holfeltz, S. 1998, NICMOS ISR-98-001 (Baltimore: STScI)
Rose, J. Akella, R. Binegar, S. Choo, T. H. Heller-Boyer, C. Hester, T. Hyde, P. Perrine R. Rose, M. A. & Steuerman K. 1995, in ASP Conf. Ser., Vol. 77, Astronomical Data Analysis Software and Systems IV, ed. R. A. Shaw, H. E. Payne, & J. J. E. Hayes (San Francisco: ASP), 429
Rose, J. 1998, in ASP Conf. Ser., Vol. 145, Astronomical Data Analysis Software and Systems VII, ed. R. Albrecht, R. N. Hook, & H. A. Bushouse (San Francisco: ASP), 344
Swade, D. A., & Rose, J. F. 1998 in proceedings of AIAA/USU Conference on Small Satellites (Logan: Utah State University)
Swade, D. A., & Rose, J. F. 1999, in ASP Conf. Ser., Vol. 172, Astronomical Data Analysis Software and Systems VIII, ed. D. M. Mehringer, R. L. Plante, & D. A. Roberts (San Francisco: ASP), 111