Table of Contents -- Index -- PS reprint -- PDF reprint

Astronomical Data Analysis Software and Systems VII
ASP Conference Series, Vol. 145, 1998
Editors: R. Albrecht, R. N. Hook and H. A. Bushouse

The Interaction of the ISO-SWS Pipeline Software and the ISO-SWS Interactive Analysis System

E.Wieprecht^1,2, F.Lahuis^1,3, O.H. Bauer², D.Boxhoorn³, R.Huygen⁴, D.Kester³, K.J.Leech¹, P.Roelfsema³, E. Sturm ², N.J.Sym ³ and B.Vandenbussche^1,4

¹ ISO Science Operations Centre, ESA SSD, Villafranca,28080 Madrid,Spain

² Max Planck Institut für extraterrestrische Physik, Giessenbachstrasse 1, D-85748 Garching, Germany

³ Space Research Organisation of the Netherlands,PO 800, 9700AV, Groningen, The Netherlands

⁴ Katholieke Universiteit Leuven , Institut voor Sterrenkunde, Celestijnenlaan 200 B, B-3001 Heverlee, Belgium

 

Abstract:

We describe the ISO SWS Pipeline software and ISO SWS Interactive Analysis system ( IA³ ) on different hardware platforms. The IA² system design (Roelfsema et al. 1993) has been reviewed and major parts of the concept have been changed for the operating IA³ system.

The pipeline software is coded in FORTRAN, to work within an environment designed by the European Space Agency (ESA). It is used for bulk processing of ISO data without human interaction, the final product being distributed to the observers. The pipeline s/w is designed in a modular way, with all major steps separated in software sub-modules.

The IA³ system is set up as a tool box in the Interactive Data Language environment (IDL- a trademark of Research Systems Inc. (RSI)). IA³ has been developed to fulfill tasks like debugging of pipeline software, monitoring and improve instrument performance and related software, as well for scientific analysis of SWS data. The FORTRAN pipeline modules are included in this system. Thus, it is possible to execute the pipeline step by step within the IA³ system. The whole IA³ system is controlled by a configuration control system.

         

1. ISO and the Short Wavelength Spectrometer SWS

The Infrared Space Observatory (Kessler et al. 1996) is a satellite observatory carrying out astronomical observations in the wavelength range from 2 to 200 $\mu$m using a telescope with a primary mirror of 60 cm diameter. The Short Wavelength Spectrometer (SWS) is one of the four instruments of ISO. The spectrometer covers the wavelength range 2-45 $\mu$m by two independent gratings and two Fabry Perots. A spectral resolution between 1000 and 20000 can be obtained.

2. The ISO Downlink and overall Data Reduction

The ISO satellite is an observatory on an highly elliptical orbit. The apogee is about 71 000 km, the perigee about 1 000 km. One period is about 24 hours. Thus ISO crosses the radiation belt every day. This limits the actual observation time to 16 hours per day. Because ISO has no on board data storage it is necessary to establish permanent ground contact during observation periods. This has been guaranteed by the ground stations Villafranca de Castillo (Spain) and Goldstone (USA). The raw data are telemetred down and directly stored in an archive. In parallel the data are processed in real time by the RTA (Real Time Assessment)/QLA (Quick Look Analysis) system to monitor the ongoing observation.

After collecting the data from one revolution, where different instruments have been active, the data are processed by the pipeline to produce products which are stored in the archive and distributed to the observer on CD.

After extraction from the archive, it is possible to use the IA³ system to run the SWS specific pipeline step by step for calibration purposes by the Instrument Dedicated Teams (IDTs) at Vilspa. Also the observer might use this s/w by visiting one of the data centers which are running IA³ . Here the most recent software is supplied together with the knowledge of the instrument experts.

3. The ISO Pipeline System

The ISO pipeline has been designed for bulk processing. Thus special requirements for robustness had to be accomplished. Due to coding standards it has been required to code the system in FORTRAN. As operation system VAX/VMS has been defined. The overall design of the SWS specific software has been in a modular way to guarantee the step by step processing within the IA³ system.

The ISO pipeline software can be split into two parts. The instrument independent part which has been done by ESA together with the overall design, and the instrument specific part designed and coded by the instrument teams. The pipeline software has been under configuration control and responsibility of ESA. The SWS instrument team developed a special configuration control system which allows the IA³ developers to cooperate in a controlled environment, even they are located at different sites with different h/w (Huygen,R. & Vandenbussche,B.). Because IA³ development and SWS instrument specific software development are related to each other, the SWS instrument specific software has been covered by this system in a special manner.

3.1. SWS Derive Standard Processed Data

After the instrument independent processing, where basically the telemetry file is split into a number of files in a certain time sequence, instrument, for every observation the DERIVE SPD step is following. Here the data are processed in blocks of one reset interval : 1.collect data, 2.determine usable data range, 3.correct reset pulse after effects, 4.measurement linearisation, 5.cross talk removal, 6.glitch recognition/removal, 7.write Trend Analysis glitch table (to database), 8.compute grating/FP positions, 9.slope calculation, 10.convert to voltages, 11.assign wavelength, 12.write product. In almost every step FITS calibration files are used. The products are FITS SPDs which contain for all 52 detectors a wavelength and a slope in $\mu$V/reset with some add on informations which are represented in bit code for each reset interval. The second result is a file containing further glitch information.

3.2. DERIVE Auto Analysis Result

This part of the pipeline covers the calibration. The modular design is even more transparent because every step is called with the complete data set of the whole observation to be processed : 1.determine observation specific parameters, 2.dark current subtraction, 3.responsivity calibration, 4.flux conversion, 5.velocity correction, 6.preparation of Auto Analysis Result, 7.Write product. Again we write to database tables during step 2 (dark currents) and step 4 (the result of the internal flux calibrator). The AAR product is the starting point for the Infrared Spectroscopy Analysis Package (Sturm et al. 1998), but within IA³ all further steps of scientific data processing can be taken as flatfielding, sigma clipping, rebinning, unit conversions, etc.

4. The SWS Interactive Analysis System

Originally it was intended to have a system which fulfils the requirements : - Debugging of pipeline software. Especially for this requirement it was necessary to design an interface between IDL and FORTRAN which considers the characteristics of different hardware platforms/operating systems. - Analyse the performance of SWS. Regular procedures which are executed to check the behaviour of the instrument has been included. - Trouble shooting. - Determine the calibration parameters. An archive of calibration files and their time dependencies is available within IA³ . The procedures are described and the modules which has been used are included. - Trend Analysis. Certain tools to study the trends of a lot of instrument parameters (as instrument temperatures) and data (as results of internal calibrator checks).

But after the development phase it turned out that IA³ was used more and more as a very handy tool for Scientific Analysis of SWS data and more effort has been spent to add tools specifically for data analysis. Now it is THE SWS data analysis software.

4.1. Design Aspects and Configuration Control

IA³ is designed to run on work station class machines (DEC Alpha, Sun Sparc, etc. ) under different operating systems (VAX/VMS, Alpha/AXP, HP-UX, SunOS, etc.). It has been decided to set up the IA³ system on IDL v3.6.1. Because IDL is hardware independent only the specific handling of the FORTRAN code had to be considered.

The SWS configuration control system is not only a configuration control system. It takes also care of system specific compiling, linking, treating object libraries, treating sharable images of the FORTRAN and C code. It is also controls the regular online HELP system update which is an add on to the standard IDL help.

4.2. The IA³ system as a tool box

IA³ is set up as a tool box running under the IDL. Although IA³ is presented as an user-oriented analysis system, it is actually designed as a programmer-oriented environment. Thus programmer freedom and flexibility is woven into the system wherever possible. This way it is possible to use the full power of IDL plus instrument specific features. The major part of IA³ is keystroke driven. But for many modules which fulfill a certain high level task proper WIDGET GUIs have been coded. IA³ contains a large number of data structure definitions representing SWS data in a form closely to the SWS standard products/files. As described it is easy to run the pipeline step by step for various reasons. For most of the pipeline steps the system contains at least one alternative which might be highly interactive or simply an improved pipeline module which has been given to the IA³ system for test. Sometimes even pure IDL might do the job. Pipeline FORTRAN modules are accessed via an IA³ shell script with the input data structure and necessary calibration file data structures, the structure elements are converted to FORTRAN77 arrays and the parameter are passed to the FORTRAN pipeline module using a special shell procedure which solves the machine dependencies of the IDL CALL_EXTERNAL procedure (library names, entry points, etc.). The user can specify the calibration files they want to use before starting the pipeline command by a GUI select calfile tool. Here the user specifies to use the actual pipeline, test or a self defined calibration file. But the user has also the possibility to specify calibration files on the command line level within the call of the pipeline module.

5. Summary and further plans

The pipeline system has reached version 6.2 and performs well for SWS. Improvements are under development. The DERIVE AAR step contains a few parts where it is recommended to use IA³ tools. The DERIVE AAR part has been recoded into IDL and it is intended to use this code also within the standard pipeline (version 7.0). It is also planned to incorporate the DERIVE_AAR software together with all relevant data reduction IA³ tools to a subset IA³ system which is basing purely on IDL and gives significant support of scientific work on SWS data. This subset will be distributed. Finally it is under investigation to switch to IDL version 5.

References:

Huygen.R & Vandenbussche B. 1997, ``A Configuration Control and Software Management System for Distributed Multi-platform Software Development", in Astronomical Data Analysis Software and Systems VI, ASP Conf. Ser., Vol. 125, eds. Gareth Hunt and H. E. Payne (San Francisco, ASP), 345

Kessler,M.F. et al. 1996, A&A 15 The Infrared Space Observatory (ISO) mission

Lahuis, F. 1997, ``ISO SWS Data Analysis", this volume

Roelfsema,P.R. et al. 1993, ``The ISO-SWS Off-Line System", in Astronomical Data Analysis Software and Systems II, ASP Conf. Ser., Vol. 52, eds. R.J. Hanisch, R.J.V. Brissenden, & J. Barnes (San Francisco, ASP), 254

Sturm, E. et al. 1998,``The ISO Spectral Analysis Package ISAP", this volume

© Copyright 1998 Astronomical Society of the Pacific, 390 Ashton Avenue, San Francisco, California 94112, USA