Next: ISAS Data Archive and Transmission System (DARTS)
Up: Archiving
Previous: Object Oriented Near Real Time Packet Distribution as Part of the FIRST Integrated Network and Archive System
Table of Contents - Subject Index - Author Index - PS reprint - Werger, M., Bennett, K., Hazell, A., O'Mullane, W., Oosterbroek, T., & Parmar, A. 2000, in ASP Conf. Ser., Vol. 216, Astronomical Data Analysis Software and Systems IX, eds. N. Manset, C. Veillet, D. Crabtree (San Francisco: ASP), 176

The BeppoSAX Low-Energy Concentrator Spectrometer Archive at the Space Science Department of ESA

M. Werger, K. Bennett, A. Hazell, W. O'Mullane, T. Oosterbroek, A. Parmar
Astrophysics Division of ESA, ESTEC, Noordwijk, The Netherlands

Abstract:

The Astrophysics Division of the European Space Agency (ESA) has developed a data analysis and archive system in order to provide scientific products of a consistent quality for the BeppoSAX Mission. The archive is principally used as a tool to support the calibration of the Low-Energy Concentrator Spectrometer (LECS) instrument - supplied by the Astrophysics Division - onboard BeppoSAX. This is an imaging X-ray detector sensitive in the energy range 0.1-10 keV with an energy resolution of 9% at 6 keV, and a 40' diameter circular field of view. The spatial resolution is around 1.5' full-width at half-maximum (FWHM) and the finest time resolution available is 16 microseconds.

With minimal operator intervention, all data handling and reduction tasks for the archive are performed in an integrated manner on a daily basis. We show the user interface, the set-up of the pipeline, and the data archive. All three components contain elements common to other scientific missions of the Astrophysics Division. While some parts are specific to the particular database system used, or to the LECS energy range, the overall design may serve as a template for future data archives.

1. Satellite and Instruments

The satellite BeppoSAX, a major program of the Italian Space Agency (ASI) with participation of the Netherlands Agency for Aerospace Programs (NIVR), was launched on April 30, 1996 from Cape Canaveral. The satellite payload consists of four narrow-field instruments (NFIs) and two wide-field cameras (WFCs) (Boella et al. 1997).

The NFIs are the Low Energy Concentrator Spectrometer (LECS, energy range 0.1 - 10 keV), the Medium Energy Concentrator Spectrometer (MECS, 1.3 - 10 keV), a collimated High Pressure Gas Scintillation Proportional Counter (HPGSPC, 4 - 120 keV), and a collimated Phoswich Detector System (PDS, 15-300 keV). Thus the NFIs cover more than three decades of energy - from 0.1 to 300 keV - with a relatively large area, a medium energy resolution and imaging capabilities in the range of 0.1 - 10 keV with an approximate resolution of 1.5'.

Both WFCs are oriented perpendicular to the NFIs looking in opposite directions. They cover an energy range from 2 - 30 keV with an angular resolution about 5'. While the NFIs image only a field of 40' diameter for LECS and 60' diameter for the MECS to 1.3$^\circ$ for PDS, the WFCs viewing field measures 20$^\circ$ by 20$^\circ$ FWHM. BeppoSAX orbits the earth in a circular orbit with an altitude of about 600 km and an inclination of 3.9$^\circ$. During the orbital period of 96 minutes, there is only a single 10 minute contact with the ground station in Malindi, Kenya, during which all data is downloaded and forthcoming commands are uploaded.

2. The Archive Software

Figure 1: The SAXbrowse User Interface using Oracle FORMS.

The archive software may be taken as a typical example of rapid application development. Data-types and handling as well as the archive structure are constrained by user requirements. The full archive software is implemented in a pre-existing environment, determined by the database, the physical data archive, and the computer network. Consistency and arbitrary usability of data is guaranteed by using a commercial database product (Oracle 7.3) to store all mission science related data. Data access is significantly constrained in methodological terms: database transactions are performed using only the Oracle tools, e.g. sqlloader and sqlplus, or the Perl DBI module. Data relationships and descriptions are written in database independent schema files. This enables easy adaption of the archive software and database tables during the development and maintenance process. Simultaneous prototyping resulted in a software suite which encompasses the full range of functions including creation of the archive database, population of the tables, and performing other routine tasks. This software suite is running on several clustered UNIX workstations with Sun Solaris 2.5 or higher. Software development is facilitated by use of the Concurrent Versions System (CVS) which allows easy tracking and merging of changes. Accordingly, a central repository holds all related files. While in principal CVS is command line driven, queries about source code may be made using CVSWEB, an optionally available CGI script which has been adapted to local requirements.

3. The Pipeline

The Final Observation Tapes (FOTs) are received from the SAX Scientific Data Centre (SDC) in Rome, Italy. Prior to any actual scientific analysis with popular X-ray software packages (e.g. Xspec, Ximage), the FOT data have to be pre-processed using a system of dedicated instrument-specific programs. This software is referred to as the SAX Data Analysis System (SAXDAS). The first step is ingesting the FOT to produce the raw FITS event files. The following steps are: creation of the linearized event lists from the raw event files, instrument response generation, and creation of standard background files. The other instruments (i.e. MECS, HPGSPC, and PDS) data are treated similarly. The data files are then scientifically exploited in several steps. First of all, data from the MECS detectors are merged to improve statistics. Then images are created from these products. Source detection is then performed with Ximage. Finally, creation of spectra and lightcurves for detected sources follows. Spectral fitting is performed separately for the LECS and MECS, and also on a combination of both data sets. The response matrix appropriate for the position in the LECS field of view is used. The LECS and MECS images are checked for spatial coincidence of sources. Thus, the database contains data of all scientific products, i.e. LECS and MECS data. This allows the use of the database for several tasks:

• Selection of suitable sources as calibration objects,
• Long-term monitoring, e.g. of instrumental background,
• Long-term checks, e.g. on temperature-gain relation.

4. The User Interface

The first of the available user interfaces has been built using the Oracle FORMS interface (see Figure 1). Here the user has full access to all project related tables and contents. Through the top level database menu data from the observation log, the list of received FOTs, the cleaned event lists, spectra, and spectral fits are shown. Simple queries may be entered directly. Complex SQL queries may be designed with the Oracle Browser. The second user interface provides a quick-look facility by means of a set of HTML pages using pre-formatted text and frames for fast access and simple navigation. The generation of these pages is done by UNIX cron jobs. These HTML pages summarize performed observations, available FOTs, related products of the pipeline, and the relations to each other. In addition, a simple HTML form interface enables primitive queries which formats the search result in the same style as the other HTML pages. A cone search of cleaned event lists is available through a Perl/Tk or an Oracle FORMS interface.

5. The Archive

All project related files are stored in a hierarchically organized archive. This consists of two Hewlett-Packard juke-boxes with magneto-optical platters (1.3 or 2.6 GB for each medium) and one tape-robot manufactured by ADIC-Grau handling Sony AIT tapes (25 GB uncompressed for each tape). The optical juke-boxes are used like external drives attached to the SCSI bus of the server, while the tape-robot communicates via TCP/IP on the LAN and via a direct SCSI connection with the server. Special server software (AMASS, DataMgr, and DAS by ADIC, formerly EMASS) makes accessing this archive fully transparent to the user. Files are cached both on NFS exported directories and on hard-disks in the server for improved access of frequently and/or recently used files. Loading/Unloading media from the drives results in a non-negligible response time which is typically less than 30 seconds; maximum read throughput is of the order of 2 MB/s. For performance issues, small files are mainly held on optical platters, while files larger than a certain limit (several 10 MB) are generally kept on tapes. The LECS archive currently holds about 1300 FOTs of approximately 400 unique observation targets. Additionally, 3000 product files are available: about 1040 images and 1950 spectra for all available observations. Mean file sizes for the raw FOTs are about 64 MB (which may exceed 200 MB for single FOTs), for cleaned event lists about 27 MB, and for housekeeping files about 21 MB. All other files sizes are typically less than 1 MB. Currently, in total 23000 files are stored on optical platters or tape and the total used space is about 222 GB. The archive is used also by other ESA projects, e.g. COMPTEL, Hipparcos, EXOSAT, and Planck. The total available space is 12 TB - if all possible media (more than 800) would be in use.

6. Conclusions

It is possible to quickly build useful systems like the one described using available tools and resources. Using a commercial database management system (DMBS) and/or a repository supports the implementation of such systems however a well engineered design of the final layout is still needed.

References

Boella, R. et al. 1997, BeppoSAX, the wide band mission for X-Ray Astronomy, A&AS, 122, 299, and references therein

© Copyright 2000 Astronomical Society of the Pacific, 390 Ashton Avenue, San Francisco, California 94112, USA Next: ISAS Data Archive and Transmission System (DARTS)
Up: Archiving
Previous: Object Oriented Near Real Time Packet Distribution as Part of the FIRST Integrated Network and Archive System
Table of Contents - Subject Index - Author Index - PS reprint -