M. Peron, D. Baade, M. A. Albrecht, and P. Grosbøl
European Southern Observatory, Karl-Schwarzschild-Straße 2, D-85748 Garching, Germany
There is a number of situations where users of astronomical observations would benefit from advanced software tools which can analyze the composition of the available data pool:
The above scenarios have in common that the user wants to define a context-specific structure on the observational database. From this, the following requirements on the desired tool can be extracted: It must be able to classify each file into user-definable categories, and it must be able to establish relations between files of different categories (e.g., associate to a given science frame a set of suitable calibration files). The Data Organizer (DO) (Peron et al. 1993) allows users to perform these tasks in an easily customizable way. The main characteristics of the on-line implementation of the DO at the ESO New Technology Telescope (NTT) are described below.
The database for all operations is a MIDAS (ESO-IPG 1993) table, the so-called Observation Summary Table (OST) which contains all relevant parameters extracted from the FITS keywords. Because the Data Organizer is built on existing capabilities of the MIDAS Table File System (Peron et al. 1992), standard relational database operations (e.g., select, merge, copy, and project) may be used.
The information contained in the OST is used to classify the images according to different sets of attributes (e.g., optical elements, calibration sources, etc.). The classification is achieved by applying user-definable rules, and the result of the classification (e.g., an optical path) is saved in the OST. EMMI (Melnick et al. 1992), one of the instruments mounted at the NTT, allows a wide range of observing modes, from wide-field imaging to high-dispersion spectroscopy, including long-slit and multi-object spectroscopy. Because of its complexity, EMMI has been chosen for the first implementation of the DO in an on-line environment.
Figure: The Graphical User Interface to the Data Organizer as installed at the NTT. The main window in the upper left corner shows (part of) the OST which is the database on which all operations supported by the DO are performed. A table-like widget ("Classification rules") can be activated by a pushbutton to edit and modify classification rules. The rule ``BIAS'' for classifying BIAS exposures is being edited in the figure. Another table-like widget (``Classify'') can be used for entering classification rules to be applied as well as the character string which in the OST will identify the selected frames. Original PostScript figure (1732 kB)
The association of scientific frames with suitable calibration images is achieved by selecting and ranking all calibration frames which for a given scientific exposure match a set of user defined selection criteria. One may expand the search by submitting a ``second choice'' set of criteria when not enough frames match the original one. For instance, a search may be expressed in natural language as: ``Find for each scientific frame two dark exposures that have been observed within the same night, and for which the mean detector temperature did not vary by more than 1 degree. If unsuccessful, look for dark exposures observed within the same observing run, and for which the mean detector temperature did not vary by more than 2 degrees.'' Selected frames can be ranked by applying weights to the attributes invoked in the same process. For example, one may give more importance to the time difference than to the detector temperature difference.
The DO has been conceived as a general purpose tool, whereas the implementation at the NTT provides very specific services. In particular, it knows about FITS keywords that characterize EMMI files. This customization was achieved by providing a set of NTT specific configuration tables which contain definitions of different exposure types (e.g., SCI, FFDOME, FFSKY, and BIAS) and instrument modes (e.g., blue imaging, red medium-dispersion spectroscopy). Each time a new file is delivered by the acquisition system, the exposure is classified according to a predefined set of rules, and the result is appended to the OST. Because it is essential that observers can interact efficiently with the tools offered to them in an on-line environment, a versatile Graphical User Interface has been fitted to the DO. It is shown and further explained in Figure 1.
After the adaptation of the association part of the DO to the particular requirements of EMMI, the basis for automatic on-line data reduction will be available. The first MIDAS package to be interfaced to it will be the CCD package. Observers and archival researchers will receive their data together with the corresponding OST. At the NTT, an observatory-supplied calibration database will be maintained. A dedicated OST will provide the overview of that database and enable observers to identify additional calibration observations which could be applicable to their own data. It is expected that at the Very Large Telescope (VLT) the Data Organizer will play a more central role in the on-line data flow.
Peron, M., Ochsenbein, F., & Grosbøl, P. 1992, in Astronomy from Large Databases II, ESO Conference and Workshop Proceedings, No. 43, eds. A. Heck & F. Murtagh (Garching, ESO), p. 433
Peron, M., Albrecht, M. A., & Grosbøl, P. 1994, in Handling and Archiving Data from Ground-Based Telescopes, ESO Conference and Workshop Proceedings, No. 50, eds. M. A. Albrecht & F. Pasian (Garching, ESO), p. 57
Melnick, J., Dekker, H., D'Odorico, S., & Giraud, E. 1994, in EMMI & SUSI, ESO Operating Manual, No. 15, Version 2.0