Next: Data Processing for the Siberian Solar Radio Telescope
Previous: Near-IR Imaging of Star-forming Regions with IRAF
Up: Science Software Applications
Table of Contents - Index - PS reprint - PDF reprint
M. Abbott, T. Kilsdonk, C. Christian,1 and E. Olson
Center for EUV Astrophysics, 2150 Kittredge Street,
University of California, Berkeley, CA 94720-5030
M. Conroy, J. Herrero, and R. Brissenden
Smithsonian Astrophysical Observatory, 60 Garden Street,
Cambridge, MA 02138
1also Space Telescope Science Institute, 3700 San Martin Dr., Baltimore, MD 21218
Our primary goal is to provide a set of generic tools for the basic manipulation of event data. We take the word generic to have several implications. The tools should be generic in function, meaning they are useful with any kind of event data and do not depend on details that may be specific to the telescope or instrument that produced the data. However, the tool set should be extensible to include new, possibly instrument-specific, tools.
The tools should also be generic in environment. By that, we mean that the tool set should function both stand-alone and within common data reduction environments, such as the IRAF CL or IDL.
Finally, the tools should not impose any particular format on the event data files, but, instead, be able to adapt to any format.
Event data refers to the very rich data sets produced by photon counting detectors such as those commonly used in high-energy telescopes. Some recent examples of missions that have produced, or will produce, event data are Einstein, ROSAT, EUVE, ASCA, XTE, and AXAF.
In event data, each event (such as a photon) on the detector is tagged with its arrival time, detector position, and other information. These tags are retained through the later processing of the events, so that a scientist analyzing the data can use filtering techniques to examine subsections of the complete data set, or form secondary data products such as images or light curves by binning the data in any desired manner.
Another typical feature of event data sets is the presence of auxiliary information from the telescope or the spacecraft on which it may be operating, in addition to the events themselves. The auxiliary data are most often time-ordered and tabular in nature, and are frequently used to determine the ``quality'' of the event data they accompany.
Because of this structure, event data lend themselves both to a very detailed and flexible analysis, in which examining small numbers of (or even individual) events can be useful, and to the more usual kinds of image analysis. Event data sets will typically consist of several event lists and tables of auxiliary information, along with various filters that may have been established, and images and light curves that have been computed.
Levels of abstraction are imposed on event data in order to analyze and understand it:
ETOOLS addresses the intermediate, second level. We build upon existing libraries, which support basic objects, and provide applications that manipulate event data. But we leave the assignation of scientific meaning and manipulation of higher-level objects to tools (and possibly libraries), which are layered on top of ETOOLS.
The ETOOLS I/O library implements the Event Data Model in an application programming interface, which is then called from the ETOOLS applications. The I/O library calls on existing, basic object libraries.
We have developed a model for event data that describes the objects and operations needed for their analysis. An overview is given here (not all details are described).
Binning is supported by allowing table objects to be opened as image objects. Each table has a list of binning keys; each key indicates the columns defining the bins, the column to sum, and the WCS object to attach to the resulting image.
Filtering is supported as an integral part of reading tables. A list of filtersets is associated with a table; the active set is applied at the time of a read. Each filterset specifies the type of filter object to be applied and the table columns it applies to.
In order to achieve our goals of environment and file format independence, the ETOOLS I/O library is implemented in two layers: generic code and kernels. These layers provide a simple way to isolate the code, which depends on the details of a particular file format.
A kernel must be written for each underlying set of file formats and/or basic object libraries that will be used. Each kernel is responsible for implementing a set of functions-the kernel programming interface or KPI-which are called from the generic code. Kernels must implement missing functionality in the underlying libraries to provide the basic object capabilities assumed by the generic code.
To extend the ETOOLS library and applications to work with a new file format, only a new kernel is needed. See the paper on the implementation of the ASC Data Model (Herrero et al. 1997) in this volume for some examples of kernels under development.
The ETOOLS library and applications are written in ANSI C for maximal portability. Kernels may be implemented in any desired manner as long as the KPI is supported. Kernels may have dependencies on certain run-time environments. ETOOLS sites can select any desired kernels for installation. For example, a site that does not have IRAF installed can build ETOOLS by not including the IRAF-based kernels (perhaps working entirely with FITS).
A suite of non-interactive tools will perform all of the operations supported by the event data model. These are aimed primarily at scripting uses, but will also be suitable for users who prefer a command line interface.
A small number of interactive tools with graphical user interfaces will support the display and editing of datasets, tables, and interval filters. We have adopted SAOtng as the image display for ETOOLS.
Applications are written in ANSI C for portability. In order to function properly inside certain common data reduction environments, applications may contain a very thin ``front end'' of environment-specific code.
The ETOOLS applications will function in at least two environments: as a package of IRAF tasks and as a set of UNIX programs. Further environments can be supported with proper additions to the front end.
The design of the library is completed and coding is ongoing. We are using a prototype kernel based on ST Tables and IRAF images to validate the software. Kernels based on FITS and on the IRAF QPOE file format are in planning and development stages at the ASC, separate from the ETOOLS effort.
Non-interactive tools that use the completed sections of the library have been developed. More will be added as the library progresses. Work is now concentrating on the interactive applications.
The ETOOLS project is planned for completion in early 1997. All products will be available from the Center for EUV Astrophysics and Smithsonian Astrophysical Observatory WWW and ftp sites, as well as other locations to be determined. Look for future announcements of ETOOLS beta and final releases on the ADASS newsgroups.
ETOOLS is supported by NASA contract NAS5-32698 through the Astrophysics Data Program. The Center for EUV Astrophysics is a division of the UC Berkeley Space Sciences Laboratory.
Herrero, J., Oberdorf, O., & Conroy, M. 1997, this volume
Next: Data Processing for the Siberian Solar Radio Telescope
Previous: Near-IR Imaging of Star-forming Regions with IRAF
Up: Science Software Applications
Table of Contents - Index - PS reprint - PDF reprint