Integral Science Data Centre (ISDC) has written 3 layers of libraries on top of CFITSIO (Pence 1999) for data access. DAL (O'Neel, Jennings, Rohlfs, & Paltani 2000) is the data access layer designed to encapsulate CFITSIO and is not ISDC specific. The top layers are called DAL3 and they consist of ISDC specific functions which are both instrument independent and instrument dependent. All of the software described in this paper can be downloaded from the ISDC Download Page and the structure can be seen in Figure 1.
The lowest layer in our system is CFITISO. CFITSIO is a fast efficient, easy to use library for reading and writing FITS files. CFITISO has the ability create FITS files from ASCII templates and this feature is used heavily by ISDC in order to produce a consistent set of products. Programs only fill in existing keywords and columns in precreated files.
Built on top of CFITSIO is ISDC's Data Access Layer or DAL. DAL is a mission independent library written in ANSI standard C that can be called from C, C++ and Fortran. It allows one to build up structures of FITS extensions in a way which is independent of the disk layout of your data but which matches the scientific processing needs for that data. Once this is done you can easily find and select different data structures by name and not just by position, independent of the file location on disk.
In order to build these structures DAL uses FITS binary tables called DAL Groups (Jennings, D., Pence, W., Folk, M., & Schlesinger, B. 1997). These groups, which are analogous to disk file directories, consist of FITS binary tables with 6 predefined columns. Entries in these 6 columns point to other FITS extensions either in the same file, in other files, or to remote systems using FTP or HTTP. Since these groups are nothing more than FITS extensions it is possible for groups to point to other groups allowing one to build hierarchies which match the science needs of the mission. An example showing DAL grouping is shown in Figure 2.
Integral, like many other multi-instrument missions, has mission specific but instrument independent routines. These are stored in our two libraries DAL3AUX and DAL3GEN. DAL3AUX is the library which gives programs easy access to attitude, orbit, and time correlation data. DAL3GEN contains the indexing functions described later as well as support functions for Integral software.
In order to allow programmers to write more consistent programs and standardize certain operations which might be difficult, ISDC wrote a an instrument specific library for each instrument. These libraries call routines in DAL3GEN, DAL3AUX, and DAL but give programmers an easier interface to the instrument data. Since Integral event data is stored in multiple tables, one of the primary purposes of these libraries is to make data selection with cfitsio selection strings easier across multiple tables for different instrument modes.
Index groups expand upon the DAL groups concept above to allow for easy selection of scientifically interesting data. This is done by adding columns to the DAL group which contain the value of interesting keywords in the FITS tables pointed to by the DAL group. For example, if each of your files has a TSTART and TSTOP keyword listing the start and stop time, one would add columns named TSTART and TSTOP to the DAL index group. Then when you added this file to the index the TSTART and TSTOP columns would be filled with the values of the TSTART and TSTOP keywords from the header of your data extension. An example index table is shown in Figure 3.
Once the index is built one can easily select, using CFITSIO selection strings, data which matches your criteria for subsequent processing. In this way you don't have a large mass of data you have a simple table which allows queries for selection.
Jennings, D., Pence, W., Folk, M., & Schlesinger, B, A Hierarchical Grouping Convention for FITS, http://fits.gsfc.nasa.gov/group.html.
O'Neel, B., Jennings, D., Rohlfs, R., & Paltani, S., 2000, in ASP Conf. Ser., Vol. 216, Astronomical Data Analysis Software and Systems IX, ed. N. Manset, C. Veillet, & D. Crabtree (San Francisco: ASP), 187
Pence, W. D., 1999, in ASP Conf. Ser., Vol. 172, Astronomical Data Analysis Software and Systems VIII, ed. David M. Mehringer, Raymond L. Plante, & Douglas A. Roberts (San Francisco: ASP), 487