Recent technological advances in observational astronomy have revolutionised the manner and depth at which astronomers image the night sky, and improvements in electronic image acquisition and reduction have increased the information potential which can be gleaned from such exposures. Systematic sky-surveys have seen the rise of image archives stored at multiple sites which can be accessed by astronomers across the globe via high-speed networks. Observed through pre-arranged strips of the sky through different broad-band wavelength filters, the accumulated data is quickly reduced and stored, and can be trawled through in search of many different astronomical phenomena. The concept of a virtual observatory to exploit this increase in the spatial and temporal coverage of the sky has gained currency in recent years and is now being independently implemented by several different groups (Szalay 2001; Quinn et al. 2002; Mann 2002). As part of the AVO's program, ASTROVIRTEL (Pierfederici et al. 2001) has been created with the aim of enhancing the returns of the ST-ECF/ESO archive. Not technically a virtual observatory, it is rather a project to develop and assess the requirements of the AVO. Here, we discuss the development for a fully automated pipeline to reduce and analyse a stream of image data, obtained from the WFI and WFPC2 archives. Automation of the image-processing pipeline is not only necessary, because of the sheer volume of data streaming in each night; it is also beneficial, as it ensures that this consistency is maintained in the data reduction.
The pipeline (see above for a schematic) consists of two stages; data acquisition and image processing. The former is controlled by ``Querator'' (Pierfederici 2001), an on-line request form linked to the ST-ECF/ESO archives. Using either a sky-box (in celestial coordinates) or a recognisable object name (which is passed into a database which provides basic data, cross-identifications and/or bibliographies for astronomical objects) the program returns all images satisfying some set conditions. The success of the pipeline is heavily dependent on the image meta-data (generally as header keywords in the FITS files, but it can also be in the form of pixel masks, observation logs, etc). Although recent survey data have near-complete meta-data, in-depth analysis of older image sources can be severely compromised by a dearth of same. Some such cases can be flagged as less than optimal without interrupting the pipeline's flow. However, where vital information is lacking, the pipeline breaks down until this can be recovered.
To approach photon-counting accuracy in long, integrated exposures requires the removal of the instrumental (CCD) signature from the data, insofar as this is possible. This includes the subtraction of a bias offset (applied to ensure positive-valued pixels) and, occasionally, a thermally-induced dark current, which scales with exposure time. Time-dependent pixel-to-pixel sensitivity variations across the detector are corrected by the division of a flat-field frame, which is an exposure of constant illumination across the detector. This images either an ``empty'' region of the twilight sky or the inside of the observatory dome in unfocused mode. Defective pixel lists are usually supplied by the observatory and can easily be corrected by either interpolating the nearest neighbours or by flagging them as defective. There also exist algorithms to deal with extrinsic or transient sources of bad pixels, such as cosmic ray hits. Fringing, caused by multiple reflections from penetrating long-wavelength photons in thinned CCDs, is corrected by subtracting a fringing template, although automatically accessing the pattern on the science image in order to determine the correct scaling is non-trivial.
Images should ideally be reduced using calibration frames (with identical instrumental configurations - readout speed, filter, etc.) taken on the same night. Otherwise the ``next-best'' calibration frames are used, and the resultant images flagged as such. After the initial image selection phase of the pipeline, preliminary CCD reduction (as mentioned above) is carried out on all WFI images for all nights. Figure 1 shows the results of the automated reduction on a WFI image of 47 Tucanae. The WFPC2 dataset is pre-reduced, on retrieval, and can be shunted straight through to the analysis stage of the pipeline.
Obtaining the geometrical transformation necessary to register multiple images of the same field but of different spatial alignment is crucial if photometry/astrometry is to be performed a set of images. An adequate representation of the World Coordinate System (WCS), which defines the relationship between pixel coordinates in the image and sky coordinates, sometimes resides within the meta-data. Otherwise we resort to pattern matching under the assumption that several stars are common to the images in question. This is an iterative process which aids the identification and exclusion of incorrectly matched tie-points from the fit. Following registration, photometry (aperture or profile-fitting eg, the DAOPHOT algorithms (Stetson 1987)) and/or astrometry can be performed on selected objects in each image. geometrical transformation is obtained, astrometry involves the simple matter of measuring star centres (either by fitting a Gaussian profile or by centroiding).
The pipeline was constructed using the tasks and scripting language contained in the IRAF (Image Reduction and Analysis Facility) package. This is complemented by Unix shell scripts (invoked directly from within the IRAF environment). For more complicated or unorthodox tasks FORTRAN code can be linked to the IRAF environment. Access to external catalogues can be provided using Perl. Various applications (for example image matching/subtraction algorithms (Tomaney & Crotts 1996; Alard & Lupton 1998) can also be incorporated into the pipeline. We have chosen this form of pipeline implementation (performed on an 40-processor SGI Origin 3800) rather than coding directly from first principles because of the obvious gains in development speed and flexibility. Although not the most computationally efficient solution, computational speed is not a bottleneck for our purposes
Although the primary science goal of this project is to assist in the detection of optical variations in brown dwarfs (indicative of atmospheric activity), it could provide solutions to a wide array of other astrophysical problems provided a robust automated pipeline is supplied with relevant data. Our next step is to add the ability to reduce data from different instruments to our pipeline. For example, adding a spectroscopic reduction and analysis component would greatly enhance the scientific return in analysing different stellar species.
Alard, C. & Lupton, R. H. 1998, ApJ, 503, 325.
Mann, R. G. et al. 2002, in ASP Conf. Ser., Vol. 281, Astronomical Data Analysis Software and Systems XI, ed. David A. Bohlender, Daniel Durand and T. H. Handley (San Francisco: ASP), 11, 3
Pierfederici, F., Benvenuti P., Micol A., Pirenne B., Wicenec A. 2001, in ASP Conf. Ser., Vol. 238, Astronomical Data Analysis Software and Systems X, ed. F. R. Harnden, Jr., Francis A. Primini, & Harry E. Payne (San Francisco: ASP), 10, 141
Pierfederici, F. 2001, SPIE, 4477, 246
Quinn, P. J. Benvenuti P., Diamond P. J., Genova F., Lawrence A., Mellier Y. 2002, SPIE, 4846, 1
Stetson, P. B. 1987, PASP, 99, 191.
Szalay, A. S. 2001, ADASS, 10, 3
Tomaney, A. B. & Crotts, A. P. S. 1996, AJ, 112, 2872.