ADASS 2003 Conference Proceedings

We are developing an automated image reduction and analysis pipeline for WFI (Wide Field Imager, mounted on the 2.2-m MPG/ESO telescope at La Silla) and HST/WFPC2 images to complement Querator, the custom search-engine which accesses the astronomical image archives based at the ST-ECF/ESO centre in Garching, Germany. The image reduction and analysis is performed using an 40-processor Origin SGI based at NUI, Galway. To increase our dataset we complement the reduction and analysis of WFI archival images with the analysis of pre-reduced co-spatial HST/WFPC2 images, and hope to include other archives as data sources. Our pipeline includes image reduction, registration, astrometry and photometry stages. We describe how we overcome such problems as missing or incorrect image meta-data, interference fringing, poor image calibration files, etc, and we discuss how such a pipeline can benefit astrophysical research, specifically the long-term optical variability of Brown Dwarfs. The pipeline was written using tasks contained in the IRAF environment, and linked together with Unix Shell Scripts and Perl.

1. Virtual Observatories

Recent technological advances in observational astronomy have revolutionised the manner and depth at which astronomers image the night sky, and improvements in electronic image acquisition and reduction have increased the information potential which can be gleaned from such exposures. Systematic sky-surveys have seen the rise of image archives stored at multiple sites which can be accessed by astronomers across the globe via high-speed networks. Observed through pre-arranged strips of the sky through different broad-band wavelength filters, the accumulated data is quickly reduced and stored, and can be trawled through in search of many different astronomical phenomena. The concept of a virtual observatory to exploit this increase in the spatial and temporal coverage of the sky has gained currency in recent years and is now being independently implemented by several different groups (Szalay 2001; Quinn et al. 2002; Mann 2002). As part of the AVO's program, ASTROVIRTEL (Pierfederici et al. 2001) has been created with the aim of enhancing the returns of the ST-ECF/ESO archive. Not technically a virtual observatory, it is rather a project to develop and assess the requirements of the AVO. Here, we discuss the development for a fully automated pipeline to reduce and analyse a stream of image data, obtained from the WFI and WFPC2 archives. Automation of the image-processing pipeline is not only necessary, because of the sheer volume of data streaming in each night; it is also beneficial, as it ensures that this consistency is maintained in the data reduction.

2. Acquisition & Preliminary Reduction

The pipeline (see above for a schematic) consists of two stages; data acquisition and image processing. The former is controlled by ``Querator'' (Pierfederici 2001), an on-line request form linked to the ST-ECF/ESO archives. Using either a sky-box (in celestial coordinates) or a recognisable object name (which is passed into a database which provides basic data, cross-identifications and/or bibliographies for astronomical objects) the program returns all images satisfying some set conditions. The success of the pipeline is heavily dependent on the image meta-data (generally as header keywords in the FITS files, but it can also be in the form of pixel masks, observation logs, etc). Although recent survey data have near-complete meta-data, in-depth analysis of older image sources can be severely compromised by a dearth of same. Some such cases can be flagged as less than optimal without interrupting the pipeline's flow. However, where vital information is lacking, the pipeline breaks down until this can be recovered.

To approach photon-counting accuracy in long, integrated exposures requires the removal of the instrumental (CCD) signature from the data, insofar as this is possible. This includes the subtraction of a bias offset (applied to ensure positive-valued pixels) and, occasionally, a thermally-induced dark current, which scales with exposure time. Time-dependent pixel-to-pixel sensitivity variations across the detector are corrected by the division of a flat-field frame, which is an exposure of constant illumination across the detector. This images either an ``empty'' region of the twilight sky or the inside of the observatory dome in unfocused mode. Defective pixel lists are usually supplied by the observatory and can easily be corrected by either interpolating the nearest neighbours or by flagging them as defective. There also exist algorithms to deal with extrinsic or transient sources of bad pixels, such as cosmic ray hits. Fringing, caused by multiple reflections from penetrating long-wavelength photons in thinned CCDs, is corrected by subtracting a fringing template, although automatically accessing the pattern on the science image in order to determine the correct scaling is non-trivial.

**Figure 1:** (l to r) Raw and processed (using the pipeline) WFI images 47 Tuc; an example of WFI fringing and the corrective fringing template.
$\begin{figure}\epsscale{.2} \plotone{P4-21_f2.eps}\plotone{P4-21_f3.eps}\plotone{P4-21_f4.eps}\plotone{P4-21_f5.eps} \end{figure}$

Images should ideally be reduced using calibration frames (with identical instrumental configurations - readout speed, filter, etc.) taken on the same night. Otherwise the ``next-best'' calibration frames are used, and the resultant images flagged as such. After the initial image selection phase of the pipeline, preliminary CCD reduction (as mentioned above) is carried out on all WFI images for all nights. Figure 1 shows the results of the automated reduction on a WFI image of 47 Tucanae. The WFPC2 dataset is pre-reduced, on retrieval, and can be shunted straight through to the analysis stage of the pipeline.

3. Image Manipulation and Analysis

Obtaining the geometrical transformation necessary to register multiple images of the same field but of different spatial alignment is crucial if photometry/astrometry is to be performed a set of images. An adequate representation of the World Coordinate System (WCS), which defines the relationship between pixel coordinates in the image and sky coordinates, sometimes resides within the meta-data. Otherwise we resort to pattern matching under the assumption that several stars are common to the images in question. This is an iterative process which aids the identification and exclusion of incorrectly matched tie-points from the fit. Following registration, photometry (aperture or profile-fitting eg, the DAOPHOT algorithms (Stetson 1987)) and/or astrometry can be performed on selected objects in each image. geometrical transformation is obtained, astrometry involves the simple matter of measuring star centres (either by fitting a Gaussian profile or by centroiding).

**Figure 2:** Colour-magnitude diagrams obtained from the automatic photometric analysis of WFI 47 Tuc images.
$\begin{figure} \epsscale{0.5} \plotone{P4-21_f6.eps} \end{figure}$

4. The Pipeline Model

The pipeline was constructed using the tasks and scripting language contained in the IRAF (Image Reduction and Analysis Facility) package. This is complemented by Unix shell scripts (invoked directly from within the IRAF environment). For more complicated or unorthodox tasks FORTRAN code can be linked to the IRAF environment. Access to external catalogues can be provided using Perl. Various applications (for example image matching/subtraction algorithms (Tomaney & Crotts 1996; Alard & Lupton 1998) can also be incorporated into the pipeline. We have chosen this form of pipeline implementation (performed on an 40-processor SGI Origin 3800) rather than coding directly from first principles because of the obvious gains in development speed and flexibility. Although not the most computationally efficient solution, computational speed is not a bottleneck for our purposes

5. Future Work

Although the primary science goal of this project is to assist in the detection of optical variations in brown dwarfs (indicative of atmospheric activity), it could provide solutions to a wide array of other astrophysical problems provided a robust automated pipeline is supplied with relevant data. Our next step is to add the ability to reduce data from different instruments to our pipeline. For example, adding a spectroscopic reduction and analysis component would greatly enhance the scientific return in analysing different stellar species.

References

Pierfederici, F., Benvenuti P., Micol A., Pirenne B., Wicenec A. 2001, in ASP Conf. Ser., Vol. 238, Astronomical Data Analysis Software and Systems X, ed. F. R. Harnden, Jr., Francis A. Primini, & Harry E. Payne (San Francisco: ASP), 10, 141

Quinn, P. J. Benvenuti P., Diamond P. J., Genova F., Lawrence A., Mellier Y. 2002, SPIE, 4846, 1

Automated reduction and analysis of images from multiple data archives