Next: A Broadband Wide-Field Survey on the Isaac Newton Telescope
Up: Data Pipelines
Previous: The SCUBA Data Reduction Pipeline: ORAC-DR at the JCMT
Table of Contents - Subject Index - Author Index - Search - PS reprint - PDF reprint

Currie, M. J., Wright, G. S., Bridger, A., & Economou, F. 1999, in ASP Conf. Ser., Vol. 172, Astronomical Data Analysis Software and Systems VIII, eds. D. M. Mehringer, R. L. Plante, & D. A. Roberts (San Francisco: ASP), 175

Data Reduction of Jittered Infrared Images Using the ORAC Pipeline

Malcolm Currie, Gillian Wright, Alan Bridger
UKATC, Royal Observatory, Blackford Hill, Edinburgh, UK

Frossie Economou
Joint Astronomy Centre, 660, N`oh ${\={o\/}}$ k ${\={u\/}}$ Place, Hilo, HI 96720

Abstract:

We relate our experiences using the ORAC data reduction pipeline for jittered images of stars and galaxies. The reduction recipes currently combine applications from several Starlink packages with intelligent Perl recipes to cater to UKIRT data. We describe the recipes and some of the algorithms used, and compare the quality of the resultant mosaics and photometry with the existing facilities.

1. Introduction

The UKIRT ORAC (Observatory Reduction and Acquisition Control) project (Bridger et al. 1998) aims to improve the efficiency of observing at UKIRT by providing a modern software suite that will be capable of fully automated observing. ORAC will also make the telescope and instruments easier to use for traditional observing, by, for example, reducing the amount of manual intervention that is needed.

One of the key components is the ORAC Data Reduction system (ORAC-DR). Its goal is to offer observers near real-time data reduction, and hence let observers accurately assess the quality of their data while they are still at the telescope. Some of the benefits are more efficient use of the telescope, a higher publication rate, and better quality data. ORAC-DR uses a modular pipeline with the intent of producing close to publication quality images and spectra.

This paper concentrates on the imaging, describing some of the data reduction recipes, and lists some pros and cons of using other people's software.

2. Recipes for Imaging Data Reduction

We have written a number of common data reduction recipes and matching observing sequences which cover a range of photometry and imaging techniques in use at UKIRT. A recipe is a series of high level instructions such as ``make a flat'' or ``generate offsets''. The implementation of each of these instructions is through a Perl script, called a primitive, which calls particular data reduction packages, currently from Starlink, to actually do the work.

Many primitives are generic, so new recipes can be created fairly quickly. Most recipes contain a steering primitive to instruct the other primitives what operations should be performed on that image or usually the group of images to which it belongs, and to classify blank-sky from target frames. The recipes run initialisation scripts to enable history recording and specify detector characteristics and subsets, and all apply a bad pixel mask.

The pipeline uses calibration indices for dark, flat, and sky frames. When a calibration frame is required, the recipe accesses the index to find a suitable image, i.e., one which satisfies a set of criteria stored in a rules file. Typical rules are that the filters match and pick the most recent calibration.

In the summary of the jitter-pattern recipes below, the letters in parentheses following the names are codes for processing steps invoked in the recipe. The codes are D=dark subtracted, F=self-flat, M=make mosaic, O=objects masked, R=automatic registration, S=sky subtracted. Section 3 describes these steps.

BRIGHT_POINT_SOURCE: (DORM) It assumes that a SKY_FLAT flat-field is sufficiently stable for a significant fraction of a night not to affect photometry.
EXTENDED_nxm: (DSORM) This is experimental for extended objects where there may be little sky in close proximity for background subtraction. It alternates between jittered blank sky and the extended source, mapping the source in a n column by m row grid with 50% overlap of adjacent target frames. The mosaic is built in rows. At the completion of a row, the interleaved blank-sky frames are combined to make a flat, which is then applied to the source frames in that row. The order within rows and the order the rows are observed is such that adjacent frames are not observed sequentially. This randomising aims to reduce systematic errors from the sky subtraction. The interpolated mode of the bracketing sky frames is subtracted prior to flat-fielding. This method of sky subtraction has worked in practice achieving uniformity to 0.15% in good sky conditions. This is a major improvement over the formerly used IRCAMDR algorithms applied to the same galaxy.
JITTER_SELF_FLAT: (DFORM) This is a 5- or 9-point jitter for point sources.
QUADRANT_JITTER: (DFORM) The original was devised by Lance Miller to search for faint extensions around bright objects, without the need to take separate flat observations, or rely upon the flat's long-term stability. The jitter pattern moves the target source to approximately the centre of each quadrant, and the cycle is repeated to the desired integration time. A preliminary flat is derived from a median of the target frames with the quadrant containing the source excluded. The improved version uses this as a trial flat and masks all the prominent objects to make a superior flat (see the Object Masking description in §3.). The refinement reduces the noise around the central target by 40% and removes artifacts. Sky regions are typically flat to better than 0.01% even in crowded fields.
REDUCE_DARK: This just applies the bad pixel mask and files the calibration.
SKY+JITTERn: (SRM) It is similar to BRIGHT_POINT_SOURCE, except that it subtracts the sky rather than a dark frame, and is intended for moderately faint sources. n = 5, 9. It files the sky calibration.
SKY_FLAT: (D) This forms and files a flat from a 5 (or more) point jittered mosaic of blank sky. There is a variant which masks sources within the frame.

3. Features of the Primitives

Primitives are the Perl scripts which actually call the application tasks to do most of the work. Some tasks were designed with CCDs in mind, so a degree of tuning is sometimes needed to allow for infrared instruments, pixel scale, seeing, or filter, for example.

Flat creation.: Frames are optionally cleaned to remove extreme outliers ( $\pm$ 3 or 6 $\sigma$ about the mean in 15 x 15-pixel neighbourhood), normalised, combined pixel by pixel using a median, and the combined array normalised.
Object Masking.: After the application of an approximate flat field, an inventory is made of objects having at least 12 connected pixels above one sigma above sky. (The optimum thresholds are still being determined.) The locations, shapes, orientations, and sizes are used to make a mask. The mask is applied to the dark-subtracted frames and a new flat created. The improved flat typically shows a uniformity at $\sim$ 0.02% of the sky. Systematic errors in the sky, which are a major uncertainty in infrared point source photometry, are also reduced significantly by this algorithm.
Automatic Registration.: This makes an inventory of the sources above a threshold in each frame. It then performs a pattern recognition (Draper 1998) to identify common features in jittered frames. If the fraction of common objects is under half or the total is fewer than 3, the registration fails, and so the script resorts to reading the offsets stored in the FITS headers or matching a central bright object. Using telescope offsets can lead to trailed sources, as occurred with the IRCAMDR package. The improved registration leads to the detection or more accurate measurement of faint sources.
Mosaicking.: This is fully automated. Observers do not have to painstakingly measure centroids and manually tile to form mosaics. In addition to Cartesian shifts derived from the automatic registration, the primitive applies a preset rotation matrix to align the image to the cardinal directions with only one resampling. Experiments are underway to use jitter offsets aligned with the chip and to put the rotation information into the World Coordinate System to minimise artifacts in the data and to save CPU cycles.
The mosaicking uses the CCDPACK algorithm (Warren-Smith 1993). Only zero-point shifts of intensity are applied to the resampled frames to create the mosaic. Depending on the recipe, the mosaic may be trimmed to the dimensions of a raw frame. Mosaicking removes virtually all the bad pixels.
Automatic Photometry.: This is adaptive. It locates the source, measures its point spread function, selects a suitable aperture size scaled in terms of the FWHM and a concentric sky annulus, integrates the flux within the object aperture, and subtracts the modal sky value. The previous software used a fixed aperture and the median sky from a narrow annulus, and hence could generate less accurate photometry. The script also evaluates a correction for light outside the aperture. The results presented include a standard extinction coefficient and zero-point and a formal measurement error.

4. Pros and Cons of Using Other People's Software

Rather than consisting of large packages dedicated to each instrument, ORAC-DR uses other people's general purpose, data reduction software. This approach reduces maintenance costs, provides greater flexibility (such as to reflect quickly a change in the performance or understanding of an instrument), and has the ability to take advantage of new astronomical software and techniques.

It has its downside too. Sometimes the packages either do not provide what we need for a recipe, or we have to invoke several atomic tasks at the cost of efficiency and intermediate files. For example, the point spread function of UKIRT images are markedly non-Gaussian. The KAPPA/PSF (Currie 1997) only fits Gaussian-like functions, whereas a two-component model is needed. Looking at an extreme case reveals a double hexagonal pattern, for which a general psf-fitting task would never be expected to model. Thus, we expect some recipes will require the odd bespoke application. See Economou et al. (1999) for further discussion on this topic.

Acknowledgments

We thank the Starlink programmers, particularly Peter Draper, for implementing enhancements to packages. We also thank the various astronomers whose data have been used to test ORAC-DR.

References

Bridger, A., Economou, F., Wright, G. S., & Currie, M. J. 1998, in SPIE Proc., Vol. 3349, Observatory Operations to Optimize Scientific Return, ed. P. J. Quinn, (Bellingham: SPIE), 184

Currie, M. J. 1997, KAPPA - Kernel Application Package, (Starlink User Note 95, Rutherford Appleton Laboratory)

Draper, P. W. 1998, CCDPACK - CCD Data Reduction Package, (Starlink User Note 139, Rutherford Appleton Laboratory)

Economou, F., Bridger, A., Wright, G. S., Jenness, T., Currie, M. J., & Adamson, A. 1999 this volume, 11

Warren-Smith, R. F. 1993, in Proc. of the 5th ESO/ST-ECF Data Analysis Workshop, ed. P. Grosbøl & R. Ruijsscher (Garching: ESO), 39

adass@ncsa.uiuc.edu