The Cambridge Infrared Survey Instrument (CIRSI), a near-IR mosaic imager containing a 2x2 array of Rockwell Hawaii I 1024x1024 detectors (Beckett et al. 1996; Mackay et al. 2000), has been in operation for about two years, obtaining almost 1TB of imaging data. The uniquely wide field accessible by CIRSI on a 2-4m class telescope makes it ideal for moderate depth large-area surveys. Preliminary results from two such current surveys include the measurement of galaxy clustering at intermediate redshift (McCarthy et al. 2001), and the demonstration of a reddening-independent quasar selection technique based on combined deep optical and near-IR color diagrams (Sabbey et al. 2001).
However, the CIRSI data reduction poses several challenges. With the large data rate (5-10GB of data taken per night, 100 nights per year, and a significant data backlog currently) the software has to be very efficient and completely automated. Also, the software should handle diverse data sets, from Galactic center observations to very sparse fields at high Galactic latitude. From thousands of individual images taken over many nights, wide-field, deep mosaic images are generated. Since the gaps between detectors comparable to detector size, filled mosaic images are made using the co-added dither sets from different chips and telescope pointings. Thus weight maps and accurate astrometry are crucial and the common simplification of clipping the dither sets to their intersection region is not appropriate.
Although existing software packages are used when possible (see below), the decision was made to write core image processing routines to satisfy certain design requirements. For example, a two pass reduction provides: object masks derived from first pass co-added dither sets, subpixel image registration/co-addition using full-weight maps without image clipping, optimizations for CPU and disk efficiency, customized artifact cleaning (destriping and defringing), and reusable tools (from having stand-alone tasks to be glued together with a high level scripting language, to making C library calls, or even extracting portions of source code). The basic processing steps, described below, are: flatfield correction, running sky frame subtraction, dither offsets measurement, dither set co-addition, and mosaic image creation.
The image stacking is done using cubemean.c, which calculates the median, robust standard deviation, or robust mean plane with a choice of weights (none, scalar, or maps) and image scaling or zero offsets using the image modes. This is not a general purpose tool like IRAF's imcombine, but for the specific task of calculating the median plane was found to be 2.5times faster (for a stack of 50data frames). The flatfield image is converted to a gain map and bad pixels identified using gainmap.c. The data are flatfield corrected using flat.c.
Running sky frame subtraction is normally a significant bottleneck in processing infrared imaging, so optimizations are important. To do running sky frame subtraction for a stack of N images, the program skyfilter.c uses a sliding window (circular buffer of image pointers) to require only N image reads and N image mode calculations. In contrast, putting this logic into a script normally involves N M image reads and mode calculations, where M is the width of the sky filter in frames. Also, the disk I/O (and storage) for non-co-added data uses short integers (2bytes deep), even though most calculations are done in floating point (4bytes). Some calculations work with short integer data however to allow optimizations. For example, the almost trivial distribution sort can be used to obtain an image histogram, sorted image array, and median value in time (5times faster than running an optimized median routine on typical data images).
The cross-correlation technique uses coordinate, magnitude, and shape information and is more reliable than matching object coordinate lists (improvement was noted in extreme cases, e.g., Galactic center images and nearly empty fields with an extended galaxy). Subpixel offset measurement accuracy of pixel is obtained by fitting a parabola to the peak of the cross-correlation image. This cross-correlation method was found to be times faster (for typical survey data and a relatively large search box of 100pixels) than IRAF STSDAS crosscor. Although the success rate is 100%, offset measurements corresponding exactly to the search area border, or a small fraction of object pixels overlapping in the aligned data images, would indicate failure.
With dithercubemean.c, dither frames are combined by calculating the weighted mean pixel value at each position of the dither stack, with pixel values from the median at each position rejected. Images borders are added during registration to avoid clipping the data to the intersection of the dither frames. The standard deviation () at each position is calculated from , where MAD is the median absolute deviation from the median (simpler methods such as minmax rejection are inappropriate when taking averages of small numbers of values, e.g., 5-9frames per dither set). The weight maps are combined by calculating the sum at each position of the stack of weight maps (for pixels not clipped during co-addition).
The current astrometry pipeline, a small SExtractor Perl script that produces an object catalog for each co-added dither set, runs APMCAT (a stand-alone C program) to download over the network the APM sky coordinates of objects in each field of view and then runs IMWCS from WCSTools (Mink 1999) to calculate the astrometry fit and update FITS header WCS information. Co-added dither sets and weight maps from different chips, telescope pointings, and nights are then drizzled onto a wide-field mosaic image using EIS Drizzle. Astrometry residuals between the mosaic image and the APM catalogue show a random error of 0.3 without significant systematic effects.
Beckett, M. G., et al. 1996, SPIE, 2871, 1152
Bertin, E., & Arnouts, S. 1996, A&AS, 117, 393
Devillard, N. 2000, The Messenger, 100, 48
Mackay, C. D., et al. 2000, SPIE, 4008, 1317
McCarthy, P., et al. 2001, in ESO ``Deep Field'' Workshop, held in Garching, Germany, October 2000, astro-ph/0011499
Mink, D., 1999, in ASP Conf. Ser., Vol. 172, Astronomical Data Analysis Software and Systems VIII, ed. David M. Mehringer, Raymond L. Plante, & Douglas A. Roberts (San Francisco: ASP), 498
Sabbey, C. N., et al. 2001, in New Era of Wide Field Astronomy, held in Preston, England, August 2000, astro-ph/0012294