J. W. MacKenty
Space Telescope Science Institute, 3700 San Martin Drive,
Baltimore MD 21218
We have developed a software package to facilitate the reduction of data from the NOAO SQIID (Simultaneous Quad-color Infrared Imaging Device) camera (Ellis et al. 1992). Our approach emphasizes the design of a software package targeted towards a specific instrument and a limited set of observational projects. Rather than build software for the general problem of reducing infrared images from a large range of instruments, this package is intended only for use with the SQIID camera. Therefore, it incorporates, and to some extent conceals from the user, considerable knowledge about the camera. Further, while the SQIID camera can be used to map large regions of the sky by constructing mosaics, this package is appropriate only for use with observations where the telescope is moved a limited fraction of the field of view between exposures and the entire set of exposures of a target possesses some degree of overlap.
The SQIID camera employs four pixel PtSi array detectors which, although their quantum efficiency is rather modest (6.6% at J to 3.4% at K), have excellent cosmetic qualities, good linearity, outstanding temporal stability, and little sensitivity to the effects of previous exposures. SQIID incorporates dichroic beam splitters and re-imaging optics to simultaneously view the same field in the J (1.2micron), H (1.6micron), K (2.2micron), and L (3.5micron) passbands. Due to its very low sensitivity, the L-band channel is not incorporated in the MACSQIID software package. SQIID has a scale of 1.'' 36 pixel and a field of view of 5.' 5 on the KPNO 1.3m telescope, and 0.'' 43 pixel and 1.' 8 on the KPNO 4m telescope.
The goal of the MACSQIID package is to produce images from a set of observations which are free from the various instrumental and atmospheric effects. An observing run consists of many (typically 5 to 20) exposures (60 to 300s each) of a series of targets. We will refer to each set of exposures as a ``dataset''. Within each dataset the telescope was offset a distance less than the SQIID's field of view and larger than the extent of the target(s). For larger targets, a few exposures of equal duration are obtained of a nearby, blank area on the sky.
In each SQIID exposure, we detect the flux distribution of the target field of view plus the sky and instrumental backgrounds. The detected image, , is related to the original scene, , by:
The individual pixels in each image have unique sensitivities (``flat field'') and signal internal to the instrument (``dark current''). The dark current is directly measured by obtaining many exposures with a cold dark slide inserted behind the instrument's input aperture. The MACSQIID package makes the following assumptions. First, that the dark current and flat field are constant over a significant number of datasets (in practice they are stable over a week long observing run). Second, that the sky background can be modeled separately as a uniform level which varies rapidly with time (``background'') and as an illumination pattern which is constant within a given dataset (``sky''). It is assumed that the ``sky'' pattern is mainly the result of large scale illumination effects (e.g., moonlight) and of the telescope structure, therefore it is taken to be constant with the small motions of the telescope which occur within a dataset.
The basic approach to the solution of the calibration problem is successive iterations which improve the quality of the flat, sky, and background calibrations. This process is implemented in two loops. The outer loop operates on each individual dataset. It calibrates (via the Basic Calibration Equation) the image from each exposure, then shifts and averages the exposures for each target. Pixels flagged as ``bad'' in the dark or flat field are excluded from this average. The averaged images from three passbands are then mapped to a common coordinate system (usually the K band's due to its lower S/N) and summed. This summed image is then scanned to locate ``sources'' based on a photon statistics model of the expected sky noise. Finally, all pixels in the original individual images which correspond to sources in the summed image are flagged.
The inner loop simultaneously utilizes all observations from one night (or even an entire observing run). Working on one passband at a time, the background for each exposure, the average flat field, and the sky for each dataset (i.e., target) is determined. To avoid the biases inherent in median filtering, a combination of the exclusion of pixels flagged as containing sources and iterative sigma clipping is used for each part of this process. The sky is determined from the residual of the spatially smoothed average of the exposures of each target. This gives the sky a S/N comparable to that achieved in the flat field (typically a few hundred samples per pixel) and is justified by the recognition that the sky illumination pattern is out of focus (i.e., does not contain significant high spatial frequency structure). The inner loop is executed a user determined number of times (typically 3 to 5) in order to achieve convergence for most of the pixels. Pixels with poor solutions may be excluded from further use in the outer loop.
The outer loop is repeated until the image quality is satisfactory. This usually requires 3 or 4 iterations. The first pass of the outer loop is provided with a simple ``bootstrap'' flat field solution constructed from a median of the images in each dataset. The user must select which exposures are to be included in the inner loop (e.g., those without bright sources and sky exposures) and which to use to determine the sky illumination pattern for each dataset.
We start with many J, H, and K images in IRAF files. IRAF tasks are used to make appropriate dark calibration frames. Also, the relative offsets, rotations, and scale changes between the J, H, and K passbands are determined with the IRAF geometry tasks. The MACSQIID package then provides four tasks:
BUILD creates a ``dsf'' file containing all images, calibration files, and miscellaneous data for a dataset. It is driven by a simple ASCII script file. The use of a script file for control rather than an interactive interface permits painless re-execution should it become necessary to start over. Non-linearities in the detectors at high signal levels are corrected for at this stage.
PROCESS provides an interactive task to calibrate and examine images. This implements the ``outer loop'' portion of the calibration process defined in § 2.2. The user is presented with a simple menu of one key commands. These support the basic calibration (including an initial bootstrap calibration using a median based flat field generated on the fly), geometric mapping between J, H, and K channels, the alignment between exposures (star marking and cross correlation of source pixels), and source detection and flagging. Considerable statistical information is displayed during the execution of the various calibration steps. The user has the option to exclude individual exposures from the combined image. This task also provides for the interactive examination of calibration data and raw, calibrated, and flagged images using a named pipe connection to SAOIMAGE.
CALIB updates the background, flat field, and sky calibrations in a large number of dataset simultaneously. This implements the ``inner loop'' of the calibration process defined in § 2.2. This batch program is driven by an ASCII script file. It provides copious reporting and statistics to assess the quality of its solutions.
EXTRACT produces IRAF format images from the internal format ``dsf'' file. This is generally used at the completion of the calibration process to allow further display and analysis within the IRAF system.
The MACSQIID package uses several programming methods which reflect the intention to produce a specific rather than a general purpose software package. The package embeds considerable knowledge about the SQIID instrument. This is mostly contained in ``include'' files as defined parameters. These serve to hide details of SQIID from the user (e.g., the noise model ``knows'' about the system gain factor). The code uses a large data structure (implemented within FORTRAN common blocks) which places all of the images in system memory. This supports fast interactive performance to encourage the user to explore the data and experiment with alternative calibrations. This data structure is common to each task and directly maps onto the ``dsf'' file format.
Crucial to the goal of keeping the coding effort limited in scope was the availability and use of existing interfaces to the external environments. The IRAF ``imfort'' interface was used to access IRAF format images at the start and end phases of the calibration process. The ``sao-iis'' package by Jim Wright (formerly at CFHT) provides simple and efficient means of displaying image data and interacting with the SAOIMAGE display tool.
This package provides a means of achieving near-optimal (i.e., background or photon noise limited) calibration of data from the SQIID camera. The use of object detection and iterative sigma clipping results in good flat fields at all spatial scales. The separation of the sky background into a time variable background component and a target (dataset) variable illumination pattern permits the recovery of the calibration accuracy inherent in the flat fields. This depends, in part, on the recognition that the sky illumination pattern is intrinsically out of focus in the detector plane.
This package was designed to encourage the user to interactively examine datasets and to understand the quality of the calibration(s) achieved while placing the computationally intensive steps into batch programs. It simplifies the data management task by grouping all of the observations and calibration files for a dataset into a single large file of a custom design yet provides IRAF format files for further analysis subsequent to the calibration process. The code development effort was kept to reasonable levels with the use of existing interface packages (``imfort'' and ``sao-iis'').
The assistance of Mike Merrill and Ian Gatley at the NOAO in understanding the SQIID camera and its data is gratefully acknowledged. Gina Jones' skill and patience in the testing of this package is also appreciated. Copies of this package are available from the author.