Next: Adapting ISO and XMM-Newton Archives to Inter-Operability VO Standards
Up: Image Restoration
Previous: SAADA: An Automatic Archival System for Astronomical Data
Table of Contents - Subject Index - Author Index - Search - PS reprint - PDF reprint

Voisin, B., Micol, A., ÓTuairisg, S., Butler, R., Golden, A., & Shearer, A. 2003, in ASP Conf. Ser., Vol. 314 Astronomical Data Analysis Software and Systems XIII, eds. F. Ochsenbein, M. Allen, & D. Egret (San Francisco: ASP), 125

Using a Reversed Exposure Time Calculator for Querying Uncalibrated Archives

Bruno Voisin, Seathrún Ó Tuairisg, Ray Butler, Aaron Golden, Andy Shearer
Computational Astrophysics Laboratory, National University of Ireland, Galway, Ireland

Alberto Micol
European Space Agency, Space Telescope - European Coordinating Facility, Karl Schwarzschild Strasse 2 D-85748 Garching bei München, Germany

Abstract:

Mining large quantities of uncalibrated archives, for specific sources can prove to be a hard task. Even an automated search engine able to use an archive metadata (instrument, a filter, exposure time...) is not completely sufficient. Indeed, without calibration it is difficult to know whether an interesting source can be seen on images without actually looking. Here, we show how a ``reversed'' exposure time calculator can be used to efficiently process the database-stored image descriptors of the ESO/Wide Field Imager (WFI) archive, and compute the corresponding limiting magnitudes. The end result is a more scientific description of the ESO/ST-ECF archive contents, allowing a more astronomer-friendly archive user interface, and hence increasing the archive useability in the context of a Virtual Observatory. This method is developed for improving the Querator search engine of ESO/HST archive, in the context of the EC funded ASTROVIRTEL project.

1. Introduction

With the increasing number of large-scale surveys in various wavebands, astronomers find themselves with more data than they can actually deal with. While so much available data makes multi-waveband studies easier, the logistics problems faced in this area are twofold: finding out which datasets amongst the archives are of interest for a specific study, and then comparing heterogeneous data coming from different sources. Though this paper's interest lies with the former, references concerning a data mining approach of the latter can be found in Voisin (2002) and Voisin & Donas (2001).

The ST-ECF/ESO archive, physically stored in Garching, Germany, is typical of this data `mine' too huge to efficiently parse by usual queries. It contains more than 10 terabytes of scientific data obtained with the ESA/NASA HST, with the ESO NTT, VLT and with the Wide Field Imager on the ESO/MPI 2.2m Telescope, and keeps growing at a rate of 4.5 terabytes per year. Thus, a simple multi-waveband study of a source in a given sky area could easily require the analysis of 50 to 100 images, from various telescopes using various filters. Of course, depending on the type of source studied, some/most of those images might prove of no interest as the depth/filter can prevent the source from being seen on the final image. It might also happen that the sky region the scientist is interested in proves to have been observed in an insufficient number of wavebands for conductiong the study. This manual archive `mining' work proves to be time consuming for the scientist, especially as a lot of data in the ST-ECF/ESO archive is uncalibrated, and makes multi-waveband research inefficient on a larger scale than a few sources.

The ASTROVIRTEL project (Pierfederici et al., 2001; Micol & Pierfederici, 2004), supported by the European Commission and managed by the ST-ECF on behalf of ESA and ESO, aims at enhancing the scientific return of the ST-ECF/ESO Archive, by focusing on the creation of tools and methods allowing one to efficiently access archives as virtual telescopes. Among those tools, a search/query engine called Querator (Pierfederici, 2001) has been developed with this multi-waveband idea in mind, making it easier to parse through the whole ST-ECF/ESO archive and find specified sky regions that have been observed in a minimum number of wavebands. This is currently used for multiple data archive studies such as the one presented in Ó'Tuairisg (2004).

Though one of the problems is solved, the other one (images of unsufficient depth, poor filter for a specific source) is only slightly improved. Querator, through the use of observation metadata (telescopes, generic filters, exposure time...) allows the user to give constraints on the images he wants to get. But these constraints proved to be difficult to use, hence the need for a simpler, more generic approach, such as specifying the faintest magnitude of the objects we want to see in the images. As the images are uncalibrated, limiting magnitude information is not available. Our approach is to use intrument simulation (a reversed exposure time calculator) for computing this value.

2. Mining Archives with Querator

Querator is a search engine developped at the ESO in the context of the ASTROVIRTEL project. Querator is a search engine addressing the need for astronomers to get multicolour image data. Linked with the naming engine SIMBAD, Querator can be given either a sky box, an object name, or a list of objects, and is then able to process the entire ST-ECF archive for every corresponding image matching a number of user-defined constraints. Main constraints available for the user to fix are:

instrument used for the observation, to select in a list of all ESO and HST instruments for which images are stored in the ST-ECF archive,
minimum exposure time of observations to include,
start time of the observation. Here a minimum/maximum date or a date range can be specified,
wavelength of the observations. This can be a specific, a list or a range of wavelength,
minimum filter number in which the area has to be observed for Querator to return images,
filter type lets the user select all filter types or restrain search to broad/narrow bands.

Though those constraints make it possible to narrow an image search, it is still difficult to use Querator to look for sources of a known spectral type and of a known magnitude. Indeed, if one doesn't want to get lots of images not deep enough to sort out, the only way of doing a pre-selection is to select a large enough minimum exposure time. But this value alone is very difficult to apprehend as it depends on other observation parameters.

3. From Exposure Time to Photometric Magnitude

To compute a limiting magnitude without reprocessing the whole ESO archive images, we chose to use the ESO exposure time calculator. This instrument simulation tool is generally used before observations for computing the minimum exposure time required for getting a specific signal to noise ratio on sources of a given spectral type/magnitude. The main formula used by the ETC for computing the number

of counts a source will give is:

$\begin{displaymath}N=\frac{F.\Delta_{i}.T.E.S.\Omega_{i}}{P}\end{displaymath}$

where is the number of electrons per bin, is the incident flux, $\Delta_{i}$ is the filter band width , the total exposure time, the efficiency, the telescope surface, $\Omega_{i}$ is the solid angle subtended by the integration element, and the energy of one photon. A signal to noise ratio can then be computed as:

$\begin{displaymath}\frac{S}{N}=\frac{N_{Obj}}{\sqrt{N_{Obj}+n_{bin}.N_{Sky}+n_{bin}.RON^{2}+n_{pi x}.DARK.T}}\end{displaymath}$

where is the contribute to the detected counts due to the Read Out Noise, is the similar term due to the dark counts, $n_{bin}$ is the number of integration bins to evaluate the contribute to , and $n_{pix}$ is the number of integration pixels to evaluate the contribute to .

Considering that we know the exposure time , and that we fix a minimum signal to noise ratio, we can rearrange the formula for computing a minimum flux a source should produce as:

$\begin{displaymath}F_{Obj}=\frac{\frac{S}{N}}{2.a.T}.\Biggl[\frac{S}{N}+\sqrt{{\Bigl(\frac{S}{N}\Bigr)}^{2}+4.c}\Biggr]\end{displaymath}$

with and such as:

$\begin{displaymath}a=\frac{\Delta_{i}.E.S}{P}\end{displaymath}$

$\begin{displaymath}c=n_{bin}.a.F_{Sky}.T.\Omega_{i}+n_{bin}.{RON}^2+n_{pix}.{DARK}.T\end{displaymath}$

From this minimum flux, we can then compute a corresponding theoretical limiting magnitude for the image. Sources fainter than this magnitude should theoretically not be seen. Computing magnitudes by this method is far easier than doing it by reprocessing the entire image, and it can be done for the entire archive at a small computing cost. Once Querator has been tuned for taking this information into account, it will be possible for uninteresting images to be discarded directly by the search engine, instead of returning them to the user. Of course, the multi-waveband aspect has to be taken care of, so the reversed ETC (or LMC for Limiting Magnitude Calculator) is able to convert magnitude between bands for various known spectral types. Finally, in case of missing metadata, the LMC can simulate some values (sky brightness according to moonphase, airmass tables for telescope locations...), at the price of a loss of accuracy.

4. Conclusion

So far, the LMC is only processing images of a few instruments, due to the low availability of metadata on some other archives. An observation metadata `data model' is actually under construction, and should be designed in accordance to the IVOA interoperability requirements. Another interesting thought is the question whether the LMC should just be used for computing limiting magnitudes that Querator would then use, or if the LMC should be entirely plugged in the search engine, and compute limiting magnitude information `on-the-fly'. The latter would definitely be the more interesting since it would allow the user to fix herself the signal-to-noise ratio she'd like to have on certain sources, but this might require too much computation, and delay further the query answering.

Acknowledgments

This work was carried out as part of the CosmoGrid project, funded under the Programme for Research in Third Level Institutions (PRTLI) administered by the Irish Higher Education Authority under the National Development Plan and with partial support from the European Regional Development Fund. The support given by ASTROVIRTEL, a Project funded by the European Commission under FP5 Contract No. HPRI-CT-1999-00081, is acknowledged. Thanks also goes to Pascal Ballester for his work on the ESO ETC (Ballester 2004), and his help.

References

Ballester, P., 2004, this volume, 481

Micol, A., Pierfederici, F., 2004, this volume, 197

Pierfederici, F. 2001, Astronomical Data Analysis, Proceedings of SPIE 4477

Ó'Tuairisg, S. et al., 2004, this volume, 444

Pierfederici, F. et al. 2001, in ASP Conf. Ser., Vol. 238, Astronomical Data Analysis Software and Systems X, ed. F. R. Harnden, Jr., Francis A. Primini, & Harry E. Payne (San Francisco: ASP), 141

Voisin, B., 2002, Ph.D, Université de Toulon et du Var

Voisin, B., Donas J., 2001, Astronomical Data Analysis, Proc. of SPIE 4477