Next: Adapting ISO and XMM-Newton Archives to Inter-Operability VO Standards
Up: Image Restoration
Previous: SAADA: An Automatic Archival System for Astronomical Data
Table of Contents -
Subject Index -
Author Index -
Search -
PS reprint -
PDF reprint
Voisin, B., Micol, A., ÓTuairisg, S., Butler, R., Golden, A., & Shearer, A. 2003, in ASP Conf. Ser., Vol. 314 Astronomical Data
Analysis Software and Systems XIII, eds. F. Ochsenbein, M. Allen, & D. Egret (San Francisco: ASP), 125
Using a Reversed Exposure Time Calculator for Querying Uncalibrated Archives
Bruno Voisin, Seathrún Ó Tuairisg, Ray Butler, Aaron Golden, Andy Shearer
Computational Astrophysics Laboratory, National University of Ireland, Galway, Ireland
Alberto Micol
European Space Agency, Space Telescope - European Coordinating Facility, Karl Schwarzschild Strasse 2
D-85748 Garching bei München, Germany
Abstract:
Mining large quantities of uncalibrated archives, for specific sources can prove
to be a hard task. Even an automated search engine able to use an archive
metadata (instrument, a filter, exposure time...) is not completely sufficient.
Indeed, without calibration it is difficult to know whether an interesting
source can be seen on images without actually looking. Here, we show how a
``reversed'' exposure time calculator can be used to efficiently process the
database-stored image descriptors of the ESO/Wide Field Imager (WFI) archive,
and compute the corresponding limiting magnitudes. The end result is a more
scientific description of the ESO/ST-ECF archive contents, allowing a more
astronomer-friendly archive user interface, and hence increasing the archive
useability in the context of a Virtual Observatory. This method is developed for
improving the Querator search engine of ESO/HST archive, in the context of the
EC funded ASTROVIRTEL project.
With the increasing number of large-scale surveys in various wavebands,
astronomers find themselves with more data than they can actually deal with.
While so much available data makes multi-waveband studies easier, the logistics
problems faced in this area are twofold: finding out which datasets amongst the
archives are of interest for a specific study, and then comparing heterogeneous
data coming from different sources. Though this paper's interest lies with the
former, references concerning a data mining approach of the latter can be found
in Voisin (2002) and Voisin & Donas (2001).
The ST-ECF/ESO archive, physically stored in Garching, Germany, is typical of
this data `mine' too huge to efficiently parse by usual queries. It contains
more than 10 terabytes of scientific data obtained with the ESA/NASA HST, with
the ESO NTT, VLT and with the Wide Field Imager on the ESO/MPI 2.2m Telescope,
and keeps growing at a rate of 4.5 terabytes per year. Thus, a simple
multi-waveband study of a source in a given sky area could easily require the
analysis of 50 to 100 images, from various telescopes using various filters. Of
course, depending on the type of source studied, some/most of those images might
prove of no interest as the depth/filter can prevent the source from being seen
on the final image. It might also happen that the sky region the scientist is
interested in proves to have been observed in an insufficient number of
wavebands for conductiong the study. This manual archive `mining' work proves to
be time consuming for the scientist, especially as a lot of data in the
ST-ECF/ESO archive is uncalibrated, and makes multi-waveband research
inefficient on a larger scale than a few sources.
The ASTROVIRTEL project
(Pierfederici et al., 2001; Micol & Pierfederici, 2004), supported by the
European Commission and managed by the ST-ECF on behalf of ESA and ESO, aims at
enhancing the scientific return of the ST-ECF/ESO Archive, by focusing on the
creation of tools and methods allowing one to efficiently access archives as
virtual telescopes. Among those tools, a search/query engine called Querator
(Pierfederici, 2001) has been developed with this multi-waveband idea in mind,
making it easier to parse through the whole ST-ECF/ESO archive and find
specified sky regions that have been observed in a minimum number of wavebands.
This is currently used for multiple data archive studies such as the one
presented in Ó'Tuairisg (2004).
Though one of the problems is solved, the other one (images of unsufficient
depth, poor filter for a specific source) is only slightly improved. Querator,
through the use of observation metadata (telescopes, generic filters, exposure
time...) allows the user to give constraints on the images he wants to get. But
these constraints proved to be difficult to use, hence the need for a simpler,
more generic approach, such as specifying the faintest magnitude of the objects
we want to see in the images. As the images are uncalibrated, limiting magnitude
information is not available. Our approach is to use intrument simulation (a
reversed exposure time calculator) for computing this value.
Querator is a search engine developped at the ESO in the context of the
ASTROVIRTEL project. Querator is a search engine addressing the need for
astronomers to get multicolour image data. Linked with the naming engine SIMBAD,
Querator can be given either a sky box, an object name, or a list of objects,
and is then able to process the entire ST-ECF archive for every corresponding
image matching a number of user-defined constraints. Main constraints available
for the user to fix are:
- instrument used for the observation, to select in a list of all ESO and HST instruments for which images are stored in the ST-ECF archive,
- minimum exposure time of observations to include,
- start time of the observation. Here a minimum/maximum date or a date range can be specified,
- wavelength of the observations. This can be a specific, a list or a range of wavelength,
- minimum filter number in which the area has to be observed for Querator to return images,
- filter type lets the user select all filter types or restrain search to broad/narrow bands.
Though those constraints make it possible to narrow an image search, it is still
difficult to use Querator to look for sources of a known spectral type and of a
known magnitude. Indeed, if one doesn't want to get lots of images not deep
enough to sort out, the only way of doing a pre-selection is to select a large
enough minimum exposure time. But this value alone is very difficult to
apprehend as it depends on other observation parameters.
To compute a limiting magnitude without reprocessing the whole ESO archive
images, we chose to use the ESO exposure time calculator. This instrument
simulation tool is generally used before observations for computing the minimum
exposure time required for getting a specific signal to noise ratio on sources
of a given spectral type/magnitude. The main formula used by the ETC for
computing the number of counts a source will give is:
where is the number of electrons per bin, is the incident flux,
is the filter band width , the total exposure time, the
efficiency, the telescope surface, is the solid angle subtended
by the integration element, and the energy of one photon. A signal to noise
ratio can then be computed as:
where is the contribute to the detected counts due to the Read Out Noise,
is the similar term due to the dark counts, is the number of
integration bins to evaluate the contribute to , and is
the number of integration pixels to evaluate the contribute to .
Considering that we know the exposure time , and that we fix a minimum signal
to noise ratio, we can rearrange the formula for computing a minimum flux
a source should produce as:
with and such as:
From this minimum flux, we can then compute a corresponding theoretical limiting
magnitude for the image. Sources fainter than this magnitude should
theoretically not be seen. Computing magnitudes by this method is far easier
than doing it by reprocessing the entire image, and it can be done for the
entire archive at a small computing cost. Once Querator has been tuned for
taking this information into account, it will be possible for uninteresting
images to be discarded directly by the search engine, instead of returning them
to the user. Of course, the multi-waveband aspect has to be taken care of, so
the reversed ETC (or LMC for Limiting Magnitude Calculator) is able to convert
magnitude between bands for various known spectral types. Finally, in case of
missing metadata, the LMC can simulate some values (sky brightness according to
moonphase, airmass tables for telescope locations...), at the price of a loss of
accuracy.
So far, the LMC is only processing images of a few instruments, due to the low
availability of metadata on some other archives. An observation metadata `data
model' is actually under construction, and should be designed in accordance to
the IVOA interoperability requirements. Another interesting thought is the
question whether the LMC should just be used for computing limiting magnitudes
that Querator would then use, or if the LMC should be entirely plugged in the
search engine, and compute limiting magnitude information `on-the-fly'. The
latter would definitely be the more interesting since it would allow the user to
fix herself the signal-to-noise ratio she'd like to have on certain sources, but
this might require too much computation, and delay further the query answering.
Acknowledgments
This work was carried out as part of the CosmoGrid project,
funded under the Programme for Research in Third Level Institutions (PRTLI)
administered by the Irish Higher Education Authority under the National
Development Plan and with partial support from the European Regional Development
Fund. The support given by ASTROVIRTEL, a Project funded by the European
Commission under FP5 Contract No. HPRI-CT-1999-00081, is acknowledged. Thanks
also goes to Pascal Ballester for his work on the ESO ETC (Ballester 2004),
and his help.
References
Ballester, P., 2004, this volume, 481
Micol, A., Pierfederici, F., 2004, this volume, 197
Pierfederici, F. 2001, Astronomical Data Analysis,
Proceedings of SPIE 4477
Ó'Tuairisg, S. et al., 2004, this volume, 444
Pierfederici, F. et al. 2001, in ASP Conf. Ser., Vol. 238, Astronomical Data Analysis Software and Systems
X, ed. F. R. Harnden,
Jr., Francis A. Primini, & Harry E. Payne (San Francisco: ASP), 141
Voisin, B., 2002, Ph.D, Université de Toulon et du Var
Voisin, B., Donas J., 2001, Astronomical Data Analysis, Proc. of SPIE 4477
© Copyright 2004 Astronomical Society of the Pacific, 390 Ashton Avenue, San Francisco, California 94112, USA
Next: Adapting ISO and XMM-Newton Archives to Inter-Operability VO Standards
Up: Image Restoration
Previous: SAADA: An Automatic Archival System for Astronomical Data
Table of Contents -
Subject Index -
Author Index -
Search -
PS reprint -
PDF reprint