Next: Build Your Own SkyNode!
Up: Image Restoration
Previous: ISDC's DAL and DAL3 libraries
Table of Contents - Subject Index - Author Index - Search - PS reprint - PDF reprint

Micol, A., Pierfederici, F., & Benvenuti, P. 2003, in ASP Conf. Ser., Vol. 314 Astronomical Data Analysis Software and Systems XIII, eds. F. Ochsenbein, M. Allen, & D. Egret (San Francisco: ASP), 197

Review of the ASTROVIRTEL Experience at the end of its Three Approved Cycles

Alberto Micol, Francesco Pierfederici1, Piero Benvenuti2
Space Telescope European Coordinating Facility, ESA, Karl-Schwarzschild-Str.2, D-85748 Garching, Germany


ASTROVIRTEL3 has concluded its three-year life cycle. A review of the last two cycles is presented. The program selection process was instrumental in ensuring that the tools and methods developed for the successful PIs were general enough to be reused by a wider community of users of the ESO/ST-ECF archive. It will be here shown how such goal was achieved. The programs of the last two cycles will be described touching upon scientific and technical requirements, technical challenges (quite typical for any astronomical archive) technical and scientific achievements. The overall exercise of hosting scientific investigators with quite spread scientific interests has been very effective in revisiting and augmenting various ESO/ST-ECF archive functionalities and scientific products. Some of the developed tools and methods are already integral part of the ESO/ST-ECF archive, while some others are still being optimised before becoming operational. ASTROVIRTEL forced the developers to look into both the HST and ESO archives, each with its own peculiarities, and come up with solutions as general as possible. Furthermore, ASTROVIRTEL has also played an important role in the Virtual Observatory phase A study, particularly in the area of data centres inter-operability and scientific requirements.

1. The ASTROVIRTEL work flow

An introduction to ASTROVIRTEL, along with the description of its first cycle, can be found in Pierfederici et al. (2001). Here we describe the selection process and the support provided in the last two cycles, and present the programs that were allocated ASTROVIRTEL time.

1.1 Selection Process: the importance of the by-products

Call for proposals for cycles II and III were respectively issued in April 2001 and 2002, with a deadline fixed to mid June. In cycle II 12 proposals were received, 11 in cycle III. After issuing a report on the technical feasibility, the program selection panel, composed of 8 European astronomers,approved 5 proposals in cycle II and 4 in cycle III. The judgment was based not only on the scientific merit of the proposal, but it had to take into consideration the value that the technical development involved in the planned activities could bring to the ESO/ST-ECF archive and to its community of archive researchers. In other words, by-products of the scientific investigations had to be indicated and described in the proposals (e.g., new procedures for improved calibration, or, development of a pipeline to measure the seeing onto WFI images, or, development of more scientific-oriented archive search engine) and had a lot of weight in the selection process.

1.2 The support provided

The Principal Investigator and collaborators of the accepted programs were invited to ESO for a first visit, for the definition of the User Requirements, and for defining and scheduling the deliverables during the 1 year of granted support. Development and scheduling were revised on the way, via telecons and during a second visit. In the third and last visit, at the end of the year, each project was declared concluded, and a final report was issued. The small number of accepted proposals has to be compared with the limited amount of resources available ( 1 FTE for the actual development).

1.3 Cycle II & III Programs

Barbieri: Search for Trojans of Saturn, Neptune and Uranus
DeGrijs: Evolution and environmental dependence of star cluster luminosity functions
Erben: Gravitational lensing studies in randomly distributed, high galactic latitude fields
Ortiz: Variability detection in Wide Field Imager mosaic data
Prusti: Luminosity functions of young stellar clusters
Deharveng: Triggered massive-star formation
Golden: Long term optical variability of catalogued ultra-cool dwarfs
Kuijken: High proper motion stars in the solar neighbourhood
Pauli: 3D kinematics of white dwarfs

2. Excerpt of Challenges, Archive Improvements and Achievements

The scientific subjects of the various programs spanned from the formation and evolution of the solar system, to the determination of cosmological parameters, passing by star clusters evolution and stellar variability and kinematics.
Looking at the ESO/ST-ECF data holdings from so many different scientific points of view has certainly been an exciting and inspiring experience. Throughout the entire ASTROVIRTEL project various lessons were learned, some more obvious than others. The first (obvious) lesson is that the astronomers need a scientific description and characterisation of the archive contents. It is less obvious to know what kind of characterisation they need, which sometimes can be quite program specific. Among the others there is the need of removing the `data flow signature' from the data; that is, the data provider should describe in scientific terms what an observatory specific keyword and its value mean. A simple example is a filter name; a filter name should be expanded to a band pass description and throughput, keeping track that at different times the same filter name could actually mean a different filter throughput if it was changed without renaming it.

A more complex example is the name of a camera/observing mode which implicitly (via handbooks) describes the field of view, the sensitivity, the geometric distorsion, and many other parameters. In general, a search engine based on a data model describing telescope, instrument, camera, detector, readout electronics, etc., would allow any astronomer, even the one not unusually exposed to a given observatory (e.g., an X-ray astronomer looking for optical data), to easily identify instruments and data of interest.

In the following we highlight various capabilities that we have implemented in order to help the ASTROVIRTEL teams, and later the generic ESO/ST-ECF archive user.

A simple data model for ESO and HST instrument filters was implemented and it is now used by the Querator search engine; users can specify list of wavelengths or wavelength ranges, or even use standard scientific band passes names (e.g., B,V,R,I, Halpha, etc.) and the system will restrict such query to those instrument filters which better resemble the input band passes.

Querying an archive, especially a multi-instrument archive, using exposure times is not a precise way to identify data with desired depth, or signal to noise ratio. In cycle III (Voisin 2004) we have started implementing a Limiting Magnitude Calculator (LMC) in collaboration with the Exposure Time Calculator group of ESO (Ballester 2004). It will allow to associate to each image in the ESO/ST-ECF archive (starting with the Wide Field Imager) an estimate of the limiting magnitude given the overall sensitivity and parametrisation of the telescope/instrument/filter/detector and given the start time of the observation by which the sky brightness (Moon dependent) is evaluated. This will allow archive queries using the limiting magnitude at an assigned S/N ratio as an input parameter.

A pipeline that measures the seeing onto the archived Wide Field Imager images was developed; the pipeline stores the seeing measurements in a database that is used by the Querator search engine. At the same time the pipeline stores a preview image of the observation.

Another aspect where improvements were required regarded the archive search engine. Apart from adding the ability to treat different coordinate systems (ecliptic and galactic coordinates), the new requirements brought us to develop and potentiate a multi-colour search engine with the ability of uploading input files for a predefined user sample of objects, or with the ability to inter-operate with existing external facilities in order to build on-the-fly a sample of objects of given characteristics and to use such sample to query the ESO/ST-ECF archive. Querator was interfaced with both the (now expired) ADC catalogue collection and with the Lyon Extragalactic Database. A query of the type: all WFI and NICMOS images observed in at least two filters out of the U,B,V,R,I,H,J,K of any spiral galaxy closer than 30Mpc having a declination less than +20 degrees and a galactic latitude less than -20 degrees can be formulated within a single HTML form. Improvements (UCDs, VOTable, VOQL) will be needed in the way Querator and LEDA inter-operate.

The observing logs of the various observatories usually list only the individual available raw images, that is, they do not list the products that could be obtained by combining dithered or otherwise overlapping images. Hence, a search engine which makes use of the observing logs is not able to identify products satisfying given conditions (e.g., deeper than a given magnitude). To help better the user, it is necessary to describe as much as possible the possible products not the raw images. Hence, the individual raw images have to be associated in groups; the the metadata resulting from the association process is the main information that the user need to see. While for some HST instruments associations (and products) were available (Durand 2003), for various ASTROVIRTEL programs the Wide Field Imager associations were generated.

Designing and implementing a facility to allow a personalised digital view of selected objects of interest was another interesting area of development (PI Prusti). The idea is to keep track of the evolving knowledge (publications, observations, measurements) of objects of interest (the members of Chameleon I and II nebulae), through the examination of publications (manually done by the PI) or by looking at the history of changes within SIMBAD (via software updates), supporting PI criteria and history of PI decisions.

3. Conclusions

The scientific achievements of the ASTROVIRTEL cycles are summarised in the list of scientific papers published by the ASTROVIRTEL teams. The up-to-date list can be found on the ASTROVIRTEL web site.

ASTROVIRTEL, enabling improved access to existing data centres, helped the participating teams in reaching their scientific goals. Thanks to ASTROVIRTEL, the ESO/ST-ECF archive can now offer to its archive community enhanced services. Moreover, ASTROVIRTEL provided the first scientific requirements to the Astrophysical Virtual Observatory project.


Ballester, P. 2004, this volume, 481

Durand, D., et al. 2004, this volume, 209

Pierfederici, F., et al. 2001, in ASP Conf. Ser., Vol. 238, Astronomical Data Analysis Software and Systems X, ed. F. R. Harnden, Jr., Francis A. Primini, & Harry E. Payne (San Francisco: ASP), 141

Voisin, B., et al. 2004, this volume, 125


... Pierfederici1
now at NOAO, Tucson, Arizona
... Benvenuti2
now Istituto Nazionale di AstroFisica, Viale del Parco Mellini n.84, 00136 Roma, Italy
ASTROVIRTEL is a project supported by the European Commission under the 'Access to Research Infrastructures' action of the 'Improving Human Potential Programme', FP5 contract No. HPRI-CT-1999-00081

© Copyright 2004 Astronomical Society of the Pacific, 390 Ashton Avenue, San Francisco, California 94112, USA
Next: Build Your Own SkyNode!
Up: Image Restoration
Previous: ISDC's DAL and DAL3 libraries
Table of Contents - Subject Index - Author Index - Search - PS reprint - PDF reprint