Next: XML in the ADS
Up: Distributed Data Systems, Data Mining
Previous: SIMBAD as a Test Bed for two Object Oriented Database Management Systems: Objectivity/DB and O2
Table of Contents - Subject Index - Author Index - PS reprint -

Wirth, G. & Patton, D. R. 2000, in ASP Conf. Ser., Vol. 216, Astronomical Data Analysis Software and Systems IX, eds. N. Manset, C. Veillet, D. Crabtree (San Francisco: ASP), 251

VERDI: A Web Database System for Redshift Surveys

G. D. Wirth
W. M. Keck Observatory

D. R. Patton
University of Toronto


The Victoria Explorer for Redshift Databases on the Internet (VERDI) is a Web-based data retrieval system which allows users to access tabular data, images, and spectra of astronomical objects and to perform queries on the underlying database. We developed VERDI for use with the CNOC2 Field Galaxy Redshift Survey, but designed it to be generally applicable to deep galaxy redshift surveys. The software is freely available at, can easily be reconfigured and customized by the user, and performs well enough to support databases of many thousands of objects.

1. The Problem

As the CNOC2 Field Galaxy Redshift Survey neared completion in 1998, we had a problem. This deep pencil-beam redshift survey of galaxies in four ``patches'' distributed across the northern sky offered a sizable dataset including spectra for several thousand objects, UBVRI images for many more, plus measured properties such as positions, magnitudes, redshifts, and spectral indices (Yee et al. 2000). We were eager to do science involving several of these types of data; for example, using galaxy positions to identify galaxy groups and comparing the properties of these objects to those of the global sample. However, we needed a tool which would integrate the various data involved in the survey, including:

  1. Database files of object characteristics,
  2. Direct images of each field stored as FITS files, and
  3. Raw (2-D) and extracted (1-D) spectra stored as FITS files.
Since our co-investigators were widely distributed, we desired a web-based tool which would allow collaborators to access the data remotely. We had little time to invest in software development and no money to put towards an expensive database server. Still, we managed to assemble a working database system in just a few weeks of work by combining homemade scripts with readily available off-the-shelf freeware.

2. The Solution

The Victoria Explorer for Redshift Databases on the Internet (VERDI) is a general database program for redshift surveys which allows anyone with a Web browser to access galaxy images and spectra. VERDI runs under Unix and is otherwise entirely based on available free software. The input Web forms communicate with the database through CGI scripts written in Perl. These scripts access the database either directly or through self-generated Gawk scripts for running fast queries. VERDI uses IRAF directly from the Unix shell for image manipulation and Java to power the interactive spectral display. All data are stored in flat ASCII files or FITS images.

2.1. Notebook Mode

The heart of VERDI is Notebook Mode, so called because it displays a page summarizing the key information for a single object. One accesses Notebook Mode one of three ways: by entering the serial number of the desired object in the Notebook Mode entry page, by clicking the image of an object in Patch View Mode, or by clicking an object link in the output of Query Mode. Each notebook page features four main elements (see Figure 1):

Data table.
A data table summarizes pertinent characteristics of the object including serial number, position, magnitudes, and redshift. Each line in the table contains a hyperlink to an associated help document describing the contents of that line, explaining (for example) the units of measurement and how that property was determined.

Vicinity map.
A greyscale image displays the object and its immediate vicinity. This GIF image is generated on demand from the full FITS image of the corresponding field by invoking the IRAF task export directly from the Perl CGI script. The vicinity map also functions as a navigation tool: clicking the mouse on the image of a neighbor will bring up the notebook page for that object. Two other selections lie below the vicinity map: the Mark Objects button generates a version of the image on which targets with measured redshifts are marked in green and others are marked red; and the Field View item invokes Patch View Mode to give a larger-scale view of the object's environment.

Spectrum plot.
For objects with spectra, a long window below the data table and vicinity map is reserved for plotting the spectrum. If no redshift is measured the user has only the Plot Observed option, which will graph the spectrum using observed wavelengths. For objects with redshifts measured, a second Plot Rest button allows plotting flux versus a rest-frame wavelength scale. Pushing either button invokes a Java applet which fills the window with a line plot of the spectrum.

Download buttons.
A series of buttons at the bottom of the page give the user the option of retrieving various kinds of data for the object. These include: ASCII Data, which will download a text version of the data table from the top of the page; FITS Image, which will download a FITS version of the Vicinity Map image, 1-D FITS Spectrum to download the extracted spectrum, and 2-D FITS Spectrum to receive a FITS image containing the raw (un-extracted) spectrum.

Figure 1: Sample VERDI Notebook Mode page. Key elements of each notebook page are: (1) data table featuring hyperlinks into documentation for each property; (2) vicinity map displaying the region around the object, with optional marking of cataloged objects; (3) spectral plot allowing selection of rest-frame or observed wavelength scales; and (4) buttons enabling download of all available data as ASCII tables or as FITS files.

2.2. Patch View Mode

The CNOC2 survey regions include four patches on the sky, each of which consists of about twenty contiguous fields that correspond to the field of view of the CFHT MOS used to carry out the survey. After the user selects a patch and a field, Patch View Mode displays a clickable image for that field. By pointing the mouse to a target, the observer is informed of the redshift, magnitude, and serial number for the object. Clicking on a cataloged object then brings up the corresponding Notebook Mode page. Generation of the image maps used in Patch View Mode is done automatically through a script. Patch View Mode is particularly useful for quickly scanning images and retrieving relevant information for objects of interest.

2.3. Query Mode

Query Mode is a basic database querying system, allowing the user to retrieve data for a set of objects which match the criteria that he or she specified. Samples can be selected based on magnitude, redshift, position, serial number, or any other quantity in the database. Each field can be qualified with a lower bound, and upper bound, or a range, and the criteria on different fields can be combined through AND or OR logic.

The data for objects which match the selection criteria are displayed in tabular form, with hyperlinks that allow access to the corresponding Notebook Mode pages to provide more detailed information. Despite the lack of a true database server (queries are executed by spawning a Gawk script to parse the database), complex queries can be formed and searches can be completed quickly.

3. Final Words

VERDI is free, fast, and adaptable, and has enormously simplified the task of accessing the CNOC2 database. We wrote VERDI with the intention of sharing it, and are pleased that the DEEP survey team at UCSC has adapted it to meet their needs. Enquiries from other interested parties are welcomed.

Ralf Moros generously allowed us to bundle his XYMeter Java applet into VERDI for displaying spectra. Development of VERDI was supported by the Canadian Network for Observational Cosmology (CNOC) project through a grant from NSERC.


Yee, H. K. C., Morris, S. L., Lin, H., Carlberg, R. G., Hall, P. B., Sawicki, M., Patton, D. R., Wirth, G. D., Ellingson, E., & Shepherd, C. W. 2000, ApJS, in preparation

© Copyright 2000 Astronomical Society of the Pacific, 390 Ashton Avenue, San Francisco, California 94112, USA
Next: XML in the ADS
Up: Distributed Data Systems, Data Mining
Previous: SIMBAD as a Test Bed for two Object Oriented Database Management Systems: Objectivity/DB and O2
Table of Contents - Subject Index - Author Index - PS reprint -