Next: User Defined Functions for the ISO Post Mission Archive
Up: Archiving and Information Services
Previous: Numerical Observatory: a Web Site for the Retrieval, Analysis, and Visualization of Simulated Cluster Data
Table of Contents - Subject Index - Author Index - Search - PS reprint - PDF reprint

Barg, I., Ferro, A. J., & Stobie, E. B. 1999, in ASP Conf. Ser., Vol. 172, Astronomical Data Analysis Software and Systems VIII, eds. D. M. Mehringer, R. L. Plante, & D. A. Roberts (San Francisco: ASP), 245

Spiders and Camels and Sybase! Oh, My!

Irene Barg, Anthony J. Ferro, Elizabeth Stobie
University of Arizona, Steward Observatory, NICMOS, Tucson, AZ 85721


The Hubble Space Telescope NICMOS Guaranteed Time Observers (GTOs) requested a means of sharing point spread function (PSF) observations. Because of the specifics of the instrument, these PSFs are very useful in the analysis of observations and can vary with the conditions on the telescope. The GTOs are geographically diverse, so a centralized processing solution would not work. The individual PSF observations were reduced by different people, at different institutions, using different reduction software. These varied observations had to be combined into a single database and linked to other information as well.

The NICMOS software group at the University of Arizona developed a solution based on a World Wide Web (WWW) interface, using Perl/CGI forms to query the submitter about the PSF data to be entered. After some semi­automated sanity checks, using the FTOOLS package, the metadata are then entered into a Sybase relational database system. A user of the system can then query the database, again through a WWW interface, to locate and retrieve PSFs which may match their observations, as well as determine other information regarding the telescope conditions at the time of the observations (e.g., the breathing parameter).

This presentation discusses some of the driving forces in the design, problems encountered, and the choices made. The tools used, including Sybase, Perl, FTOOLS, and WWW elements are also discussed.

1. Introduction

During a meeting involving many of the NICMOS GTOs, considerable interest was expressed in establishing a database containing the PSF star observations obtained as part of GTO programs. A centralized database was seen as a good way of making the existing PSF observations available to everyone on the team. This paper discusses how this PSF database and the user interface evolved.

2. Design Issues

Submissions to the database of processed PSF images are the responsibility of the individual team members. These team members are geographically dispersed, may have used different data reduction software, and may have named their images differently.

A network interface using CGI forms addressed all these issues. Submissions could be made remotely. Users would only need a Web browser to access the submission form. To enforce commonality among submissions, the submission form was structured in a way to ask specific questions regarding reduction procedures and details about the PSF star. The submission form needed to be straight forward and easy to use, it had to impose standards (file naming conventions, minimum descriptive information), and it had to be as short as possible without omitting key information about the PSF star. Our overall design goal was to automate the maintenance of the database as much as possible. Figure 1 illustrates the submission and retrieval process in which a PSF star observation travels from the submitter, is checked and ingested at University of Arizona (UofA), and is later retrieved and sent from the UofA to the requester.

Figure 1: PSF Automated Archive Process

3. Submission Process

3.1 PSF Submittal Form

The PSF Star Submittal form is fairly detailed and requires some effort on the part of the submitter. They are asked to enter information about the PSF star and provide details of their data reductions. Some of the data entered are used as search parameters on the PSF Data Retrieval form, so it is important that the submission form be completed as completely as possible. A small text window in which to enter comments is included. Comments can be used to add any information that is important, but was not included in the form. Information provided on this form is transferred to the FITS headers of the submitted data file as HISTORY keywords. File naming standards were adopted and other standards were enforced through the form itself by requiring entries on selected fields.

3.2 PSF Submittal Checking and Database Population

Upon completion of the submission form, the data are automatically FTPed from the submitter's site to the database server at the UofA. An automated message is sent via e-mail to the user indicating the status of the transfer (received, failed, etc.). A copy of this notification goes to the NICMOS Database Administrator who then runs a set of Perl scripts that perform the following functions.

4. Retrieval Process

4.1 PSF Data Retrieval Form

This Web interface allows users to search the PSF archive for data that match criteria such as camera, filter, spectral type of the PSF star, breathing ephemeris (relative focus), and x- and y-position on the array. A successful search will present the user with a table of PSF star images from which they may select the PSF's they wish to retrieve. Figure 2 shows the results of such a search.

Figure 2: Results from a PSF Retrieval search.

With the exception of the breathing parameter, all of the data displayed from the results of the search represent data from the PSF database. The breathing parameter is a measurement of the change in the focus due to the temperature changes (breathing) of the telescope. The amount of this change varies and is highly time dependent. However, this value is not known at the time of the observation and therefore is not found in the FITS headers. There are two steps for determining relative focus (breathing) at a given time:

  1. Look up the nominal value for the observation date, either in MJD, day of year, or days since 1 Jan 1994.
  2. Look up the modeled variation from this nominal value for the exact time of the observation and add this value to the nominal.

The resulting value is in secondary mirror microns and is relative to best focus at WFPC2 PC (zero microns). Calculating this value presents a problem because the nominal value and modeled variation are embedded in two separate Space Telescope Science Institute (STScI) HTML documents that are updated occasionally as the model is improved. A means to extract the necessary information from the most recent versions of these documents became a programming challenge that was met by using a Perl module called libwww-perl (LWP). LWP allows us to locate the required two documents by extracting the links from the parent HTML page, fetch the required documents, and parse the necessary information on the fly. A description of how LWP accomplishes this would make a paper of its own.

5. System Software

This entire project was written using Perl V5.004 utilizing the following modules:, Sybperl, libnet, and libwww-perl (LWP). The verification scripts, also written in Perl, utilized a set of FITS utility programs from the FTOOLS package created by the High Energy Astrophysics Science Archive Research Center (HEASARC). The SQL server is Sybase1 SQL Server V11.02.

6. Conclusions

The goal to automate the entire PSF submittal process drove our decision to use Perl as our programming language. As shown in Fig. 1, the submittal process involved communicating with the WWW, transferring data across a network using FTP, verifying the data using FTOOLS, and then adding the new PSF entries to our our SQL Server. The Perl programming language gave us the tools to provide the glue between all of these separate systems to make automation possible. To view some of the PSF forms, go to the (NICMOS) Web page and follow the links to the Database Page.


... Sybase1
SYBASE is a registered trademark of Sybase, Inc.

© Copyright 1999 Astronomical Society of the Pacific, 390 Ashton Avenue, San Francisco, California 94112, USA
Next: User Defined Functions for the ISO Post Mission Archive
Up: Archiving and Information Services
Previous: Numerical Observatory: a Web Site for the Retrieval, Analysis, and Visualization of Simulated Cluster Data
Table of Contents - Subject Index - Author Index - Search - PS reprint - PDF reprint