Next: Interfacing with the Gemini Data Handling System
Up: Data Pipelines
Previous: Pipeline Processing of Data in the VLT Era
Table of Contents - Subject Index - Author Index - Search - PS reprint - PDF reprint

Hill, N., Gaudet, S., Dunn, J., Jaeger, S., & Cockayne, S. 1999, in ASP Conf. Ser., Vol. 172, Astronomical Data Analysis Software and Systems VIII, eds. D. M. Mehringer, R. L. Plante, & D. A. Roberts (San Francisco: ASP), 155

The Client Server Design of the Gemini Data Handling System

Norman Hill, Séverin Gaudet, Jennifer Dunn, Shannon Jaeger, Steve Cockayne
National Research Council Canada/Herzberg Institute of Astrophysics, 5071 West Saanich Rd. Victoria B.C. Canada V8X 4M6

Abstract:

The Gemini Telescopes Data Handling System (DHS) developed by the Canadian Astronomy Data Centre (CADC) has diverse requirements to support the operation of the Gemini telescopes. The DHS is implemented as a group of servers, where each performs separate functions. The servers use a client server model to communicate between themselves and with other Gemini software systems. This paper describes the client server model of the Gemini Data Handling System.

1. Introduction

The role of the DHS in the Gemini Telescopes systems is to a provide data handling infrastructure for the other Gemini systems and to isolate the other principal systems as much as possible from the details of how and where the data is stored. In addition the DHS provides an infrastructure for data quality assessment, which includes processing data to remove instrument signatures, and displaying data to telescope users. The DHS runs on Sun workstations which use the Solaris operating system.

The DHS servers communicate with each other, and with Gemini systems outside the DHS, using a client server model. The reasons for designing the DHS as separate servers are:

2. The DHS Library

The DHS library (Hill, Gaudet, & Kotturi 1998; Hill & Gaudet 1998) is a C library which was developed to provide an interface between the DHS and its client programs. The library sends messages between programs, which consist of commands and attributes (parameters) or bulk data (image data and descriptive attributes produced by Gemini instrument, Dunn et al. 1999). Both commands and bulk data are DRAMA SDS structures (Baily 1993) and are sent with the DRAMA IMP (Shortridge 1997) messaging system (see the DRAMA home page). Responses to commands or bulk data can be received asynchronously via a callback from the DHS library, or synchronously by explicitly waiting for a response.

The DHS library was ported to the VxWorks real-time operating system by the International Gemini Project Office to allow instruments to directly transfer data to the DHS. The Gemini Observatory Control System (OCS) development group incorporated the DHS library into OcsWish, the Tcl/Tk interpreter used by the OCS (Walker & Gillies 1998). The Tcl/Tk interface is used by the DHS consoles (Jaeger et al. 1999), and by the Quick Look System (Hill et al. 1999).

IMP was selected as the message passing mechanism for the interface to software outside of the DHS because it provides an efficient interface for sending both large and small data messages between systems, and because it is available for both the VxWorks and Sun/Solaris systems to be used by the Gemini project. IMP was also used for internal communications within the DHS because it fulfilled the communications needs of the DHS; it was very efficient for message passing between tasks on the same computer, and it avoided problems associated with having different interfaces for internal and external communications.

3. Client Server Communications

The DHS requires a very flexible communications architecture. A client may need to communicate with more than one server, a server may receive commands and data from more than one client, and servers may communicate with several other servers.

Figure 1: Server event processing message diagram.
\begin{figure}
\epsscale{1.00}
\plotone{hillnr2_1.eps}
\end{figure}

Figure 1 is a Booch event trace diagram that depicts the events which occur when two commands are received by a server. The two commands may have been initiated by the same client, or by two separate clients. Vertical lines on the diagram represent threads or processes in the interaction, the vertical bars overlaying the vertical lines indicate when the thread or process is active. Horizontal lines represent messages or function calls between the threads or processes.

Servers use the DHS library to specify callback functions which will executed when commands or bulk data requests are received. When a callback function is executed, the server is responsible for processing the command or bulk data request. The DHS library also provides functions to allow responses to be returned for commands or bulk data requests.


4. The DHS Servers

The following servers comprise the Gemini DHS:
The Command Server
provides a common command interface to all of the DHS servers, and eliminates the need for external clients of the DHS to be aware of which DHS servers are actually executing commands.
The Status Server
provides a status interface between the DHS servers and Gemini systems outside of the DHS. The Status Server allows status information to be distributed to the Gemini system, without each server being required to know how or to whom the status is being reported. The Status Server also monitors and reports DHS disk and database space usage.
The Data Server
accepts bulk data requests from other Gemini systems, supplies data labels for other Gemini systems, creates FITS files from the data, and informs other DHS Servers when new data is available.
The Quick Look Server
is responsible for distributing data to appropriate Quick Look Tools (Hill et al. 1999).
The Storage Server
creates storage media for the Gemini archive and for delivery to Gemini Telescope users.
The History Server
monitors history log records in the Gemini EPICS (Experimental Physics and Industrial Control System) databases and creates a permanent record of all logged messages.
The On-Line Data Processing Server
processes data collected by the Gemini Telescopes. The processed data is then archived and/or sent to the Quick Look Server for display.
The Synchronous Data Processing Server
performs data processing tasks for other Gemini systems. This server returns processed results to the Gemini system which initiated the data processing.

5. Conclusions

The modular design of the DHS has several advantages over a monolithic design:

The ability to run a subset of the DHS has already been exploited by the CADC to set up experimental systems to transfer data directly from the Canada France Hawai'i Telescope (CFHT) and the James Clark Maxwell Telescope (JCMT) on Mauna Kea to the archives in Victoria.

There were also disadvantages to the client server design.

However, overall the advantages of the client server design far outweighed the disadvantages.


References

Baily, J. 1993, ``Self-Defining Data System (SDS) Version 1.4'', Anglo-Australian Observatory, http://www.aao.gov.au/drama/doc/ps/sds.ps

Dunn, J., Jaeger, S., Hill, N., Gaudet, S., & Cockayne, S. 1999, this volume, 167

Hill, N. & Gaudet, S. 1998, ICD 3 - Bulk Data Transfer, (Gemini Data Handling System Rep. No. dhs_pdr_icd3/23, Gemini Project)

Hill, N., Gaudet, S., Dunn, J., Jaeger, S., & Cockayne, S. 1999, this volume, 163

Hill, N., Gaudet, S., & Kotturi, D. 1998, ICD 1c - Baseline DHS Interface (Gemini Data Handling System Rep. No. dhs_pdr_icd1c/05, Gemini Project)

Jaeger, S., Dunn, J., Cockayne, S., Gaudet, S., & Hill, N. 1999, this volume, 159

Shortridge, K. 1997, Interprocess Message Passing (IMP) System,
(AAO/IMP_MANUAL_8, Drama Software Report 8) (Sydney: Anglo-Australian Observatory)

Walker, S. & Gillies, K. 1998, OcsWish Technical Manual,
(ocs.ocs.015-OcsWishTechnical/02, Gemini Project) (Tucson: Gemini
Observatory)


© Copyright 1999 Astronomical Society of the Pacific, 390 Ashton Avenue, San Francisco, California 94112, USA
Next: Interfacing with the Gemini Data Handling System
Up: Data Pipelines
Previous: Pipeline Processing of Data in the VLT Era
Table of Contents - Subject Index - Author Index - Search - PS reprint - PDF reprint

adass@ncsa.uiuc.edu