Next: NICMOS Software: An Observation Case Study
Up: Dataflow and Scheduling
Previous: OPUS-97: A Generalized Operational Pipeline System
Table of Contents -- Index -- PS reprint -- PDF reprint

Astronomical Data Analysis Software and Systems VII
ASP Conference Series, Vol. 145, 1998
Editors: R. Albrecht, R. N. Hook and H. A. Bushouse

Remote Observing with the Keck Telescope:
ATM Networks and Satellite Systems

P.L.Shopbell, J.G.Cohen, and L. Bergman1

California Institute of Technology, Pasadena, CA. 91125

1Jet Propulsion Laboratory, California Institute of Technology




As a technical demonstration project for the NASA Advanced Communications Technology Satellite (ACTS), we have implemented remote observing on the 10-meter Keck II telescope on Mauna Kea from the Caltech campus in Pasadena. The data connection consists of ATM networks in Hawaii and California, running at OC-1 speeds (45 Mbit/sec), and high data rate (HDR) satellite antennae at JPL in Pasadena and Tripler Army Medical Center in Honolulu. The ACTS network is being used to enable true remote observing, as well as remote eavesdropping. The software environment is identical to that used for on-site observing at the Keck telescope, with the added benefit of the software, personnel, and other facilities provided by observing in a local environment. In this paper, we describe our high-speed remote observing network, assess the network's level of performance, and summarize the benefits and difficulties encountered in this project.

1. Introduction

Remote use of astronomical telescopes has been a topic of interest for many years, even before space-based observing platforms (e.g., IUE) began to demonstrate total remote operation out of sheer necessity. However, only very recently are optical telescopes beginning to realize the benefits of true remote observing: for example, observations with modest size detectors at Apache Point Observatory are being carried out remotely using the Internet (York 1995, BAAS, 186, 44.04 ). For this project, we have established remote interactive observing capabilities for Keck Observatory on Mauna Kea for observers at Caltech, in Pasadena, California. The recently commissioned twin 10-meter Keck Telescopes are the largest optical/infrared telescopes in the world and thereby typify the data and network requirements of a modern observatory. In undertaking this project, we were motivated by several operational and scientific advantages that remote observing would offer, including alleviating altitude-related difficulties, saving time and money due to travel, enabling remote diagnosis of hardware and software problems, and simplifying access to the telescopes for educational use.
Figure 1: Schematics of the final terrestrial networks in Hawaii and California, as used for the Keck remote observing project. These networks were connected via NASA's ACTS satellite.  

2. Network Architecture

Our network consists of three major segments: the ground network in California, the satellite link across the Pacific Ocean, and the ground network in Hawaii.

The ground network in California connects Caltech with JPL, the site of the ACTS ground station. This segment was established as part of Pacific Bell's extant CalREN fiber optic network and has proved to be the most reliable portion of our network.

The satellite connection was made available to us through a grant from NASA as part of the Gigabit Satellite Network (GSN) testbed program. NASA's Advanced Communications Technology Satellite (ACTS) was built to explore new modes of high speed data transmission at rates up to OC-12 (622 Mbit/sec). The 20-30 GHz frequency band has been employed for the first time by a communications satellite, with extensive rain fade compensation.

The ground network in Hawaii, which connects Keck observatory with the other ACTS ground station at Tripler Army Medical Center in Honolulu has been somewhat more complex in its evolution. This was primarily due to the relative inexperience of GTE Hawaiian Telephone and a lack of prior infrastructure in Hawaii. This segment initially consisted of a combination of underwater fiber, microwave antennae, and buried fiber. The higher bit error rates (BER) of the non-fiber segment produced noticeable instability in the end-to-end network. Fortunately, in January of 1997 this portion of the ground network in Hawaii was upgraded to optical fiber. The improved performance for high-speed data transfers of the final all-fiber network was immediately apparent.

In order to support standard higher-level (IP) networking protocols, we installed an Asynchronous Transfer Mode (ATM) network over this infrastructure. The transfer of 53-byte ATM cells is performed by hardware switches throughout the network, at speeds of OC-1 (51 Mbit/sec) and above. Several vendors have supplied the ATM switches and Network Interface Cards (NICs), providing a stringent test of compatibility in the relatively new ATM environment. Although we have encountered several interoperability problems, none have been serious, and the ATM and telephone vendors have been extremely helpful.

In order to facilitate reliable data transfer, as well as to allow the use of the wealth of software tools already available, we are running the standard IP protocols over ATM using a pseudo-standard implementation known as ``Classical IP''. This enables the use of the standard network-based applications that are in widespread use on the Internet. Tools such as ftp and telnet are part of every observing run, as are additional special-purpose applications, such as an audio conferencing tool (rat) and a shared whiteboard tool (wb).

Figure 2: Bandwidth test results between Keck Observatory and the Caltech campus in Pasadena, California, over the ACTS satellite network. TCP exhibits a remarkable dependence on the bit error rate.  

3. Network Performance

The most important impact of a satellite component on a high-speed network is the relatively large delay introduced by the round-trip signal travel time to the satellite. In our network, this travel time is approximately 0.55 seconds, which corresponds to 3.5 Mbytes of data at OC-1 speeds (51 Mbit/sec). The problem has to do with the connection-oriented nature of TCP/IP: TCP sends a very specific amount of data, known as a ``window'', after which time it expects an acknowledgment from the other end of the connection. However, this window size is often very small; the default value for workstations running the SunOS 4.1.4 operating system is only 4 Kbytes.

Fortunately, this problem is well-known in the high-speed networking community. Networks such as ours are known as ``long fat networks'' (LFN; see RFC 1323 ). In the case of the SunOS operating system (to which we are constrained by legacy control software at Keck), we obtained the TCP-LFN package from Sun Consulting, which purports to support the RFC 1323 extensions. Unfortunately, a number of limitations of SunOS 4.1.4 conspire to prohibit one from obtaining extremely large window sizes, regardless of the TCP-LFN software. In our case, the compiled-in kernel limit of 2 Mbytes of Mbuf memory (i.e., IP packet wrappers) turned out to be the major constraint, limiting our window size to no more than 1 Mbyte. Indeed, our final tuned network delivered the expected maximum TCP/IP performance of approximately 15 Mbit/sec ($\sim
{1\over3}$ of OC-1). Although perhaps disappointing in a relative sense, this bandwidth is far in excess of T1 Ethernet speed (1.44 Mbit/sec) and allows an 8 Mbyte image to be transferred in approximately 5 seconds. As a further comparison, this bandwidth exceeds by 50% that which is available on the local area Ethernet network at the Keck Telescope itself. Figure 2 illustrates typical bandwidth measurements of our network for UDP and TCP, the latter before and after the network was upgraded to fiber in Hawaii. While network performance was perhaps not at the level desired, due to developing infrastructure in Hawaii and idiosyncrasies within the operating system, issues of network reliability had far greater impact on our remote observing operation. The experimental and limited nature of the ACTS program created a number of difficulties which one would almost certainly not face if using a more developed and/or commercial satellite system. The impact of the reliability issue is that at least one observer must be sent to Hawaii to use the telescope, in case of ACTS-related problems.

4. Conclusions

This experiment has explored the data requirements of remote observing with a modern research telescope and large-format detector arrays. While the maximum data rates are lower than those required for many other applications (e.g., HDTV), the network reliability and data integrity requirements are critical. The former issue in particular may be the greatest challenge for satellite networks for this class of application. We have also experimented with the portability of standard TCP/IP applications to satellite networks, demonstrating the need for alternative TCP congestion algorithms and minimization of bit error rates.

Reliability issues aside, we have demonstrated that true remote observing over high-speed networks provides several important advantages over standard observing paradigms. Technical advantages include more rapid download of data and the opportunity for alternative communication facilities, such as audio- and videoconferencing. Scientific benefits include involving more members of observing teams while decreasing expenses, enhancing real-time data analysis of observations by persons not subject to altitude-related conditions, and providing facilities, expertise, and personnel not normally available at the observing site.

Due to the limited scope of the ACTS project, future work from the standpoint of Keck Observatory will be concerned with establishing a more permanent remote observing facility via a ground-based network. At least two projects are under way in this direction: remote observing from the Keck Headquarters in Waimea, from where up to 75% of observing is now performed every month, and remote observing from multiple sites on the U.S. mainland using a slower T1 connection (Conrad et al. 1997, SPIE Proc. 3112). Trial tests of this latter approach over the Internet have been extremely promising.

© Copyright 1998 Astronomical Society of the Pacific, 390 Ashton Avenue, San Francisco, California 94112, USA

Next: NICMOS Software: An Observation Case Study
Up: Dataflow and Scheduling
Previous: OPUS-97: A Generalized Operational Pipeline System
Table of Contents -- Index -- PS reprint -- PDF reprint