Imagine an astronomer working a few years from now. She has just received some data from recent observations and notices an interesting feature. Using the feature as a template, a search is conducted of both literature and archived observations. The search returns a few relevant papers from the literature as well as data from other observatories. She concludes that further investigation is promising and uses integrated observing tools to develop new observations to provide more definitive data. These tools work in the same environment as the data analysis, archive and search tools, and allow generation of simulated data. The astronomer is assisted by the tool in selecting the best observatory and instrument. All the relevant information is stored in such a way that it can be directly used in a proposal. After a short time, the proposal has won time from the observatory. The astronomer refines the proposal in the integrated tools environment to generate an observing program. The observing tools not only help in improving the signal-to-noise, but guide her in avoiding a particular detector defect that could affect the data quality. It then also suggests observational strategies that generate an efficient program. The observations are executed and data are returned to the astronomer so that she can verify her original idea.
This sketch of the future is in contrast to today's environment for using most observatories. Astronomers are given a variety of tools (for example, in the case of HST, Exposure Time Calculators, Phase I Template, RPS2, StarView, STSDAS) which do not seemly communicate with each other, and often have significant learning curves. In addition, the astronomer must be familiar with hundreds of pages of documentation (Call for Proposals, Phase II Instructions, Instrument Handbooks, Data Handbook,...). As a consequence of this complexity, at the STScI, Program Coordinators and Contact Scientists must review every observation, provide extensive user support, and write a large amount of documentation. These shortcomings are exacerbated if an investigation requires observations with more than one observatory because each observatory has different sets of tools and documentation.
A textbook description of astronomical investigation would describe a linear process of hypothesis, observation and analysis. In the real world, astronomical discovery and investigation is not so linear. An interesting feature found in archival data may lead to a new observation, or a variation on a theory may be checked with archival data. The key aspect of the investigative process is that it is not compartmental and seamlessly flows from one aspect to another. The tools and user support should reflect this seamlessness.
We see two major areas where user support (both software tool and expert human support) can be improved so that astronomers do not encounter barriers as they move through the investigative process:
Although we provide a number of tools to astronomers, these tools are not well integrated. Our users are required to learn one set of tools for proposal preparation, another set for observation development, another set for archive use and yet another set for data analysis. Although each system strives for good user interface design, the basic paradigm is self-contained tools with limited interoperability, different presentation styles and different terminology.
We can take an alternate view by not looking at the phases in the investigative process, but rather at what kinds of tools are used. For example, visualization tools provide a means to display information and can be used for selection of targets, determination of observing constraints (e.g. orientations, timing), display of data and determination of the usability of archival data.
Another important aspect of software tools is documentation, which we take in a broad sense to include help files, manuals, technical memoranda, WWW pages, etc. With few exceptions, at present, it is difficult for both users and software to efficiently find information contained in the documentation. For example, for HST, reference information (e.g. filter transmission curves) is currently stored in two forms: one for humans (in the Instrument Handbooks), and one for software. These two expressions of the same data must be maintained and synchronized. As another example, since documentation is largely unstructured text, software can provide very little assistance to answer user's questions. This directly increases work load of STScI staff. Documentation needs to be considered as an integral part of the toolset and structured to allow straightforward access not only to users but also to software and search tools.
Direct staff support is currently a manually intensive (and therefore costly) process, whether it is direct interaction with observers or writing documentation. One cause for this is the lack of integration described above: since tools are not integrated, a larger than necessary amount of support is needed for routine use. However, there is another, more subtle (and therefore more problematic) root cause: it is currently cumbersome to put expert knowledge into a form which is easily used by humans and software. As a result we continue to rely on human support in areas even when it could be automated. The traditional way of capturing this information is by writing software tools, but this is an expensive and slow process. Non-traditional approaches (often called ``expert systems'') have had success in some areas, but have not had widespread utility.
We believe that the paradigm outlined in this paper provides benefits to the observatory and the observer. It allows the observatory to streamline operations and offer improved support by providing the right tools and information. Observatory staff will spend more time on innovative applications and exceptional circumstances rather than on routine questions and mundane operations, thus making efficient use of staff. The observer is able to more directly interact with the object being studied rather than be distracted with poorly integrated tools and hard to find information.
This latter point is particularly important for the future of astronomy. The concern has been voiced that observing tools and new modes of observing such as ``queue scheduling'' remove the observer from the data and will prevent graduate students from gaining practical experience in the process of scientific observation. The paradigm presented here fosters the development of good observers by reinforcing the connection between the scientist and the science and eliminating artificial distractions from software, documentation and processes.
Another long-term benefit of this approach is to encourage collaborative investigation. As tools become more integrated, and more interoperable, it becomes easier to share information and results with collaborators whether they are across the hall or across the planet. The barriers between wavelength domains can be lowered by integration, and will enable multi-wavelength, multi-observatory investigations. This is in contrast to the primitive state of collaboration available to most astronomers today: email and ftp-ing data files and programs. Moreover, effective collaboration will also benefit software developers at different observatories by encouraging common approaches and reuse of ideas and tools.
There are a number of compelling reasons why now is the right time to design and implement these changes. The theme of reuse and common tools is being embraced by other observatories as well, as demonstrated by interest at the Workshop on Observing Tools (STScI/GSFC Oct 98), the Proposal Process Workshop (NOAO Aug 98), and the the SPIE Conferences on Observatory Operations to Optimize Scientific Return (March 1998 and 2000). STScI's Spike planning and scheduling tools are being used by FUSE, AXAF, SIRTF, ESO/VLT and Subaru. (The latter two are notable since they are ground-based, not space-based missions.) The Multi-mission Archive at Space Telescope (MAST) serves multiple missions. Although the ``not invented here'' syndrome can inhibit reuse of tools, it is clear that several major observatories have found that reuse and commonality have powerful advantages.
In many cases, the technology available in the late 1980s or early 1990s still limits our thinking today. However the landscape of information processing has changed dramatically over the past 5 years. Languages such as Java allow tools to run on many different types of computers and over the WWW. Scripting languages (e.g. Python, Tcl) and request brokers (e.g. CORBA) make it relatively easy to seamlessly combine tools written in different languages. One side effect of the WWW is that we are accustomed to ``tagging'' our documents with HTML (Hypertext Markup Language), either directly or indirectly. The next generation, XML (Extensible Markup Language) provides a more powerful and flexible scheme which will allow us to offer context-sensitive help and searches which return focused answers.
We gratefully acknowledge discussions with our colleagues at STScI, ESO, Gemini, GSFC, SIRTF, Subaru, and other observatories which have led to many of the ideas presented here.