68 kB PostScript reprint
G. H. Jacoby
National Optical Astronomy Observatories, P.O.
Box 26732, Tucson, AZ 85719
National Optical Astronomy Observatories, operated by the Association of Universities for Research in Astronomy (AURA) under a cooperative agreement with the National Science Foundation.
The ADASS conferences are intended to foster discussions between astronomers and software developers so that the computational needs of researchers are understood and met. Are the software developers, in fact, addressing the needs of the astronomical community?
I estimated the applicability of the papers presented at this conference by counting the number of papers in each of 12 categories identified as steps in an observational research project. Table 1 lists these steps along with the paper count. Some papers provide software in more than one area; others do not fall into any of these classes, but still provide critical support (e.g., seven papers on FITS, an essential tool usually taken for granted).
The plurality of papers fall into the reduction and analysis step, as expected from the original focus of the ADASS conferences (and suggested by the conference title). There is, however, a healthy diversity outside this theme. In particular, there is an impressive number of papers in the crucial modeling step, which represents ``science extraction'' from the data. Additionally, there is a large force dealing with data distribution and access, a step that has grown in importance now that large databases are available.
No one seems to be dealing with judgment and subjective issues by using software (e.g., identifying important scientific problems, judging proposal merit, writing papers). It is probably wise to avoid these until artificial intelligence software matures further. Thus, the ADASS conferences now transcend reduction software, and are not at all IRAF users meetings as some originally feared.
Table: Applications Where Software Helps Astronomers.
I see two primary drivers for software needs five years hence. On this timescale, it is the very big projects which get our attention because they require 3--5 years of development. Additionally, recurring and growing problems demand solutions if we wish to avoid being consumed by them.
The biggest projects are the new telescopes. My view is biased toward ground-based optical work, but astronomers working in other regimes will have similar concerns. The large advanced technology telescopes expected by the year 2000 present special challenges to software developers. Adaptive optics telescopes will require very complex, highly distributed computer systems to achieve the high spatial resolution they promise, and to control their exotic instrumentation; spatial resolution drives detectors toward tiny pixels in huge arrays. To run the telescopes and instruments, and to diagnose failures rapidly, essential software includes clear and carefully engineered GUIs. To quickly estimate data quality, fast and insightful visualization techniques are needed. The vast data arrays will tax networks, I/O systems, and storage capabilities, so compression techniques will be crucial. Perhaps lossy compression will be used most of the time rather than rarely, as is current practice.
Large databases are being built (e.g., Sloan digital sky survey,
HST
archives, NOAO archives) which already are changing the way astronomers
do research. Using telescopes may become a minor part of observational
astronomy, but first, access to archives must be easy and fast. Software
will be needed to browse the catalogs and to download data ``samples''
(i.e., representative subsets) to verify that the catalog delivers the
advertised information. Again, the need for compression techniques and
the acceptance of lossy data may be necessary to live within network
bandwidths and on-line storage limits. Also, the data quality must
be documented and attached to the database entries (e.g., was it
photometric, were images 
, what were the filter characteristics?).
A comment I hear often is ``I missed your paper---I don't have time to read the journals anymore.'' If this trend continues, research will become pointless. One extreme view is to eliminate paper journals, for which no one has shelf space anyway. The move to electronic publishing provides a solution: rather than subscribing to a few journals, astronomers can subscribe to selected topics in all the journals. For example, one could subscribe to all papers dealing with lithium abundances. When a paper is ``published'' on lithium in any journal, an e-mail message is sent to the subscriber indicating that the paper can be downloaded. The embarrassment of missing a critical paper in one's own specialty is avoided, and the astronomer is protected from information overload. A concern is that scientists will be channeled into narrow disciplines without serendipitously reading interesting material in other fields.
One journal tool is nearly here: full text on-line searches. We already have abstract searches, but these place researchers at the mercy of authors who may not realize the value of all aspects of a paper.
Another frequent complaint is that researchers spend an increasing fraction of their time writing grant proposals. With success rates dropping, funding is a major time sink. This becomes a vicious circle; rather than learning about nature, scientists are learning about funding.
Despite the public's keen interest in astronomy (e.g., the Comet SL9/Jupiter encounter), money continues to be a problem. While public excitement and support for astronomy may not correlate perfectly, there is some relation. Perhaps, if we keep new discoveries interesting and frequent by tightly integrating the research environment with education and the media, the proposal writing cycle can be broken. To do so requires that reduction, analysis, and modeling software be smarter so that scientific results can be released within days rather than years. We also need visualization tools; not everyone has a graphic artist to turn a scientific result into a picture for non-specialists.
Most of what scientists do is a sequence of operations they've done before. Can they teach software to perform those repetitive steps so they can concentrate on the subjective aspects of interpreting results? Expert software in photometry, spectroscopy, and statistics, for example, could alleviate a lot of the tedium. Can a photometrist program be given 15 images of a field to find all variable stars having periods of 5 to 60 days to build a Cepheid finding expert? Can a spectroscopist program be taught to classify stars and nebulae and derive abundances? Can a statistician program be taught to answer questions like: ``Are these two distributions different, what is the confidence level that these lithium detections are real, what are the errors on Cepheid period determinations?''
After four ADASS meetings, clear software trends are emerging.
Graphical User Interfaces (GUIs) -- I prefer command line interfaces because they can be built into scripts, allow type ahead, and require little screen area and few computer resources. Nevertheless, most people prefer GUIs because they provide easy navigation through increasingly complex software packages. But, if a task doesn't need a GUI, don't build one just because you can.
World Wide Web (WWW) -- ``The Web'' has become a major resource, growing from a curiosity to a serious research tool in just two years.
Tbytes -- Disk space used to be measured in MB. Now we talk about GB. The standard is becoming TB.
Object Oriented Systems (OO) -- OO programming concepts seem esoteric, but there are advantages to thinking this way. Improved performance and high level programmability are driving databases and programming tools to use OO.
FITS -- FITS usage has been a trend for over 10 years. FITS continues to develop (7 papers) and to generate tremendous interest among software folks. Astronomy is fortunate to have a highly viable data standard; congratulations to the originators of FITS (Wells, Greisen, & Harten 1981)!
IDL -- IDL grew from the field of astronomy to become a viable commercial product. IDL's future appears healthy thanks to its use in more lucrative markets (e.g., medicine and earth sciences) which feed software back to astronomy.
IRAF -- After 
years in the community, and with development
for various projects (e.g., HST, PROS, EUVE, and NOAO), IRAF has
become a basic research tool in astronomy.
It is difficult to identify a trend which has not yet begun, but let me suggest that astronomy needs the following two.
Education software -- The public deserves to know as much about
the universe as we do and in a timely manner. Electronic picture books
(see the paper by Brown p. 
) can improve information
turnaround, but we need hands-on experiments, too.
Error Propagation -- The failure to track errors properly from detected photons to final answers has plagued subfields of astronomy and created controversies lasting decades. Analysis systems need to help astronomers with the subtle details of forming valid statistics and errors.
More and more, astronomers rely on software in every aspect of their jobs. ADASS conferences provide programmers the opportunity to present new software that helps astronomers in their daily work, and astronomers are encouraged to attend the ADASS meetings to become more effective researchers.
Wells, D. C., Greisen, E. W., & Harten, R. H. 1981, A&AS 44, 363
68 kB PostScript reprint