66 kB PostScript reprint
Next: Data Modeling and
Up: Software Systems
Previous: A Retrospective View
Astronomical Data Analysis Software and Systems IV
ASP Conference Series, Vol. 77, 1995
Book Editors: R. A. Shaw, H. E. Payne, and J. J. E. Hayes
Electronic Editor: H. E. Payne
W. B. Landsman
Hughes STX Co., Code 681, NASA/GSFC, Greenbelt, MD 20771
Abstract:
IDL (Interactive Data Language) is a commercial plotting, image processing,
and programming language that is widely used in astronomy. Although IDL
is a powerful language for spectral and image processing, it does not contain
applications specific to astronomy. Therefore, in 1990 I created the IDL
Astronomy User's Library (AUL), which is a collection of astronomy-related
procedures written in the IDL language, available via anonymous FTP from
idlastro.gsfc.nasa.gov.
I summarize recent developments in the IDL language of particular
interest to astronomers. I then mention some additions to the IDL Astronomy
Library since the previous report of Landsman (1993). Finally, I
critically examine possible drawbacks to the use of IDL for astronomical data
analysis.
IDL (Interactive Data Language)
is a commercial plotting, image processing, programming, and graphical
user interface (GUI) development language. It is a language designed to
allow much more rapid scientific data analysis than is possible using
FORTRAN or C. Features of IDL that promote this ``hands-on'' approach to
data analysis include an interpreted, vectorized compiler, an expressive
syntax, and numerous built-in spectral and image processing functions.
IDL is widely
used in astronomy, especially in the analysis of space-based data, and in the
fields of solar and planetary astronomy. Recent examples of astronomical
software packages based on IDL include Bloch et al. (1993), Brekke (1993),
McGlynn et al. (1993), Ewing et al. (1993), and Hall et al. (1994). In
this paper I summarize recent additions to the native IDL language and to the
IDL Astronomy User's Library, and consider possible drawbacks to the use of
IDL for astronomical data analysis.
IDL has undergone substantial evolution since its initial release in 1982.
Recent enhancements to IDL of particular interest to astronomers include the
following:
- IDL is now available for Microsoft Windows, Macintosh OS, most Unix
workstations, and Vax/Alpha VMS. Little or no programming is required to
transport IDL software between these different platforms.
- Many of the routines from the book ``Numerical Recipes'' (Press
et al. 1992) are now licensed and incorporated into native IDL. The
ease of use of the popular ``Numerical Recipes'' routines is significantly
enhanced within the interpreted IDL environment, as the routines can be
tested at the command level, and the results immediately printed or plotted.
- Native IDL now supports numerous disk file formats including
CDF, HDF, netCDF, TIFF, GIF, and JPEG. (Complete FITS and STSDAS I/O support
is available from the IDL Astronomy Library.)
Note that because IDL operates in virtual memory, most steps of a procedure
can be written without any particular disk format in mind. One simply
needs an initial procedure call to convert a disk file into IDL variables,
and a final procedure call at the end of the session to write variables
back to disk.
- The GUI (widget) development tools have been significantly enhanced.
For example, it is now possible to design a GUI without writing
any code. The IDL widgets require a significantly higher level of
programming sophistication than the traditional IDL command-line programming.
Nevertheless, the widget tools are sufficiently simple that astronomers can
develop their own personalized GUI tools, with only a modest investment of
programming time.
- IDL is now sold by distributors in Japan and throughout Europe. About
one-third of the users of the IDL Astronomy Library come from outside the
United States.
IDL is a general software package, used in such fields as remote sensing and
medical imaging, in addition to astronomy. As such, it does not contain
any procedures specific to astronomy.
In 1990, I created the IDL
Astronomy User's Library (IAUL), which is a collection of astronomy-related
procedures written in the IDL language, available via anonymous FTP from
idlastro.gsfc.nasa.gov (Landsman 1993). Astronomy-related IDL software
contributed from the community is checked for appropriateness, accuracy, and
programming standards. Thus, the site is intermediate between an
unmoderated bulletin board and a unified data analysis package.
The IAUL does not contain any instrument specific software, although it does
contain pointers to other anonymous FTP sites containing instrument specific
IDL software. (Anonymous FTP sites exist containing IDL reduction
software for the IUE, HST/GHRS, COBE, SOHO, and
ROSAT instruments.)
An important addition to the IAUL is the support for the 25 astronomical
coordinate systems discussed by Greisen & Calabretta (1995). The
coordinate conversion software was written by Rick Balsano and the more
complicated transformations were verified by Imannuel Freedman. Additional
procedures exist that recognize three ways that the world coordinates may
be stored in FITS keywords: (1) the original FITS/AIPS system with a reference
pixel, rotation, CROTA2, and pixel scale CDELTi, (2) the IRAF/ST ScI system
with a reference pixel and a coordinate description matrix CDi_j, and (3)
the Greisen & Calabretta proposal which includes both a pixel scale and
a rotation and skew matrix. The software also recognizes headers from the
ST ScI Digitized Sky Survey and will apply the appropriate nonlinear
transformation to the pixel coordinates.
A recent contribution from Tom McGlynn is a generalized FITS reader
that supports both variable-length binary tables, and random groups. An
especially important feature of this software is that binary table columns
are directly mapped into the tags of IDL structure arrays. The full
IDL data analysis capabilities (e.g., plotting, sorting, and subscripting) can
then be applied to the structure variable.
Other recent additions to the IAUL include mathematics and statistics code
to complement the intrinsic ``Numerical Recipes'' routines. Examples
include code for principal components analysis (Murtagh & Heck 1987),
Kolmogorov-Smirnov statistics, and cubic-spline smoothing.
Every language has its strengths and weaknesses. In this final section I
examine five possible drawbacks a hypothetical astronomer might make to the
use of IDL for his data analysis.
- IDL is an interpreted language, and thus slower than compiled
languages such as FORTRAN or C. This statement is technically true, but
properly vectorized IDL code suffers little performance penalty compared to
FORTRAN or C. (IDL code is usually most
readable when it is properly vectorized, so that, in this case, ``the good
coincides with the beautiful.'') In fact, IDL often outperforms pedestrian
FORTRAN or C code, since the many built-in image and spectral processing
functions of IDL are highly optimized.
For the rare cases (such as generalized median filters) where the code
cannot be vectorized, and where performance speed is crucial, IDL offers
several different ways to link to a FORTRAN or C program, including a
fast dynamic link to the executable.
- The use of virtual memory in IDL is not suitable for very large
images. While the virtual memory capabilities of workstations have
dramatically improved in the past few years, it is also true that the increase
in the size of CCDs has been just as dramatic.
Very large ( 16 MB) images can still be processed with IDL, but the array
must be processed in pieces, so that the simplicity of the IDL code is then
lost.
- Professor X cannot use my IDL software since she does not have an
IDL license. This objection is valid, although Professor X can probably
decipher the algorithm, since the IDL syntax was designed for clarity rather
than brevity.
In addition, IDL supports remote procedure calls (RPC), so that a licensed
client can support a server without an IDL license.
- I cannot find a galaxy surface photometry package in IDL. Most
astronomical software is not written in IDL, and the software in the IDL
Astronomy Library is an order of magnitude less complete than that in the
large public domain systems such as IRAF or MIDAS. For this reason, IDL is
unlikely to be an astronomer's sole working environment, although it is likely
to be his preferred environment for developing new code.
- I am not going to purchase an IDL license when there is public domain
software available. The purchase of a commercial license can be worthwhile
if it saves programming time and costs, and significantly enhances one's data
analysis capabilities. Some IDL features are duplicated by various types of
public domain software. IDL may or may not be a worthwhile investment,
depending on an astronomer's needs, skills, and budget.
Acknowledgments:
The IDL Astronomy Library is funded under NASA grant NAS5-32583 to Hughes
STX.
References:
Bloch, J. J., Smith, B. W., & Edwards, B. C. 1993, in Astronomical Data Analysis Software and Systems II, ASP Conf. Ser., Vol. 52, eds. R.J. Hanisch, R.J.V. Brissenden, & J. Barnes (San Francisco, ASP), p. 243
Brekke, P. 1993, ApJS, 87, 443
Ewing, J. A., Isaacman, R., Gales, J. M., Chintala, S.,
Kryszak-Servin, P., & Galuk, K. G. 1993, in Astronomical Data Analysis Software and Systems II, ASP Conf. Ser., Vol. 52, eds. R.J. Hanisch, R.J.V. Brissenden, & J. Barnes (San Francisco, ASP), p. 367
Greisen, E. W., & Calabretta, M. 1995,
Hall, J. C., Fulton, E. E., Huenemoerder, D. P., Welty, A. D., &
Neff, J. E. 1994, PASP, 106, 315
Landsman, W. B. 1993, in Astronomical Data Analysis Software and Systems II, ASP Conf. Ser., Vol. 52, eds. R.J. Hanisch, R.J.V. Brissenden, & J. Barnes (San Francisco, ASP), p. 246
McGlynn, T. A., White, N. E., & Scollick, K. 1993,
in Astronomical Data Analysis Software and Systems III, ASP Conf. Ser., Vol. 61, eds. D. R. Crabtree, R. J. Hanisch, & J. Barnes (San Francisco, ASP), p. 34
66 kB PostScript reprint
Next: Data Modeling and
Up: Software Systems
Previous: A Retrospective View
adass4_editors@stsci.edu