Next: Developing a Wavelet CLEAN Algorithm for Radio-Interferometer Imaging
Up: Software Applications
Previous: Development of Radio Astronomical Data Reduction Software NEWSTAR
Table of Contents - Subject Index - Author Index - Search - PS reprint - PDF reprint

Devillard, N. 2001, in ASP Conf. Ser., Vol. 238, Astronomical Data Analysis Software and Systems X, eds. F. R. Harnden, Jr., F. A. Primini, & H. E. Payne (San Francisco: ASP), 525

ESO C Library for an Image Processing Software Environment (eclipse)

Nicolas Devillard
European Southern Observatory

Abstract:

Written in ANSI C, eclipse is a library offering numerous services related to astronomical image processing: FITS data access, various image and cube loading methods, binary image handling and filtering (including convolution and morphological filters), 2-D cross-correlation, connected components, cube and image arithmetic, dead pixel detection and correction, object detection, data extraction, flat-fielding with robust fit, image generation, statistics, photometry, image-space resampling, image combination, and cube stacking. It also contains support for mathematical tools like random number generation, FFT, curve fitting, matrices, fast median computation, and point-pattern matching. The main feature of this library is its ability to handle large amounts of input data (up to 2GB in the current version) regardless of the amount of memory and swap available on the local machine. Another feature is the very high speed allowed by optimized C, making it an ideal base tool for programming efficient number-crunching applications, e.g., on parallel (Beowulf) systems.

Running on all Unix-like platforms, eclipse is portable. A high-level interface to Python is foreseen that would allow programmers to prototype their applications much faster than through C programs.

1. Introduction

The eclipse library is intended as a low-level software facility to offer image-processing and data analysis functionality through calls to a C library. End-users can also access most functions through Unix commands, but the main idea is to program stand-alone pipeline commands running without user interactions.

Begun in 1995 to provide Adonis users with a usable pipeline environment to reduce their data in real-time in the telescope control-room, eclipse has been further extended to support more instruments at ESO (ISAAC, CONICA, WFI, possibly VIRMOS) and also in other institutes (Subaru, CFH).

2. Base requirements

From the eclipse developer's manual, its base requirements include:

3. Portability

As a portable library, eclipse should not only be able to be compiled on various platforms of today, but it also should be compilable on future platforms. Adhering to programming standards whatever they are (POSIX, ANSI, or simply coding conventions) ensures a reasonable survival of the library over time.

More generally speaking, writing portable software increases the quality of the code. By compiling the source with various compilers and running it on different platforms, weaknesses and defects are put under stress and are therefore revealed, whereas they might hide indefinitely or be very hard to find with a single development environment. It is a good idea to respect global programming conventions whenever possible.

Specializing a piece of software for a given platform is usually done for reasons of speed, to access the dedicated hardware or software present on the local system. Specializing software also means reducing its life expectancy, because underlying hardware or software may not be vendor-supported in the future.

4. File formats

The following file formats are supported by eclipse:

5. Coding standards

Coding style for eclipse conforms to that specified in the document ``Recommended C Style and Coding Standards.'' Any eclipse developer should have most of the rules described there in mind when coding.

Following coding rules is often seen as a matter of taste. After all, if a programmer writes thousands of lines without indentation, it will also compile, and if it runs in the end, who cares about the style!

The philosophy behind coding rules and conventions is that a piece of software is defined by more than just the executable file that is produced after compilation. Pages of code are very often modified to correct bugs, enhance functionality, include into other (unforeseen) contexts, etc. It is vital that any developer in the team be able to modify somebody else's code with minimal overhead.

6. Modules

Most eclipse modules can be detached from the library to be reused as code snippets in other applications, thereby offering versatile functionality:

7. Memory handling

Memory handling in eclipse has been carefully studied to avoid the usual limitations met in image processing software systems. The upper limit for allocatable memory is not fixed by the amount of RAM and swap found on the system, but by the amount of disk space accessible to the current user. The price paid is degraded performance as soon as the system runs out of memory, but the gain is a uniform interface to the programmer who does not have to care at all about memory handling. The same source code will process a gigabyte of data no matter what the locally available memory.

8. Algorithms

Several ESO pipelines are now based on eclipse for number-crunching developments. This has produced a number of high-level algorithms specialized for data processing, for one instrument or another. Some of these algorithms are adaptable enough to be used for other instruments not initially foreseen (e.g., the high-level data reduction tasks written for CONICA, a VLT instrument, also perform very well on CFHT Adaptive Optics data).

9. Parallel processing

Since eclipse offers its functionality through a Unix command-line call, it is possible to parallelize it through coarse-grain parallelism. A master machine can send a list of Unix commands to run to a group of slave nodes and get the results back. Since eclipse is C-based, it is easy to add potential communication procedures (based e.g., on MPI or PVM) to make the algorithms fully parallel.

Wide-Field Imager (WFI) data from the 2.2-mtelescope at La Silla is currently processed on the ESO Beowulf system using eclipse.

10. Python/eclipse

Like any other C library, eclipse can easily be interfaced to appear as a Python extension. This would allow image processing programs to be object-oriented, and to benefit from all the advantages of using Python.

The eclipse source code is currently under modification to make it easier to interface to Python. The wrapper code will be generated by SWIG and a Python-compatible procedure will be distributed to offer eclipse as a Python extension.


© Copyright 2001 Astronomical Society of the Pacific, 390 Ashton Avenue, San Francisco, California 94112, USA
Next: Developing a Wavelet CLEAN Algorithm for Radio-Interferometer Imaging
Up: Software Applications
Previous: Development of Radio Astronomical Data Reduction Software NEWSTAR
Table of Contents - Subject Index - Author Index - Search - PS reprint - PDF reprint

adass-editors@head-cfa.harvard.edu