Next: The Chandra Xray Center Data Archive Interfaces
Up: Data Pipelines and Quality Control
Previous: Observation Interval Determination for the Chandra X-ray Observatory
Table of Contents -
Subject Index -
Author Index -
PS reprint -
Wolk, S. J., Petreshock, J. G., Allen, P., Bartholowmew, R. T., Isobe, T., Cresitello-Dittmar, M., & Dewey, D. 2000, in ASP Conf. Ser., Vol. 216, Astronomical Data
Analysis Software and Systems IX, eds. N. Manset, C. Veillet, D. Crabtree (San Francisco: ASP), 453
The Chandra Monitoring System
S. J. Wolk1, J.
G. Petreshock2,
P. Allen3, R. T. Bartholomew4,
T. Isobe5, M. Cresitello-Dittmar6
Harvard-Smithsonian Center for Astrophysics
D. Dewey7
Massachusetts Institute of Technology
Abstract:
The NASA Great Observatory
Chandra was launched July 23, 1999
aboard the space shuttle Columbia.
The Chandra Science Center (CXC) runs a monitoring and trends
analysis program to maximize the science return from this mission.
At the time of the launch, the monitoring portion of this
system was in place.
The system is a collection of multiple threads and programming
methodologies acting cohesively.
Real-time data are passed to the CXC. Our real-time tool, ACORN (A
Comprehensive object-ORiented Necessity), performs limit checking
of performance related hardware. Chandra is in ground contact
less than 3 hours a day, so the bulk of the monitoring
must take place on data dumped by the spacecraft.
To do this, we have written several tools which run off of the CXC data
system pipelines.
MTA_MONITOR_STATIC, limit checks FITS files containing hardware
data. MTA_EVENT_MON and MTA_GRAT_MON create quick look data for
the focal place instruments and the transmission gratings. When
instruments violate their operational limits, the responsible
scientists are notified by email and problem tracking is initiated.
Output from all these codes is distributed to CXC
scientists via HTML interface.
The goal of the Chandra X-Ray Center's Monitoring and Trends
Analysis project (M&TA) is to maximize science return
by keeping the science operations team (SOT) informed of the
scientific performance of the spacecraft. The SOT then uses this
information to maximize the science return of the spacecraft.
M&TA occupies a middle ground between flight operations,
chiefly concerned with the health and safety of the spacecraft, and
validation and verification, concerned with the
scientific validity of the data taken and whether or not they fulfill
the observer's requirements.
M&TA requirements, initially set down four years before launch
were for 130 tools which form a miniature data system. These tools
ranged from monitoring the temperature stability of
the exquisitely accurate mirrors, to the total number of photon's
incident on the microchannel plates of the
High Resolution Camera (HRC) and measuring the total
observing efficiency of the observatory. The reality of budgets
stepped in; the project was assigned less than 5 man years of
programming support, leaving about 2 weeks to develop each tool. This
was deemed unrealistic and the entire system
was redesigned, leaving the goals intact with careful re-prioritization
made with respect to those portions which were flight critical and
those which could wait.
What follows is a success story. We redesigned the M&TA system to
have a symbiotic relationship with the main of the data system
pipeline processing. At each stage of the processing, we have been
able to examine the existing data product, evaluate it and rapidly return
needed information to the SOT. Along the way,
we have developed an infrastructure which was able to operate independent
of the overall readiness of individual Chandra X-ray Center Data
System (CXCDS) functionality in order to support pre-flight testing.
Later, components have been seamlessly merged into the rest
of the data system as
higher level pipelines come on line. In the following sections, we
describe how we handle data of various forms and at various levels of
processing.
Raw data are the data packets as they come from the spacecraft.
These are sent to the CXC in two forms: (1) Realtime data, which
arrive at the science center seconds after the recorded events have
occurred; and (2) Dump data which are sent to the ground during a 45
minutes pass approximately once every 8 hours.
In some sense, these are the most important data the SOT sees as they
give
the most rapid visibility into the spacecraft's behavior. The main CXC
pipeline is required the process raw data by converting it to FITS
format. This inevitably requires a minimum data size, adds bulk to
the data and time to the processing.
For the M&TA project, raw data are processed by the ACORN tool.
This tool is a collection of C++ objects integrated
under a Tcl/Tk GUI. Data can either stream into the tool via UDP
or can be loaded from individual files. The former method is
primarily used for realtime data, the latter for dump data or analysis.
When analyzing file based data, ACORN can step through the file
either forward or backward at a user
adjustable rate. These features are very desirable for data analysis
just before or after a specific spacecraft event (such as a safe mode
entry).
ACORN uses the NASA compliant databases to decommutate the data.
It can output the HEX pattern, calibrated or
uncalibrated value for any location in the data stream. It can also
sense limit violations and output the nature of these alerts (red or
yellow). Alerts are written to disk where the MTA Problem Response System
(MTA_PRS) acts on them by cataloging the error and mailing the
responsible scientist. The GUI directly displays
all selected data to the screen. Multiple display
pages can be set up with associated values logically grouped by the
user. Data can also be captured in multiple files as tab delimited
fields. This allows post processing analysis by any number of software
tools including RDB, IDL and EXCEL. ACORN can also stream the tab
delimited data the STDOUT. This allows us to use ACORN as part of any
UNIX pipeline process.
The CXCDS pipelines add a great deal of value to the Chandra data.
Raw values are calibrated, events are identified and the spacecraft
pointing and dither are taken into account. Instead of performing
these calculations anew, the M&TA pipelines interrogate the files
generated by the main processing.
Data produced by CXCDS consists of FITS files. These are
generally of two types - table files and event lists. The tabular
data tends more to contain spacecraft status information, which instrument
is at focus, power consumption by the instruments, spacecraft pointing
and radiation count rates for example. These data come through CXCDS
in atomic units of about 1 hour. The MTA_MONITOR_STATIC monitors all the
spacecraft data as soon as the file is closed by the
tool which produces it. Using a collection of C and C++ tools,
selected data are checked that they are within expected limits and
statistics on the performance of the monitored system (mean, standard
deviation, min. max. etc.) are calculated and reported out. The
pipeline uses IDL to produce a strip chart for each monitored value.
If limit violations occur, they are written to disk and handled by MTA_PRS.
As a data product, X-ray data are fundamentally different than lower
energy data in that each individual photon is recorded. The FITS
incarnation of this is an event list. This is a file which
contains the time, location and energy of each photon which struck a
given detector. The MTA_MONITOR_EVENT pipeline uses
tools similar too those used to monitor
spacecraft data to provide a snapshot
of the focal plane at all levels of the processing. Since the focal
plane is quite large (up to 32K pixels), the tool locates
the brightest regions and produces full scale representations
of those regions. In addition to an
image of the focal plane, the pipeline produces histograms of the event
shape (grade) and the energy (pulse height) on various scales of
instrumental relevance. The data provide the various instrument
scientists quick visibility into the performance of the instruments and
the pointing reconstruction.
One of the unique aspects of the Chandra mission are the transmission
gratings. The M&TA project has 7 tasks specifically for the
monitoring of the gratings, the bulk of these are handled via the High
energy transmission grating Analysis tool Kit (HAK).
The HAK code is a collection of IDL procedures
developed during ground calibration with an eye toward flight. This
pipeline consecutively produces an image of the focal plane, plots a
timing analysis, finds the zero order and characterizes it, finds the
dispersed spectrum, identifies lines and fits the line. The outputs are
RDB and ISIS files (Houck et al. 2000) containing line energy and width data.
The most critical part of any software design is not the
functionality. This is usually determined by the problem. The most
critical aspect is the user interface, the software must be easy and
intuitive to use or it will be eschewed and the programming effort will
be duplicated by users.
Our interface problem was complicate by several different programmers of
various skill levels using different software. All the output
however, is directed to a series of web pages. The top level
web page http://asc.harvard.edu/mta/sot.html
allows the user the
view realtime data (processed by ACORN), telemetry (processed by
MTA_MONTIOR_STATIC), photons (Processed by MTA_MONITOR_EVENT) or
alerts. In addition to being able to view the data from anywhere in
the world, it is easy for the user to view the data to navigate
to deeper levels of the web hierarchy which is arranged by date,
subsystem and observation ID. It is also easy for the data
system to secure proprietary data as needed.
The trending functions of the Monitoring and Trends Analysis system
are currently the least robust. The long term plan is to have an
integrated system using the full CXCDS data base. It was clear early
during the development of the system these resources would be unavailable
by flight. Our approach was to use the statistics generated
by the earlier pipelines. We simply convert the HTML tables
back into text. These data have low time resolution, typically
one day, and are not easily time correlated. However these products
provide access to the performance of all spacecraft systems over the
duration of the mission.
Development of the M&TA system is ongoing, currently fulfilling a
little less than half of our pre-flight requirements. The pre-flight
development focused on broad scope tools such as event monitoring and
data inspection (Petreshock et al. 2000). Over the next year
we will continue the development of broad focus tools, such as the
databases and computations of existing data. We will also begin
to develop tools targeted at specific problems such as charge transfer
inefficiency, telemetry saturation, and focus quality.
Acknowledgments
This project is supported by the Chandra X-ray
Center under NASA contract NAS8-39073.
References
Houck, J. et al. 2000, this volume,
591
Petreshock, J. et al. 2000, this volume,
475
Footnotes
- ... Wolk1
- Chandra X-ray Science Center
- ... Petreshock2
- Chandra X-ray Science Center
- ... Allen3
- Chandra X-ray Science Center
- ... Bartholomew4
- Chandra X-ray Science Center
- ... Isobe5
- Chandra X-ray Science Center
- ... Cresitello-Dittmar6
- Chandra X-ray Science Center
- ... Dewey7
- Chandra X-ray Science Center
© Copyright 2000 Astronomical Society of the Pacific, 390 Ashton Avenue, San Francisco, California 94112, USA
Next: The Chandra Xray Center Data Archive Interfaces
Up: Data Pipelines and Quality Control
Previous: Observation Interval Determination for the Chandra X-ray Observatory
Table of Contents -
Subject Index -
Author Index -
PS reprint -
adass@cfht.hawaii.edu