| prev toc next |
Data discovery will be a core utility of the Virtual Observatory (VO). Registries that contain high-level descriptions of resources such as archives and services are essential for making data discovery efficient in a distributed environment. We review a framework architecture for VO registries currently under development within a International Virtual Observatory Alliance (IVOA) working group. We present an overview of a prototype implementation of the framework developed as part of the National Virtual Observatory (NVO) project. We illustrate how institutions can publish descriptions of their resources within their own registries. Other registries specialize in harvesting these descriptions to centralized locations where users may search them. These searchable registries can be global in their holdings or specialized toward particular subjects, audience, or resource types. Because the availability of resources changes over time, the framework must allow registries to automatically update their contents readily. We show how our prototype registry supports the NVO's first publicly released service, a Data Inventory Service, as well as a client-end analysis tool, Mirage. Components of this prototype are described in more detail in related posters (the metadata schema: Hanisch et al.; searchable registry: Greene et al.; publishing registries: Williamson and Plante; DIS: McGlynn et al., Mirage: Carliles).
We describe the architecture of a general cross-comparison engine capable of spatially matching sources in one astronomical source catalog with those in another. The software is highly modular and is written in portable C++. By performing many cross-comparisons of small sky regions in parallel, the software will scale to very large input catalog sizes. Support is provided for common catalog formats and data sources (e.g. local disk, database servers), and the addition of support for custom data formats and sources is simplified by the modular architecture employed. Hooks for customized source pre-processing and match-list post-processing are also available. Taken together, these attributes will make Compare a powerful package for cross-comparing astronomical catalogs on all scales and for cross-identifying sources between catalogs, allowing it to serve the needs of both large projects and individual astronomers. In particular, the package will be installed at San Diego Supercomputer Center, where it will perform cross-comparison between large-scale catalogs (such as MACHO and 2MASS) housed there. When complete, it will be a cornerstone compute service for the NVO. We have applied an early version of the package to the cross-comparison of the SDSS Early Data Release and the 2MASS 2nd Incremental Data Release catalogs, a computation central to the NVO Brown Dwarf demonstration project. Despite being performed sequentially, the comparison of 9.8 million SDSS sources to 0.5 million 2MASS sources completed in approximately 100 seconds when run on a 4 CPU Sun V480 with 16GB of memory.
The architecture of Montage, which delivers custom science-grade astronomical images, was presented at ADASS XII. That architecture has been tested on 2MASS images computed on single processor Linux machines that hold all image data in memory. This year, we describe the design of a grid-enabled version of Montage, suitable for large scale processing of the sky. It exploits to the maximum the parallelization inherent in the Montage architecture, whereby image reprojections are performed in parallel. All the re-projection jobs can be added to a pool of tasks and performed by as many processors as are available. We show how we can describe the Montage application in terms of an abstract workflow so that a planning tool such as Pegasus can derive an executable workflow that can be run in the Grid environment. The execution of the workflow is performed by the workflow manager DAGMan and the associated Condor-G. The grid processing will support tiling of images to a manageable size when the input images can no longer be held in memory. When fully tested, Montage will ultimately run operationally on the Teragrid. We will present processing metrics and describe how Montage is being used, including its application to science product generation by SIRTF Legacy Program teams and large-scale image processing projects such as Atlasmaker (this conference).
The eSTAR Project uses intelligent agent technologies to carry out resource discovery, submit observation requests and analyse the reduced data returned from a network of robotic telescopes. The agents are capable of data mining and cross-correlation tasks using online catalogues and databases and, if necessary, requesting additional data and follow-up observations from the telescopes on the network. We discuss how the agent technologies used in the eSTAR prototype have matured and have now been deployed in the field on research class telescopes that on first inspection don't fit into the autonomous robotic telescope paradigm.
AstroGrid (see http://www.astrogrid.org), a UK eScience project with collaborating groups drawn from the major UK data archive centres, is creating the UK's first virtual observatory. Together with the other major world-wide virtual observatory projects (see the International Virtual Observatory Alliance pages at http://www.ivoa.net), AstroGrid is creating a set of co-operating and interoperable software systems that will: allow users to interrogate multiple data centres in a seamless and transparent way; provide powerful new analysis and visualisation tools; and give data centres and providers a standard framework for publishing and delivering services using their data.
AstroGrid is developing a standardised framework to allow creative diversity, which will:
* to improve the quality, efficiency, ease, speed, and cost-effectiveness of on-line astronomical research * to make comparison and integration of data from diverse sources seamless and transparent * to remove data analysis barriers to interdisciplinary research * to make science involving manipulation of large datasets as easy and as powerful as possible.
AstroGrid has now completed its requirements capture and design stages, and has begun to release software capabilities on a three monthly cycle. It is using the iterative process, with 8 iterations, with each successive iteration release building a working system comprising increasing capabilities.
This presentation will describe the first functional release with it's 'Iteration 2' product, and the capabilities and functionality that this provides. The AstroGrid 'roadmap' will be highlighted, indicating how each successive three monthly iteration release will build on the foundation iteration 2 release, with increased end user capabilities (linking and accessing more diverse data sets, allowing manipulation of data with a wider range of tools and so forth).
The presentation will note the component based AstroGrid architecture, showing how external projects may be able to deploy components of interest in constructing there 'VO' - for instance the use of MySpace to provide secure intermediate 'grid' user storage areas.
More detailed descriptions of elements of AstroGrid, together with early demonstrator products developed in conjunction with the AVO (see http://www.euro-vo) and Australia-VO (See http://www.aus-vo.org) will be given in a number of supporting poster presentations at this meeting.
Two years ago, the Chandra Data Archive started the development of a bibliography database, tracking publications in refereed journals and on-line conference proceedings that are based on Chandra observations. As a result, we were the first mission that provided full coverage in allowing our users to link directly to articles in the ADS from our archive, and to link to the relevant data in the archive from the ADS entries. Subsequently, we have been working closely with the ADS and other data centers, in the context of the ADEC-ITWG, on standardizing the literature-data linking. Part of the objective is to enable authors to insert the links directly into their papers, since the exact linking is a significant part of the labor-intensive manitenance of the database.
Even when these links are automated, though, the coverage will still not be complete and tools are needed to update the database. In addition, we have extended our bibliography database to include all Chandra-related articles. Added categories are conference abstracts which may or may not identify specific observations, and instrumental, calibration, software (such as this paper), theoretical, and review articles. We are also keeping track of the number of citations of each paper. Obviously, in addition to providing valuable services to our users, this database allows us to extract a wide variety of statistical information that is of great interest to our mission.
The project comprises five components: the bibliography database-proper, a maintenance database, an interactive maintenance tool, a user browsing interface, and a webservices component for exchanging information with the ADS. All of these elements are completely mission-independent and we intend make the package as a whole available for use by other data centers.
This work is supported by NASA contract NAS 8-39073 (CXC).
| prev toc next |