NVO Architecture Virtual Observatory meeting Garching, Germany June 2002 Reagan W. Moore San Diego Supercomputer Center moore@sdsc.edu Abstract: The National Virtual Observatory is assembling web services, data analysis pipelines, and grid software to support astronomical research. The initial goal is to facilitate access to existing sky surveys and provide standard services for manipulating catalogs and image collections. The long-range goal is to support analyses of entire sky surveys and enable applications that examine multiple collections. The layers of software infrastructure required to provide these goals are examined. 1. National Virtual Observatory Testbed The National Virtual Observatory will provide mechanisms to discover, access, and analyze images and information stored in all-sky surveys. There are multiple ways to characterize the desired capabilities, in terms of interactivity, scale of analysis, and type of discovery capabilities. Interactivity requirements vary from interactive web services, to sustained processing pipelines, to large-scale analyses that require the massive computing power of the National Science Foundation Teragrid. The scale of analysis varies from the interactive picking of individual images from a sky survey or a small number (say 10-100) records from a survey catalog, to applications that will require processing of an entire image collection. Finally, discovery capabilities range from requests that can be satisfied by a single sky survey, to requests that require the integration of results from multiple sky surveys across multiple data encoding formats. The challenge in designing an architecture for the NVO is to recognize the importance of each of the different usage models, while minimizing the number of software layers needed to implement the desired capabilities. The architecture should also maintain existing capabilities already provided by many of the sky surveys and collection catalogs. The NVO architecture approach is based upon identifying common web-based services that can be easily implemented as new capabilities on existing systems, identifying the data grid requirements needed to federate existing collections to support massive computations, and identifying the information and knowledge management systems that will unify discovery across the catalogs. The system architecture is designed to build upon each of these three sets of requirements, starting through the addition of simple web services for picking images and records, and the development of a common set of semantics for describing physical quantities across the collections. Larger scale computational jobs will then be supported by migration onto the supercomputer resources provided by the NSF Partnership for Advanced Computational Infrastructure. The PACI partnership is implementing a Teragrid that couples high performance clusters at each of four sites via 10-Gbit/sec network links. The major all-sky surveys are being replicated onto resources attached to the Teragrid to enable large scale analyses. The web services environment and the large-scale analysis testbed will then be augmented with production pipeline systems that interface to the grid batch computing services. The NVO architecture is based upon the ability to off-load large computational jobs onto an NVO testbed, which is being built on top of the Teragrid. In turn, the Teragrid is being assembled by integration of the Globus Grid Toolkit that is being developed through an NSF National Middleware Initiative, with data management software that is used to implement digital libraries, data grids, and persistent archives. The data management infrastructure is based upon the San Diego Supercomputer Center Storage Resource Broker, which is used to build data sharing systems, data publication systems, and data preservation systems. The data sharing systems make it possible to manage personal collections, while the data publication systems are used to provide discipline wide access to data collections. The Teragrid will also include knowledge management systems that organize relationships between the semantic terms used to describe physical quantities in the all-sky surveys. The Uniform Content Descriptors are used as tokens that can be associated with attributes in collections. Attributes that have the same associated token are assumed to be equivalent. For collections that do not have shared tokens, ontologies are used to define additional semantic terms that can be used to build relationships between catalogs. 2. Architecture requirements Three exemplary applications are being used to drive requirements for the NVO system architecture, based upon a Brown Dwarf Search, a Galaxy Morphology analysis, and a Gamma Ray Burst analysis. The system components that are needed to support each application are listed in Table 1. An attempt has been made to identify common requirements across the applications. The technologies that are being used to support the applications include web-based access through the VOTable catalog access system, web-based access through the SRB collection management system for access to replicas of the 2-Micron All-Sky Survey and Digital Palomar Observatory Sky Survey, and web-based image cut-out services. Requirements are also coming from applications that require the reprocessing of the 2-Micron All Sky Survey. Together they exemplify the requirements for the picking and large-scale analysis environments, but do not address the batch pipeline processing environment. For pipeline processing, the Montage software from IRSA is being evaluated for its ability to interact with batch systems in grid computing environments. Table 1. Typical Application Requirements Brown Dwarf Search Galaxy Morphology Gamma Ray Burst Information repository abstraction for characterizing interactions with catalogs in databases Distributed joins across multiple catalogs Mechanism to specify relationships within a complex VOTable as a concept space Standard query language for interacting with entire catalogs. Catalog Cone Search and positional cross-matching of extended sources Cone search across multiple catalogs Standard information interchange format based on VOTable. Mechanism for formatting data into a standard catalog format - VOTable Standard information exchange format based on VOTable Web based portal to control input parameter selection. Portal for defining input and processing steps. Portal for registration of interest in desired events. Standard access mechanism based upon SOAP protocol and WSDL services. Support for interactive display of results, including parsing of VOTable format. Service registry for publishing service descriptions, and supporting a service directory Service registry for catalogs Service registry for both catalog and image data Method registry for methods that can be applied against data models. Support for distribution of notifications to registered users. Standard display mechanism for results. Support for publication of new tables Support for publishing results Procedural support for composing results from multiple catalogs. Support for visualization services including contouring. Support for tracking astronomical events Interactive response time for browsing tables and columns in databases. Support for analyses of images, including galaxy morphology analysis Integrate existing systems and tools Grid computing interface to support high-throughput analyses Support for mapping from current format to standard format Persistent interactive access to image archives Image retrieval from image archives, including partial images or cutouts Image retrieval from image archives, including partial images or cutouts Modular design to enable rapid prototyping. Support for exception handling. 3. System Components The Global Grid Forum is developing descriptions of system architectures that integrate web-based access with grid computing. Figure 1, taken from the characterizations published by William Johnston in the Grid Protocol Architecture working group, defines the set of software layers that can be used to support applications running on a grid. The components include: * Portals ? the user interfaces that manage interactions with the multiple NVO services. Portals encapsulate domain knowledge by associating explicit mediators or transformation programs with access to the chosen catalogs and archives. Portals also provide interfaces to control image processing pipelines. * Analysis and support tools ? the NVO supplied applications for processing image and catalog records. These tools provide capabilities such as conversion to a standard data format, visualization mechanisms for 2D and 3D displays, metadata extraction mechanisms for populating attributes in catalogs, and algorithms for processing data. * Application support platforms ? these typically provide an environment for interactive execution support. Examples include Java execution, Corba object management, DCOM integration, WSDL/SOAP services, and Open Grid Service Architecture services. * Grid support environment ? the set of collective operations used to build a common data management and job execution environment across administration domains. Grid services manage distributed state information. Figure 1. Application Support Environments The capabilities provided by the grid support environment can also be expanded, into the collective grid services that manage distributed state information, the common grid services that provide fundamental capabilities, and the grid fabric that represents the resources that are available within the sites that are linked by the grid environment. These capabilities are shown in Figure 2, taken from the characterizations published by William Johnston in the Grid Protocol Architecture (GPA) working group. Figure 2. Grid Infrastructure Components For the NVO testbed, both data management and collection management services are needed. In Figure 2., these services are encapsulated in the boxes at the grid services layer under the title ?Data Cataloging?. 4. NVO Testbed The NVO has implemented multiple services that integrate web-based access with grid-based computation. It is quite interesting to note that some of the web interactivity requirements are actually enabled by grid computing infrastructure. An example is support for image cut-outs on the DPOSS sky survey. Each image within the collection is about 1-GB in size. A web service that supports generation of image cut-outs would retrieve the appropriate image, process the image to reduce the size to the cut-out desired by the researcher, and then transmit the reduced image over the web. This process required the transmission of the 1-GB image file to the processing server. To create an interactive service, the ability to generate the image cut-out needed to be applied directly at the storage repository where the image collection was stored. Data grid technology provides latency management functions, such as the ability to aggregate I/O commands into remote proxies that can be executed directly on the storage repository under data grid control. For the cut-out service, the associated data subsetting commands were encapsulated into a remote proxy that was then invoked when a image cut-out request was made. This eliminated the need to move the entire file, greatly speeding up the interactive service. A second example shows the integration of image processing pipelines with grid technology. Image processing pipelines are typically based on a hardware system that has a maximum processing capacity. Grids provide a way to redirect processing requests onto alternate platforms. For jobs that are being queued through a batch system, this integration is straightforward. The batch jobs are queued onto the remote system where processing capability exists to handle the request. This approach puts additional data handling demands upon the grid. The movement of the batch job should not require any changes to data access mechanisms within the batch job. This implies the use of grid infrastructure to provide: * Logical name space supporting global, persistent identifiers for sky survey image collections * Storage abstraction for accessing data stored on any of the storage repositories used within the grid * Replication of sky survey image collections onto resources near the grid computing resources to minimize network latency Again data grid infrastructure provides the needed capabilities. Logical name spaces are provided to map from the global persistent identifiers to the physical file names used to store the sky survey images. Common access APIs are provided that map from the storage repository protocol to the access mechanism preferred by the application. Replication is used to provide multiple sources for the sky survey images. The source for the images is chosen to provide the lowest latency access. Grid technologies also manage authentication and authorization across the multiple administration domains under which the original sky surveys are managed. Authentication is managed by use of an inter-realm authentication environment based upon PKI certificates. The Generic Security Service API is used to map from the PKI authentication environment to the local Unix, Kerberos, or DCE authentication environments. The major area of development for grid technologies is the integration of knowledge management environments to support discovery across multiple image collections that use disparate attributes. The types of knowledge include not only logical relationships that describe how semantic terms are related, but also procedural relationships that describe the processing steps that are applied to create derived data products such as image cut-outs, and structural relationships that describe data encoding formats. The ability to transform from one data encoding format to a standard data encoding format can be viewed as a set of structural relationships that can be imposed on an image or astronomy data set by a remote proxy. The characterization of data encoding format transformations as knowledge is a major area of research. In summary, the NVO testbed is composed of: * Portals that explicitly integrate web services to data resources and analysis procedures. Examples are cone search, VOTable catalog query, simple image access. * Process management systems that control data processing pipelines. The process management systems map from a description of the steps used to create a derived data product to the grid computational and data resources * Data access layer for application of methods on data models. The methods can be executed within the portals, or within the process management systems, or within the data grid as remote proxies. The data access layer encapsulates the knowledge needed to map from the data encoding format to the desired representation, and basically provides the support for transformative migrations of data. * Data grid for managing distributed collections. The persistent replication of data between multiple resources can be managed through a logical name space. * Persistent data caches for interactive access to sky survey image collections. Because of the rapid decrease in disk system prices, it is now feasible to keep 10-TB collections on line on disk farms. This makes it feasible to provide interactive access to data picking environments for the retrieval of arbitrary image cut-outs from a survey. * High performance disk caches for data analyses. The ability to process collections relies on the ability to keep data close to the computational resources. One goal of the NVO testbed is to support data intensive analyses at the rate of 3 GB/sec, or ten Terabytes per hour from high performance disk cache. * Computational grid. The distribution of image processing steps onto remote resources can be automated through grid technology. * Persistent archives. The original sky surveys are replicated across multiple platforms to guarantee that the collections will survive through all possible types of data loss. Data grids are used to provide infrastructure independence, meaning that the replicas can be managed across arbitrary choices for storage architecture or media. 5. Ontologies The NVO testbed can be viewed as a set of software infrastructure layers that enable the processing of images from all-sky surveys. A more aggressive viewpoint is to consider the NVO testbed as a set of hierarchical ontologies. Each level of the ontology hierarchy specifies a different set of naming conventions that are applied to the images as they are manipulated. The NVO testbed provides a way to map between the different naming conventions. The processing steps that are applied are shown in Table 2. Transformative Operation Ontology ? Name space Manage transformations between physical quantities Concept spaces ? the relationships used to describe transformations on physical quantities Global naming convention Data grid ? the logical name space that is used to create global persistent identifiers, and map to physical file names Organize discovered physical quantities Catalogs ? the attributes used to label the digital entities to support discovery, the semantic labels (UCDs) used to describe physical quantities. Turn bits into physical quantities Data model ? the encoding format (structure) of the digital entity, the semantic terms used to label the parts of the structure, and the processes (methods) that can transform the structure Table 2. Ontology Hierarchy in the NVO Testbed It is worth noting that there is a tight coupling between the data model and the transformations that can be applied to physical quantities. This represents one of the major challenges to the NVO testbed, as it implies one can choose to manage the knowledge required to do the transformations within the methods that are applied to the data, or within a concept space that is managed independently of the data. Within the grid community, both approaches are being pursued. The experiences of other disciplines in the use of grid technology to federate collections can help the NVO testbed resolve where knowledge should be managed. 6. Acknowledgements The characterization of the NVO architecture is being developed by Doug Tody (NRAO), Roy Williams (Caltech), John Good (IRSA), Bruce Berriman (IRSA), Ray Plante (NCSA), Eva Deelman (IAI/USC), and Carl Kesselman (ISI/USC). Implementations of the NVO architecture are being done by Roy Williams, Alex Szalay (JHU), John Good, George Kremenek (SDSC), and Ray Plante. This project is supported by the NSF cooperative agreement on the National Virtual Observatory AST0122449.