myLead metadata catalog

























































































































































































































































































































































































































































MyLEAD is a personal metadata catalog for storing and searching widely distributed, terascale, federated scientific data products. The metadata catalog service plugs into the LEAD service oriented architecture (SOA). It is built on top of the UK OGSA-DAI OGSI R6, Globus Toolkit 3.2.1, and uses mySQL 5.0.18. Larger data products are stored to a local file system managed by the Globus GT4.0 DRS.

Data products are stored and retrieved as XML documents defined by the LEAD Metadata Schema (LMS). The LMS is a profile of the Federal Geographic Data Committee (FGDC) metadata standard, and as such it extends and specializes the FGDC standard. XML documents are stored in the relational database by a hybrid technique that combines shredding and selective use of CLOBS (character blobs). The technique is described in "A Hybrid XML-Relational Grid Metadata Catalog," February 2006.

A major strength of the myLEAD metadata catalog is that it supports attribute extension. New attributes or annotations to a data product can be added on the fly to the database. Thus there is very little of the relational schema that is specific to the meteorology domain. The LMS has this same extensibility property, smoothing its adoption and adoption of the metadata catalog by other scientific or non-scientific domains.

The project's next steps are 1.) porting myLEAD to OGSA-DAI WSRF 2.1, 2.) deploying a hardened myLEAD in a distributed configuration on the LEAD grid, and 3.) performance optimizations to minimize the number of copies of a data product that must exist.

For help with this release, please contact Yiming Sun (yimsun -at- cs.indiana.edu). Additional questions and comments can be addressed to the project director, Professor Beth Plale (plale -at- cs.indiana.edu).

Release V1.3

Source Code

Documentation

Release V1.2

Source Code


Documentation

Java Docs

Release V0.3alpha

Source Code

Documentation

Java Docs

Constraints of myLEAD v0.3alpha release

1.) XML schema used for adding a metadata record and retrieving results: the "LEAD metadata schema" is not supported. Currently supported schema is AHM05 schema. Will change when the LEAD metadata schema becomes available.

2.) myLEAD and Globus RLS: MyLEAD provides guarantees over the data products it 'manages'. It uses Globus RLS as the service interface to the repository where data prodcts themselves exist. The current approach to adding a new data product to myLEAD:


This lighter-weight solution will give us some interoperability with RLS in release v0.3alpha while we work out the optimal tradeoff between autonomy of the two components and atomicity, and put in place a storage container.

3.) Replication, publishing, versioning, and sharing are not supported features in the v0.3alpha version of the release.

Select Publications

See DDE lab publications page for full list.