Calder:data stream access middleware








IU's Calder. Peau Rouge Indiana, a stabile by sculptor Alexander Calder, enhances the entrance to the Musical Arts Center at IU, Bloomington.


































































Web service access to data streams for data driven applications

The use of real-time streaming data in scientific computations is growing in prevalence as greater amounts of data are generated by sensors and instruments and more systems are monitored in real time.

This project enables data driven grid application services to retrieve and operate on data flowing in real time from a data stream. Our prototype continuous query grid service, Calder, enables application web services to submit long running continuously executing queries to the system.

A query can aggregate, filter, and transform one or more data streams on behalf of the application, generating a new stream tailored to the needs of the application service. Calder buffers the resulting stream to, enabling temporal synchronization between the stream and the application service.

The Calder system operates over a realistic stream load generated by a computational science application so provides a realistic framework in which to investigate a number of timely research issues in stream query processing:

Calder extends dQUOB (http://www.cs.indiana.edu/dde/projects/dquob.html). dQUOB v1.0, which is available for release, includes the stream processing system components of Calder.

Calder Architecture

Calder has two subsystems:

1. Data management subsystem - comprising of the grid data service, planning service and the rowset service. A registry service and provenance service are currently under development.

2. Query processing subsystem or the Query Processing Engine (QPE).

Base Calder functionality for query planning and distribution, discovery, service interface, and optimization will be available as part of Calderv1.0 and beyond. Calder v1.0 is slated for release December 2005.

Name Reasoning

'Calder' is an old English word with Celtic/Gaelic origins meaning 'streams'.

Publications

Contact Us

Beth Plale , Ying Liu , Nithya Vijayakumar