Details of Iteration 4

Final DEMO Stage

Contents

1. General Description
2. Server Setup
3. Client Setup
4. Poetry data

General Description

This last stage in the Text Track Project is essentially a persistent version of the previous BETA version of the project. All the iterations performed so far have been used with main-memory only databases, so the size of the database, which imposed two restrictions:

• The size of the database is restricted to whatever could fit in the main memory,
• The database needs to be recreated every time the program runs, since the main memory databases are not generally constant in nature.

The persistence of the database is achieved through the use of the E Programming Language, developed in Wisconsin, as a part of the Exodus Project. The original program is converted from a main-memory based database to persistent database by converting the class declarations for the Elements and Attributes to special persistent dbclass declarations in E, and using E's transaction processing syntax for accessing and modifying the persistent data. E was chosen as opposed to the normal Exodus structure because of E's high level syntax and easy migration capabilities from a non-persistent program to a persistent program.

Server Setup

The server is set up using the Exodus 3.0 storage manager. The version of E that is installed presently did not seem to be compatible with Exodus 3.1, so after many futile attempts of trying to run the examples, we decided to move back to Exodus 3.0. Setting up the server consists essentially of creating a data and log volume, and running the server on the machine specified in the server configuration file. Details of how to set up the server can be obtained from the Exodus Users' Manual.

Client Setup

The client in our case is simply the set of executable programs in this project. In this iteration, we have two executables:

1. Parse: This is the parser which uses sgmls to parse the raw SGML data, and builds the index based on the output from the Parser. This program also starts the interpreter and lets the user input queries after parsing and building of the index is completed. The syntax for running parse is:

   % parse [SGML-filename]

   The SGML filename, as before in the previous iteration, should have the DOCTYPE information with the full pathname of the DTD. If the filename is not specified, the program will automatically use ~asengupt/mmdb/dtd/starwars.sgm.
2. Query: This is the interpreter, which uses the index built by the parser program, and lets the user enter query strings. This program is run with no command-line parameters.

Both programs are located in /u/asengupt/mmdb/exodus/demo on cs machines.

Poetry Data

For use with more sophisticated demos, some subsets of poetry data can be used. One such subset is kept in /u/asengupt/mmdb/test/poetries.sgm, and another subset, with more than 85000 elements, is in /u/asengupt/temp/poetry/wrdswrth/ww.sgm. This data contains all the works of William Wordsworth, and the index is already built on megamouth. The total index building time was about half an hour - and the total data volume size dumped by Exodus was about 100 MB.