OGSA-DAI Evaluation

See http://www.ogsadai.org.uk/.

These notes are a result of an Access Grid meeting with Kostas Tourlas of the OGSA-DAI team (with JonBlower, AditSantokhee, ChunLeiLiu and KeithHaines). The purpose of the meeting was to try to establish whether it is worth looking at OGSA-DAI to assist with the ESSC's data serving interests. Kostas' PowerPoint slides are here.

We discussed this particularly with regard to the DEWS project, within which ESSC will probably implement a Web Coverage Server (spec, PDF) for serving geospatial data. One apparent limitation of WCS for this purpose is that the WCS assumes that data can be delivered synchronously, as a direct reply to the query (in XML or as a MIME attachment). This may not be the case for large queries: we probably want some other data delivery mechanism.

Potential benefits.

There are a few features of OGSA-DAI that might be of benefit to us:

Data delivery

OGSA-DAI uses Web Services as the "control channel" for handling queries, but allows a separate channel (e.g. FTP, GridFTP, HTTP) to be used as the delivery channel. For example OGSA-DAI supports the activities deliverToURL and deliverFromURL that can be used as follows: A client makes a request to extract data, then pipe the output to the deliverToURL activity. The server replies with the URL at which the data are available. The client then makes a request using the deliverFromURL activity to get the data (and might also be able to simply download from the URL - Kostas is checking this).

OGSA-DAI does not currently support SOAP attachments, however, future support of that feature is being considered based on an evaluation of available third party code.

Asynchronous behaviour

OGSA-DAI is moving strongly in the direction of WSRF. It is making as much of its architecture as possible asynchronous; in other words, removing the assumption that data will be delivered down the same channel as the data request (a poor assumption for large data extraction tasks). As well as the above-described data delivery scenario, OGSA-DAI clients can request to be notified when the data are ready. This notification could be a message sent to the client (requiring the client to run a server program to receive the message) or via another mechanism such as email.

An OGSA-DAI client may poll a service to receive status updates (may be useful if clients are behind firewalls).

Security

OGSA-DAI provides message-level security via X.509 certificates.

Federation

If our data are stored in a mixture of flat-files and a database, OGSA-DAI can make these two data sources look like a single one from the point of view of the client. OGSA-DAI can make these two data sources uniformly accessible through the same client application.

More notes

1. Major point to note is that OGSA-DAI is flexible and extensible. So, with effort, we could fill in any functionality that is missing.
2. OGSA-DAI currently supports OGSI, WS-I and WSRF "flavours". Support for OGSI will soon disappear (although it will remain in the software as legacy code).
3. There is a client toolkit library (Java) so XML messages do not have to be hand-crafted.
4. There will be a new release (version 6) of OGSA-DAI at the end of May 2005. This is not a "production release": it is intended to give developers a flavour of what is to come in terms of WSRF orientation and to help them migrate from OGSI-based software.
5. The next production release (version 7) will be out in October 2005. In this release, all the API will be fully specced out (even if all functions are not yet implemented). Version 7 will incorporate Distributed Query Processing.
6. The OGSA-DAI team can help us get started (we would be considered "advanced users").
7. Kostas estimates that "a few weeks" of effort is required to get a prototype system based on OGSA-DAI up and running.
8. OGSA-DAI is written in pure Java and so should be portable: however it has been tested most extensively under Windows and Linux (Red Hat).
9. There wil be a publicity meeting (suitable for managers of new projects: not technically-focussed) around July 18th. This will also include participation from the NGS.

Questions

So, is it worth expending some effort in seeing if we can use OGSA-DAI in ESSC's activities, and in the DEWS project in particular?

1. How could we organise our architecture such that we maintain maximum compatibility with the WCS specification, whilst dealing with WCS's limitations such as synchronous data delivery?
2. What if we (in DEWS) build OGSA-DAI into the system and then find after 2 years that the Met Office does not want to use OGSA-DAI in its systems? Can we loosely couple the OGSA-DAI components from the rest of the system so that OGSA-DAI can be removed if necessary?
3. Given that OGSA-DAI has a wide, international user base, can we encourage OGC to "gridify" their specifications by incorporating OGSA-DAI features?
4. Is the emphasis on WSRF a good or bad thing for us? How much can we do with plain Web Services? Would OGSA-DAI's WS-I version be better for us?
5. What is the risk of OGSA-DAI finishing as a funded project and leaving DEWS unsupported? Support activities are funded until this time (May) next year, but funding for development activities is less clear. Hopefully its inclusion in the Globus Toolkit will increase its chances of longevity.

-- JonBlower - 12 May 2005