Installation guide for GADS version 0.1

Introduction

This is an installation guide for GADS (Grid Access Data Service). It describes how to install GADS under a Tomcat server using Apache Axis. See the GadsUserGuide010 for information about using GADS once it is installed. Please report any questions to JonBlower.

System Requirements

At the moment, GADS is only known to work under Linux (and has only been tested under Red Hat 9). I believe it should work under Solaris out of the box too, and under Windows with minor modifications (which are not yet documented). In order to install GADS, you will need:

• Apache Tomcat (4.1.24 has been tested)
• Apache Axis (1.0 has been tested) (latest GADS, not released yet, supports Axis 1.1)
• Apache Ant (version 1.5.3 has been tested)
• Java NetCDF core library (from http://www.unidata.ucar.edu/packages/netcdf-java/)
• Java HDF/HDF5 libraries (from http://hdf.ncsa.uiuc.edu/hdf-java-html)

Installation steps

Installation consists of three steps:

1. Prepare and start the application server
2. Compile and install GADS
3. Configure GADS

1. Prepare the application server

• Install Apache Tomcat (only version 4.1.24 has been tested but others should work too)
• Install Apache Axis in the Tomcat server (only version 1.0 of Axis has been tested)
• Set the environment variable $CATALINA_HOME to the root of the Tomcat installation.
• Start the Tomcat server (run $CATALINA_HOME/bin/startup.sh)
• Verify that both Tomcat and Axis are installed correctly by pointing your browser at http://<host>:<port>/axis/happyaxis.jsp and checking for error messages. (N.B. The default port number for Tomcat is 8080.)

Don't stop the Tomcat server: it needs to be running in order to install GADS.

Note:

There are a couple of things you may need to do to Tomcat to get GADS to work properly:

• Increase the memory available, to prevent the notorious OutOfMemoryErrors .
• Tell Tomcat where to find libraries of native code. Many libraries for encoding/decoding file formats (e.g. HDF) are not 100% pure Java and so rely on shared libraries.

Both these birds can be killed with one stone: Add the following line to $CATALINA_HOME/bin/catalina.sh (make it the first executable line in the file, below the first comment block):

JAVA_OPTS="-Xms512M -Xmx512M -Djava.library.path=$CATALINA_HOME/webapps/axis/WEB-INF/lib/native"

This increases the memory available to Tomcat to half a gigabyte. You may wish to change these figures, depending on the amount of RAM your server has. This line also tells Tomcat that you have put your native libaries in a folder called $CATALINA_HOME/webapps/axis/WEB-INF/lib/native.

2. Compile and install GADS

• Get the 0.1.0 release package of the server code from here.
• Create a temporary directory called, for example, ~/gads_temp.
• Extract the .tar.gz file into this directory, and cd into it.
• Edit the build.xml file to suit your system. Not much needs to be done here, just make sure the properties called axis-lib.dir, netcdf-lib, hdf5-lib.dir, tomcat-dir, tomcat-host and tomcat-port have the correct values.
• Run ant install to compile and install the source. If there are any problems, check the error messages, make sure you have set all the properties in build.xml correctly and make sure that Tomcat is still running.
• When you're satisfied that everything is working, you may delete ~/gads_temp (you may wish to leave this until you've configured everything and done some testing).

3. Edit configuration files

There are four configuration files that need editing (by hand, unfortunately ) before GADS can be tested. If installation went OK, they should all be in the directory $CATALINA_HOME/webapps/axis/WEB-INF/classes. Note that if you change any of these files you may have to restart Tomcat! Let's start with the easy ones first:

log4j.properties

This controls the behaviour of the logging system. The only properties that need changing are log4j.appender.logfile.file, log4j.appender.requestlogfile.file and log4j.appender.querylogfile.file. Change the values of these properties to set the locations of the log files.

gads.properties

This sets some global properties for GADS. Change the following properties:

• CacheDirectory: Set this to the path to the directory which will be used to store extracted data, ready for download. It must be visible under an HTTP server (see the next property)
• HttpPrefix: Set this to the correct URL prefix for accessing the cache directory over HTTP. (Actually, if the server is an FTP server this could be changed to an ftp:// URL).
• UsersXMLFileName: Set this to the full path of the XML file used to store user information (see below).
• MetadataXMLFileName: Set this to the full path of the XML file used to store metadata information (see below).

users.xml

This contains information about the users of the system, and restricts access to certain users. The file already contains some examples, and has an inline DTD for validation. The structure should be pretty self-explanatory apart from the following points:

• Passwords are stored unencrypted. In this early release of GADS, security has not been addressed very seriously. Therefore, users must not use sensitive passwords. (Better encryption will follow in future versions.)
• The <level> tags are used to control access to datasets. They correspond with the accessRequired attributes for datasets in the metadata file (see below). Users can have multiple <level>s. The root user always has access to all datasets.
• The file includes an anonymous user. Delete this user if you do not wish to allow anonymous access.

metadata.xml

This contains information about the data being served by GADS. Basically, it consists of a set of <dataset>s, each of which are described as detailed here (again, there are examples in the file in the distribution, as well as an inline DTD for validation):

• Each <dataset> must have a name, and a date on which it was last modified (actually, the date of last modification is not used much in the system). Optionally, you may add an accessRequired attribute to limit access to people with certain privileges (see the users.xml file described above).
• Each dataset contains a set of <variable>s. Each variable has a name (this is the externally-visible, "public" name), a datatype (currently only float is supported), units and an internalName, which is the name that identifies the variable within the data files themselves.
• Each variable sits on a <grid> (this is another attribute of the <variable> tag). A <grid> consists of a set of <axis> tags. Of course, many variables may share the same grid. You may define as many grids as are needed (for example, temperature and velocity data may sit on different grids; furthermore, some variables are four-dimensional whereas others are three-dimensional).
• Each <axis> in a grid corresponds to a <coord_axis>, which describes where the "ticks" along the axis lie. A <coord_axis> has a name, units, and may be regular or irregular=. Regular axes have evenly-spaced "ticks" and are described by start-stride-count semantics. Irregular axes need to have the position of each "tick" explicitly specified. Generally, longitude and latitude (x and y) axes are regular, whereas depth and (usually) time axes are irregular. Depth axes have an additional attribute describing which direction (up or down) is taken to be positive.
• Finally, each dataset consists (physically) of a set of <datafile>s. Each file is assumed to contain all the data for a given time (this assumption will be relaxed in future versions of GADS). Each set of <datafile>s has a root directory and a format. Each individual data file has a location (i.e. its position relative to the root directory) and a time, which corresponds to the entries for the time axis.

You will need to edit this file to match your data holdings. This task looks complicated at first, but hopefully with the above guidance and the examples in the file, it should not be too hard. (We are hoping to create a metadata management tool to make this job rather easier.)

Testing the installation with a simple client

Download the client software from here. Extract the archive into a new directory. To run the client type:

java -cp $AXISCLASSPATH uk.ac.nercessc.gads.client.GADSClient <url to GADS Service> <method name> [arguments]

where $AXISCLASSPATH contains all the .jar files of the Axis libraries. So to test that you can reach the service, call the getInfo method (in this example GADS is running on the localhost):

java -cp $AXISCLASSPATH uk.ac.nercessc.gads.client.GADSClient http://localhost:8080/axis/services/GADS getInfo

To see the list of arguments required for dataQuery and dataRequest, call these methods with no arguments: you will see a message giving the syntax of the call.

Troubleshooting

If, when running the client, you get a ClassNotFoundException, make sure that the third-party libraries (e.g. netCDF, HDF libraries) are in Tomcat's classpath. The easiest way to do this is to copy the jar files to $CATALINA_HOME/webapps/axis/WEB-INF/lib and restart Tomcat. If the libraries use native code (as is the case with the HDF libraries) make sure the necessary shared libraries (.so files) are in your $LD_LIBRARY_PATH. (TODO: add this to ant install script?)

Adding/removing supported file formats

The current distribution of GADS is capable of reading NetCDF and HDF5 files (next version: GRIB) and outputting NetCDF files and raw data. More formats will be supported in future releases.

This section of the guide is incomplete. Please come back later or contact JonBlower with any queries.

-- JonBlower - 20 Feb 2004