The SGS server namespace

All resources on any kind of Styx server are represented as a set of files in a hierarchical structure (see the JStyx tutorial). This file hierarchy is known as a namespace. Styx Grid Services are no different. When an executable is exposed as a Styx Grid Service, a namespace is created. Clients interact with the SGS by reading from and writing to files in this namespace.

This page describes the namespace of a general Styx Grid Service. This information is intended for developers who want to know what is going on behind the scenes in the SGS system, perhaps in order to develop custom clients or add new features. Knowledge of the SGS namespace is not at all necessary for most users.

Namespace description

The diagram below shows the namespace of a general Styx Grid Services server. The hyperlinks will take you to a detailed explanation of the purpose of each file and directory. Note that the structure of the namespace mirrors closely the structure of the XML configuration file, which is described here. 

    /
    |-- mySGS/
    |   |
    |   |-- clone
    |   |-- config
    |   |
    |   |-- docs/
    |   |   |-- description
    |   |   `-- readme.txt
    |   |
    |   `-- instances/
    |       |-- 0/
    |       |    |-- ctl
    |       |    |-- args
    |       |    |-- params/
    |       |    |   |-- param1
    |       |    |   `-- param2
    |       |    |-- inputs/
    |       |    |   |-- stdin
    |       |    |   `-- myinputfile
    |       |    |-- outputs/
    |       |    |   |-- stdout
    |       |    |   |-- stderr
    |       |    |   `-- myoutputfile
    |       |    |-- serviceData/
    |       |    |   |-- status
    |       |    |   |-- exitCode
    |       |    |   `-- customSDE
    |       |    |-- steering/
    |       |    |   `-- steerable1
    |       |    `-- time/
    |       |        |-- currentTime
    |       |        |-- creationTime
    |       |        `-- terminationTime
    |       `-- 1/
    |
    `-- mySGS2/

Server files

The files in this section belong to the server as a whole, rather than to a particular Styx Grid Service.

The root directory

The slash represents the root directory of the namespace on the server.

Styx Grid Services

Directories immediately beneath the root of the server are the roots of Styx Grid Services. The name of the directory is the name of the Styx Grid Service in the config file.

Styx Grid Service files

The files in this section are "global" to a particular Styx Grid Service: they do not belong to a specific instance of the Service.

The clone file

The clone file is used to create a new SGS instance. When a client reads from this file, a new SGS instance is created and the URL to the root of this instance is returned to the client. Note that the instance might be created on a different server for load-balancing reasons, which is why a full URL is returned (however, in the current release, instances are always created on the same server). Writes to this file are not allowed.

The config file

When a client reads from this file, an XML string will be returned, representing the configuration of the Styx Grid Service in question. This XML string is very close to the XML configuration of the Styx Grid Service, except that the <docs> tag does not appear, nor does the "command" attribute of the root <gridservice> tag. Neither of these things are relevant to the client.

The docs directory

This directory contains all the documentation files that the server has chosen to expose for the benefit of the client. These files can be in any format and they are here to provide more information to the client about the use of the SGS in question.

The description file in this directory is present in all Styx Grid Services. It is a short description of the purpose of the SGS, and is gleaned from the "description" attribute of the XML configuration file. The readme.txt file shown in the above diagram is simply an example of a documentation file and is not present in all SGSs. Note that the docs/ directory can contain any number of files and subdirectories.

The instances directory

The instances/ directory contains a set of directories, one for each instance of this particular Styx Grid Service. The name of the directory is the unique ID of the instance. The diagram above shows two instances for the mySGS service, with IDs 0 and 1.

Styx Grid Service instance files

The files in this section belong to a particular instance of a Styx Grid Service.

The ctl file

This is a very important file: it is used to control the Styx Grid Service ("ctl" is short for "control"). When all parameters have been set and all input files have been uploaded, the client starts the service running by writing the string "start" into this file. The service can be stopped at any time by writing the string "stop" into this file. The instance can be destroyed and its resources freed by writing the string "destroy" into this file.

The arguments file

The args file is used to read and write the full set of command-line arguments that will be passed to the underlying executable. Reading this file is very useful for debugging purposes.

The parameters directory

The params/ directory contains a set of files, one for each parameter that can be set by clients. There is one parameter file for each parameter in the configuration file, except for parameters that represent output files. Clients set parameters by writing values into these files. The parameters are translated into command-line arguments to be passed to the underlying executable. With each write, the server will check the validity of the input, returning an error if the parameter value was not valid. These files can be read to get the parameter values.

The inputs directory

The inputs/ directory contains files to which clients can write input data. If the "stdin" file is present then this means that the underlying executable is expecting data to be passed to its standard input. Clients can write data to this file while the service is running in order to stream data to the standard input.

The other files in this directory represent input files that will be read by the underlying executable. These files must be uploaded before the service is started. Some input files have fixed names: these are always present in the namespace. Other input files are specified by the value of a certain parameter (see the configuration instructions). In these cases, the corresponding file will not appear in the namespace until the value of that parameter is set. Note that several filenames can be written to the parameter (separated by spaces), in which case a file will appear in the namespace for each filename that is written.

The outputs directory

The outputs/ directory contains a file for each output file or stream that is produced by the underlying executable. The files called "stdout" and "stderr" are read to obtain data from the standard output and standard error of the executable. Other files represent output files that are produced by the executable.

The service data directory

The serviceData/ directory contains files, one for each element of service data that the service exposes. "Service data" is data about the state of the service instance.

In the above diagram there are three elements of service data shown. The first two service data elements (SDEs) are present in all Styx Grid Services. The "status" SDE can be read to find out the status ("created", "running", "finished", "aborted" or "error") of an SGS instance. If a client reads from the "exitCode" file, no data will be returned (i.e. the read will block) until the underlying executable has finished running, at which point the exit code from the executable will be returned. The "customSDE" element is a user-specified service data element (see here for details of how this is done). User-specified SDEs might be used, for example, to provide a way to monitor the progress of an executable in a finer-grained way than is possible with the "status" SDE.

Files that represent service data elements exhibit blocking behaviour. When a client reads from the file from the first time, the data are returned immediately (except in the case of the exitCode file as described above). If the same client reads from the file again without closing the file, the read request will block until the underlying data change, at which point the data are returned to the client. This is how clients can receive asynchronous messages without the need for callbacks.

The steering directory

In some cases, programs permit some parameters to be adjusted while the executable is running. This is known as computational steering. The steering/ directory contains the files that are used to do this, but this is considered an advanced feature and will not be discussed here (yet).

The time directory

The time/ directory contains files that are pertinent to the lifecycle of the SGS instance. The currentTime/ file can be read to give the current time according to the SGS server. The creationTime file can be read to give the time at which the instance was created. The terminationTime file contains the time at which the service instance will be automatically terminated. When the instance is first created it is set to never terminate (the terminationTime file will be empty). Clients can set a lifetime for the instance by writing a termination time into this file. In order to terminate the instance immediately, clients should write the string "destroy" into the ctl file (it is not legal to write a time in the past into the terminationTime file).

All times are read and written in the xsd:dateTime format, for example "2006-01-23T17:34:56+00:00".