Jump Over Left Menu
Issue 32: June 1994
Frontcover shows the IBM 3494 tape library dataserver, showing the robot and two of the four drives.
Atlas Data Store
There has been much activity with the Atlas Data Store since the article in FLAGSHIP 30.
The IBM 3494 tape library dataserver arrived punctually on 15 February and testing started later that week. Some parts had been delayed at Frankfurt airport, and the installation was completed on 21 February.
The IBM 3494 was integrated into the Atlas Data Store and began to be used for "real" data in the middle of March. The device now holds about 2 Tbytes of data. In a week there are typically 10,000 tape mounts and over 1 Tbytes of data are read or written.
Since the installation was completed, the device has not lost or corrupted any data (we can assert this with some confidence because every time data is read from the robot, the Atlas Data Store tape-reading application checks the redundancy check data which it wrote with the data), and there have been no failures which have required an engineer to visit before service could be restored.
There have been some minor problems, most of which occurred only in the first few weeks of operation and have now been fixed. Of the fourteen calendar weeks since installation, seven recent weeks have been completely free of any problem requiring manual intervention. The reliability of the device has been very good, and compares favourably with the other older automatic tape robots that have been installed at the Atlas Centre.
In order to improve the performance and throughput of the Atlas Data Store, in May we replaced the disk I/O software layer used by the Atlas Data Store on the IBM RS/6000 workstations which drive the 3494 tape drives and the network. The new I/O software can drive the disks (Seagate elite-3 devices) at their full potential - between 4.5 Mbytes/sec (data near the outer edge of the disk) and 3.5 Mbytes/sec (data near the spindle) and has a smaller CPU requirement (which leaves more power available for driving the network). This enables the four tape drives in the robot to be used at almost their full speed, sufficiently fast that the time spent transferring data is generally less than the time spent mounting, positioning, rewinding and unloading the tapes. On average the system transfers data at about 2.0 Mbytes/sec in each of the four drives simultaneously, which approximately trebles the data throughput potential of the system compared to the situation before the IBM 3494 arrived.
Higher data rates to applications using data in the data store are also possible. We have measured data rates between a client application connected across the FDDI network to the Atlas Data Store of 4.19 Mbytes/sec (reading) and 2.38 Mbytes/sec (writing). We have seen real user applications in real life reading data at over 2.25 Mbytes/sec. The data rates can of course be limited by the power of the client machine, especially if the data is to be read or written in its filestore.
Another vital component of the Atlas Data Store is the tape command which is used on Unix and VMS client machines to access data in the store. There have been a number of enhancements made to this recently.
Unix and VMS versions of this are available for a wide range of hardware and operating systems. The program is available for use on machines at RAL and machines further afield; there are already several remote applications using the Atlas Data Store.
Version 3.3 of the tape command is just being installed on a number of CCD service machines. This version includes the following enhancements:
- If the tape is not available for use on the first attempt, the reason for the delay will be reported and then no more messages will be issued until either the tape is ready for use or the transfer attempt times-out.
- When writing to the tape, the user may specify that the new file is to be written after the last file, without knowing how many files are already on the tape.
- The tape record format Variable Blocked and Spanned (VBS) is now supported for both reading and writing.
- By default, the tape command will stop writing data to the tape and finish Standard Label (SL) / ANSI Label (AL) tapes with End of Volume (EOV) labels if data is written past the silver paper mark.
- When writing to a tape, the silver paper mark can be ignored. If data writing stops before the physical end of the tape, SL/AL tapes will have End of File (EOF) labels written. If data writing continues past the physical end of the tape, the data that fits onto the tape will be saved on the virtual tape image (rather than losing it as previously).
- The default wait time is now 24 hours. This has been increased from the default of 1 hour as jobs were failing merely because of a long queue for the tape robot or if the robot was out of action for a few hours.
- When reading a tape, a particular file may be accessed by its data set name (SL and AL tapes only) as well as the file number.
- The default labels type when reading is "any", that is the labels type of the tape being read, this removes the requirement to specify the tape type for NL and AL tapes.
- The tape record format may be specified which will override any value in SL and AL tape headers as well as specifying the value to use for NL tapes.
- The Unix version of the tape command provides a queue-file feature. The queue-file is a list of commands that allow a number of tapes to be processed in one job (for example, the application may read 20 tapes into one file or down one named pipe). This facility will be extended into the VMS version.
In addition to the changes to the tape command, the underlying libraries providing access to the tape data have been enhanced. Anyone using old versions of the LIBVTP and LIBREC libraries are asked to upgrade to the new libraries.
Tim Kidd, David Rigby Systems Group
Compilers for Fortran 90
Optimizing compilers have begun to appear for Fortran 90 and it seems clear that Fortran 90 will replace Fortran 77 over the next few years. The main advantages of Fortran 90 are summarized in the Figure. The purpose of this article is to tell you about the compilers currently available.
The NAG compiler has been out for over two years now and is a mature product. I use it on my three-day courses at the Atlas Centre. The students use it freely throughout all three afternoons and therefore give it quite a severe test. The execution speed of the compiled code has improved, too, although it should be appreciated that it is designed as a debugging compiler, rather than an optimizing compiler. We have a site licence for Unix and VAX/VMS.! It is mounted on UNIXFE and the Alpha OSF/1 farm. The command name is f90 and man pages are available. In collaboration with Salford University, NAG markets a version for PCs. NAG has made a version freely available as a verifier on World Wide Web.
he Cray compiler was released at the beginning of 1994 and is mounted on the Y-MP at the Atlas Centre. It is an optimizing compiler. Cray reports that currently it runs typically 6-10% slower than cf77 on Fortran 77 programs. In the long term, any slower execution of a Fortran 77 program will be treated as a bug to be corrected, the Fortran 77 compiler will be frozen and development will be concentrated on the Fortran 90 compiler. My experience with this compiler has been happy, though I have found a very small number of bugs. It is not as friendly as the NAG compiler for accessing modules in different files. NAG simply looks in the current directory and gives a warning if the source for the module has changed since it was last compiled. Cray requires every such file to be explicitly mentioned with the -p option on the command line. The debugging features are comparable with those of cf77. Cray plans to market versions for the workstation and PC markets.
The Edinburgh Portable Compilers (EPC) compiler was released at the beginning of 1994 and is mounted on my Sun workstation. It has all the features that I have grown to love with the EPC Fortran 77 compiler: fast compilation, fast execution, optional checks for subscript errors, optional checks for undefined variables, optional checks for invalid correspondences between actual and dummy arguments, optional very-readable dumps (the value of every variable of the program unit in which the program stopped is printed alongside its name and you can choose how many elements of arrays are printed). The Fortran 90 compiler comes with optional checks on the shapes of array arguments to intrinsic procedures and optional checks on references to pointers and allocatable arrays. I have found a very small number of bugs in the current release, but I anticipate that this will be the workhorse that I will use for development of Fortran 90 software.
The latest release of IBM's xlf compiler for RS/6000 systems is a Fortran 90 compiler. I do not have any personal experience of this compiler, butunderstand that its quality is, again, good. There are plans to mount it on UNIXFE.
We have a beta-test version of Digital's Fortran 90 compiler mounted on an Atlas alpha. One expects a beta-test system not to be quite perfect and indeed I have found some bugs, but I understand that they have all been corrected. It promises to be a reliable product, and will support HPF (High Performance Fortran) for data-distributed programming on farms.
To summarize, the early releases of Fortran 90 compilers are of high quality. It is time to start using (and enjoying) the new facilities. Since Fortran 90 is upwards compatible with Fortran 77, you can start by treating the Fortran 90 compilers as new Fortran 77 compilers.
The main advantages of Fortran 90 over Fortran 77
- Dynamic storage allocation.
- Operations on whole arrays and many intrinsic procedures for arrays.
- User-defined derived data types composed of arbitrary data structures and operations upon those structures.
- The ability to write internal procedures and recursive procedures, and to call procedures with optional and keyword arguments.
- Parameterization of the intrinsic types, to permit processors to support short integers, very large character sets, more than two precisions for real and complex, and packed logicals.
- Facilities for defining collections called 'modules', useful for global data definitions and for procedure libraries.
- A source form that is not oriented towards punched cards.
- New control constructs such as the SELECT CASE construct and a label-free form of the DO.
- Improvements to the input-output facilities.
- Many new intrinsic procedures.
- Requirements on a compiler to detect the use of constructs that do not conform to the syntax of the language or are obsolescent.
John Reid, Numerical Analysis Group Advanced Research Computing Unit
For X-terminal or PC users, accessing World Wide Web could not be easier than with the browser program Mosaic. Some people consider Mosaic as their graphical interface to the Internet.
World Wide Web (WWW)
Since the concept of World Wide Web has already been discussed in an article in FLAGSHIP 28, I will say very little about it and concentrate on Mosaic which is one of the many user interfaces to WWW. WWW (sometimes called W3) is a distributed hypermedia system for sharing information globally. To access the web (another term for WWW) you need a user interface known as a browser. The browser can then be used to access hypertext objects available on the Internet. An object in this sense can be one of many things: a document, a picture, a service (ftp, gopher) or some sort of resource on the Internet. The objects may in turn have references to many other objects anywhere on the Internet. Different objects are accessed by a Uniform Resource Locator (URL) which is an address that uniquely locates an object on the Internet. URLs are the key to locating documents; because of this the term will come up frequently when talking about WWW.
Mosaic is one of many WWW browsers available. It was developed and is now maintained by the National Center for Supercomputing Applications (NCSA) at the University of Illinois. The reason it has been chosen as the feature of this article is because it is a particularly good browser. It provides a very simple to use interface to the wealth of WWW accessible information. It also works on many different platforms.
There are three flavours of Mosaic browser: an Xll one using a Motif style interface, a MS-Windows one for PCs and an Apple Mac one. They are similar; they have the same function but each user interface has a slightly different look and feel to it. In CCD we have Mosaic running on the OSF/1 farm and on UNIXFE. Pre-compiled binaries are available for several other platforms. If you want to run a browser on your own workstation then please contact User Support; we will try and find a binary for your platform.
Starting and using Mosaic is very simple. Generally all you need to do is run the command mosaic. If you have problems with this there are a few things to consider. Mosaic requires an X-display in which to start its window. As with most X tools you need to ensure your DISPLAY environment variable is set to point to a valid X-terminal display or you need to specify the display name on the command line with -display <display-name>. Further technical information can be found in the man page and also on the Web.
Mosaic Home Page
The home page is the page Mosaic displays when it starts up. On UNIXFE and the OSF/1 Farm we have set the default home page to http://www.rl.ac.uk/home.html by setting the environment variable WWW _HOME. You are able to set your own home page by resetting the WWW_HOME environment variable to a URL of your choice or by specifying -home option with a valid URL when you invoke Mosaic.
Useful Features of Mosaic
The best way to get a feel for what Mosaic can do is to try it. I am not going to describe how to drive Mosaic; the information I give here is just to point out some of the more useful features.
Selecting Pages: Once you have started Mosaic the easiest way to select pages is to use your mouse to click on anchors. Anchors are the name given to parts of a viewable page that are hypertext links to other pages on the Web. You can spot anchors because they are highlighted in a different colour and are underlined. If you place your mouse over an anchor the URL that the anchor refers to will be displayed at the bottom of the window.
If you know the URL of a Web page that you wish to read, you can tell Mosaic to go straight to it. To do this select Open URL... from the File pull down menu. You will then be prompted to supply a URL. Enter the URL then select the Open button. Mosaic will fetch the page you have requested.
List of Commonly Used Pages (Hotlist):
Mosaic has facilities to maintain a list of URLs that you can access with just a few clicks of your mouse. This feature is very useful for recording pages that you are likely to use often. You can access and maintain your hotlist from the Navigate menu.
Cancelling a Request: Versions of Mosaic before 2.0 did not have a facility to cancel a request. Now you can stop Mosaic when it is fetching a document by clicking your mouse pointer on the revolving globe in the top right hand corner. This is very useful if a request is taking unusually long to complete.
Figure 2. Some useful and interesting pages
Ivan Fabian, Applications & User Support Group
Video Facility Expanded
Completely new ways of handling video are now possible with the most recent extensions to the Atlas Video Facility. The original 2D hardware is now complemented by a 3D system, and the video capabilities have been improved by the addition of a rewritable Pioneer laser disc system, which can hold 32 minutes of video+stereo sound. The main recorder has now been upgraded to Betacam SP, and software written for video input, complementing the Facility's previous output capability. With these additions, the Atlas Video Facility provides new ways for both the scientific and commercial communities to incorporate video into their applications.
The first Atlas Video Facility was designed in 1987 and was based on a Primagraphics computer. Animation was direct to U-Matic videotape, using a Lyon-Lamb animation controller. The facility was provided for the users of the Atlas Centre's Cray X-MP and IBM 3090 supercomputers.
This system was replaced in 1990 by a more powerful system (TOPAZ2), also from Primagraphics. The Unix-based processor was coupled to a TV display controller and digital video interface from Primagraphics. This interface fed video frames over a parallel digital link onto a digital video disk from Abekas, which held 30 seconds of (silent) video. Frames of the animation sequences could now be stored on the Abekas disk in about a thousandth of the time required for writing to videotape. The processor was also about 40 times faster, allowing much longer and more complex animation sequences to be produced. These have been broadcast on BBC, ITV and Channel 4.
Extensions to 3D
The original video systems were targeted at the production of two-dimensional images. Images of three-dimensional scenes could be produced (and many were) but the time required was often considerable: one synthetic flight around the floor of the Indian Ocean required about 6 minutes per frame, totalling 70 hours for a 30 second video!
Early in 1993 the Atlas Video Facility was enhanced by a new system, supplied by Primagraphics and containing both Primagraphics and Silicon Graphics boards. The Silicon Graphics boards provide the 3D capabilities of the system, being a complete Silicon Graphics Elan V board set, engineered as four VME boards. The controlling computer is a SGI MIPS R3000 processor running at 36 MHz. The resulting system is faster overall, but the principal improvement is the speed with which it renders 3D scenes: the Elan boards can achieve up to a million (flat-shaded) polygons per second. The result is dramatic - the flight around the Indian Ocean can now be performed in real-time!
TOPAZ3 also has various specialist boards for graphics operations, including one with 64 Mbytes of RAM for image storage and another that allows real-time (25 frames per second) compression of full-screen, full colour images. This allows the 64 Mbytes of RAM to hold about 90 seconds of compressed video - three times as much as the Abekas disk.
The new system does not replace TOPAZ2, but complements it, so the digital video capabilities of the TOPAZ2 - Abekas system are unchanged. The two systems are connected by a VME-to-VME link, running at 5 Mbytes/second, allowing very rapid transfer between systems. In addition, the two systems use NFS (Network File System) to access each other's disks, filestores in the Atlas Centre's disk farm, and 60 Gbytes of the Cray's filestore. Connectivity is by FDDI for TOPAZ3 and Ethernet for TOPAZ2. These high speed data connections are essential since it is common for a video sequence to require many hundreds of megabytes, or even gigabytes, of input data.
Early in 1994, two major additions to the systems were made, bringing the system up-to-date in video technology, and simplifying the production of longer video sequences.
A Pioneer VDR-VIOOOP disk recorder is now in service, storing 32 minutes of video and stereo audio. This enables smooth assembly of sequences longer than those handled by the Abekas and also provides a buffer store when processing video input. The disk uses exchangeable rewritable laser disks, allowing parallel work on different projects, and has a built-in editing capability. Although the format on disk is digital, it interfaces to the rest of the video facility using analogue component signals.
The main editing recorder, which since 1988 has been a HiBand U-Matic SP recorder, has now been replaced by a Betacam SP recorder which uses analogue component signals, like the Pioneer. Betacam SP is of a much higher quality than U-Matic SP, so the resulting videos are now a great deal sharper. Another advantage is that Betacam SP is now the most common format used by broadcasting companies.
Video Conference Links
Five years' experience of running the facility showed that some users wanted more "editorial control" over their video sequences, but there was no economic way of providing this. Recently, however, the UK academic network (JANET) has been improved with the SuperJANET network, providing 34 Mbits/second links at many sites. At the same time, desk-top video conferencing has triggered the development of affordable display systems for compressed video sequences.
Using the various components of the new video facility, together with video codecs for conventional video conferencing, video sequences stored in any part of the Atlas Video Facility (RAM, Abekas disk, Pioneer disk, videotape in many formats) can now be viewed over the SuperJANET ATM pilot system.
The new system provides an industry standard computer system on which many applications of direct interest to the facility's users can run. In particular, since the introduction of the new system many videos have been made using the AVS (Advanced Visualization System) system, which includes comprehensive support for video through the AVS Animator package.
Video can also be captured from cameras or video-conference links, can be compressed, can be transferred to users in raw or compressed form over the data network or can be used as part of a video conference. Any picture in the system can be printed on paper or foil using dye sublimation printers at the Atlas Centre.
New applications for the system are constantly emerging; one recent request was for video sequences on videotape to be captured in real-time and converted to JPEG/MPEG form so that they could be included on multimedia CD-ROMs. Another was the capture and printing of frames from a video so that measurements could be taken of the images. Commercial users have also appreciated the possibilities of the system and the Atlas Video Facility now has commercial customers as well as UK academic users.
Since 1987 the Atlas Centre has built up a video facility with unique capabilities which have been extensively used by the UK academic community. The introduction of the latest system, including both the system upgrades and the new video hardware, has extended the range of the facility while considerable increasing its performance. The UK academic community continues to have a state-of-the-art facility for producing and processing video, allowing them to visualize their work and communicate their results to others.