Jump Over Left Menu
Issue 18: January 1992
Flagship Issue 18
- The Virtual Tape Protocol - for tape access over TCP/IP
- FDDI installation at Atlas
- Report from the Eighth Cray User Meeting, 9 December 1991
- RAL-GKS Validation
- Colour PostScript Service
The Virtual Tape Protocol - for tape access over TCP/IP
This system enables programs and applications running in workstations and other machines connected to the VM system via TCP/IP, to read and write data on magnetic tapes in the main tape library as if the tapes were mounted on a drive connected directly to the workstation. Without modification, workstation applications benefit from the tape and data management facilities of the central VM system, by automatically making effective use of mainframe disks and the Automated Cartridge System (ACS), in order to minimize delays waiting for mounts and positioning tapes.
There are currently two major areas of use for this software.
- User applications processing data on tape. The alternative would be to transfer whole tapefuls of data onto the workstation disks and process it there. The remote tape system has all the advantages of record-level I/O - no need to transfer large amounts of data which will not be used, and high speed positioning within a tape. It is also intended to be as fast as other access methods for large sequential transfers.
- The tar and dump UNIX utilities, for performing backup and archive functions for the workstation. Such backups could be run automatically, without human intervention. The use of labelled tapes and file expiry dates adds significantly to the security of such operations.
The Atlas Centre is participating in a Joint Study Agreement with IBM, investigating co-operative processing between the IBM mainframe (3090-600E +6VF) and (UNIX based) workstations.
One of the first projects in this area is called HUMPF ( Heterogeneous UNIX Montecarlo Production Facility). This project is investigating the production of simulated particle physics data on powerful workstations whilst accessing a large amount of data currently stored on 3480 cartridge tapes on the IBM 3090. The project is still in progress and although we have a working protocol and application, it is not complete.
The system comprises:
- A tape server on the IBM 3090 (currently handling up to 30 concurrent tape streams).
- A Virtual Tape Protocol library (VTP), written in C which is available on the client workstation.
- A tape command, built on the VTP library, which accesses the tapes and provides handling for files on No Label, Standard Label and ANSI Label tapes.
Virtual Tape Protocol
The Virtual Tape Protocol
The Virtual Tape Protocol passes data to and from the tape in binary mode (that is, no translation takes place). The protocol itself has no notion of tape labels: it regards the tape data stream merely as a stream of data blocks and "tape-marks", just as a real tape drive does.
The usual tape controls are provided (rewind, unload, forward space, backward space, write tape-mark, read, write, query location and set query location).
The virtual tape protocol runs between the workstation running the application (the client) and the tape server program running continuously in the central IBM 3090 mainframe. The protocol is not symmetric, since all requests come from the client and the server responds. Typical requests are "mount tape xyz", "read next block", "rewind tape" or "forward space to next file-mark". The protocol is general enough to allow all possible functions which can be performed locally on an IBM3480 tape drive to be performed remotely. The client can send further requests before the reply to the first has arrived, which will allows VTP to make full use of the network bandwidth.
The communication path over a TCP/IP socket is completely transparent, allowing maximum flexibility to the applications. The system works with any format of tape, since it simply views a tape as a series of blocks and tape-marks. It leaves the problem of formatting the data on the tape to the applications, just as a real tape drive does.
The VTP library
The VTP library provides C programs in a UNIX environment with an interface to the Virtual Tape Protocol. Each connection to a tape via VTP is termed a "tape stream" and each tape stream has a unique number. The library can currently sustain a maximum of twenty concurrent tape streams.
The VTP library automatically handles loss of the data connection to the server machine and attempts to reconnect the client to the tape server transparently with no loss of data. This is necessary, as the workstation applications typically take many hours to complete and must be able to survive a loss of service on the mainframe without wasting the computing effort up to that time.
The IBM server
The IBM server runs under the VM XA SP2 operating system using the C interface into IBM's TCP/IP product running under CMS. The server currently supports up to thirty concurrent tape mounts.
The server provides the authorization checks to ensure that the client is allowed to access the tape. The mount requests from clients provide a userid, password and tape identification which are used by the server to validate the request.
The first application that has been written using the VTP library is the tape command. This initiates tape mount requests to the server and allows access to that data in a number of ways: either directly to a file, or on the stdin or stdout, or as a pipe.
This command also handles the tape labels; it recognises and can generate standard IBM EBCDIC labels, the ANSI standard ASCII labels, and No Label tapes.
It has been successfully used to back-up and restore data from UNIX hosts using the tar command piped to the tape command. We envisage this use of the system to increase rapidly, as much interest is already being shown of the use of IBM tapes for back-up from workstations.
The following examples of tape use show the back-up and restore of all the C files in a particular directory. The value mytape is the name of the tape (in line with normal IBM practice, the tape name may be up to six characters long). The value tk is the username, the value pass is the password used for authorisation checking.
tar cvf - *.c | tape -w mytape tk pass
This will write a tar-file, containing all the .c files in the current directory, onto a tape named mytape. The option -w indicates that the tape is to be written on. The tar command will output the tar-file to the stdout stream (indicated by the - option), this data is then piped to the stdin stream of the tape command
tape mytape tk pass | tar xvBf -
This will extract the archived files from the tape named mytape. The B option is required to enable tar to read correctly from the pipe from the tape command.
The code is written in C in such a manner as to be portable across hardware platforms. The following platforms are successfully using the VTP client code:
- Sun 3/150 running SunOS V4.1
- Cray X-MP running UNICOS V6
- DEC Station running Ultrix V4.2
- PC (386) running Sys V R4 (ESIX V/386)
The performance depends upon three factors: the IBM server; the client workstation; and the network in between. Current tests on an (old) Sun 3/150 workstation have measured data transfer rates in the range 80-110 Kbytes per second. The limiting factor may well be the Ethernet interface into the IBM 3090, which has a theoretical limit around 120-150 Kbytes per second.
Further development and tuning will concentrate on improving the performance. The installation of an FDDI (Fibre Distributed Data Interface) network running at 100 Mbits per second is scheduled for February (see article on next page). This should provide the extra speed to enable more tuning to take place.
We intend to port the VTP code onto an IBM RS6000 workstation and a VAX running VMS.
The tape command will be enhanced with support for multi-volumes (that is, a single tape data stream that actually uses more than one tape). The tape command should also work with the standard IBM record formats (currently it regards the records of data as a binary stream only).
Any other suggestions for possible applications and enhancements of the Virtual Tape Protocol are welcome.
Tim Kidd, IBM Networking Group and David Rigby, Systems Group
FDDI installation at Atlas
FDDI Installation at Atlas
Recently the IP address of the Cray was changed. This was a necessary part of the plan to install an FDDI Local Area Network in the Atlas Centre. FDDI is the "Fibre Distributed Data Interface" - it is a dual ring of fibre optic cables carrying data at 100 Megabits per second.
The FDDI kit should be installed and commissioned during early February. It will provide fast access initially between the Cray and the IBM, but will be extended to include other machines.
The installation of the FDDI network will enable us to rationalise access to the Cray. Currently all IP traffic for the Cray is sent via the IBM, which is seen as a bottleneck due to the performance of the IBM interface to the Ethernet. We hope that providing direct access via FDDI to both the IBM and the Cray will enhance the performance of both machines even for users accessing them via the site Ethernet and prepare for future higher speed access across the RAL site and JANET.
Whilst the introduction of an FDDI network gives no extra services, we hope that users will find the speed increase useful and the address changes required to install this network justified.
Tim Kidd, IBM Networking Group
Report from the Eighth Cray User Meeting, 9 December 1991
Despite a particularly cold and misty day there was a remarkably high attendance at the Cray User Meeting held at the Atlas Centre on 9th December 1991. This meeting was the first to be held under the new chairman, Professor Richard Catlow from the Royal Institution.
Survey of the Last Six Months
Roger Evans gave a short summary of the Cray service for the six month period from the previous User Meeting. There had been a noticeable reduction in demand during the summer months: instead of being totally saturated the Cray had been comfortably full, with the input queues being cleared each weekend. During this time the user service had been good with remarkably few software failures and generally acceptable turnaround.
Coinciding with the introduction of UNICOS version 6.0, the workload had increased and job turnaround had deteriorated. Over the past twelve months it is clear that software unreliability is correlated with exceptionally heavy user demand. When the scheduler and data migration processes are most under pressure then they are most likely to give problems. In terms of CPU hours delivered to users it is the last few per cent that causes difficulty all round.
Particular problems with the Cray data migration product had again resulted in the loss of some user data. The matter was being raised with Cray at the highest level but it was not possible to run the Cray service without using data migration because of the size of the user filestore.
Those with long memories were reminded that the "Forty Report" in 1986 had recommended a UNIX front-end service as well as VM and VMS. It is now finally possible to provide this service: an IBM RS6000 model 550 was delivered to Atlas in December. The RS6000 will provide a completely UNIX route to the Atlas supercomputers, will relieve the Cray X-MP of the burden of trivial interactive work, and will provide services such as interactive X-window access across JANET that are not appropriate to the Cray service itself.
New Services based on IP
Nigel Jagger described the new JIPS (JANET IP Service) and the advantages it offers particularly to Cray users. The IP protocols are widely used in the USA and many other parts of the world and are the de facto standard for UNIX workstations on Ethernet. It is now possible to run IP across the JANET networx and from a terminal or workstation to login to UNICOS, perform interactive ftp (file transfer) or run the X-windows protocol.
Users wishing to take advantage of the JIPS service are requested in the first instance to contact their local computer centre tor advice but RAL staff will assist in any details relevant to the RAL services.
UNICOS version 6.0 has several X-window based tools to improve programmer productivity and program performance:
- cdbx the interactive debugger now has an X-window interface.
- flowview and perfview assist in visualizing the control flow and "hot-spotting" the code for more detailed tuning.
- atscope assists in determining the scope of variables for auto-tasking.
- atexpert shows the performance gains from auto-tasking and identifies areas of parallel and single threaded code.
Support for auto-tasking is much improved under UNICOS version 6.0 and all users with large memory jobs are encouraged to try out the auto-tasking option. Even a modest amount of parallel execution will reduce the job charge and will help to make for more efficient use of the machine. To compile a program fred.f for autotasking use the command line:
cf77 -Zp fred.f
or if you want to see the detailed output of the autotasking preprocessor (fmp) use:
cf77 -ZP fred.f
New Application Packages
The Computational Fluid Dynamics code FLOW3D from Harwell is now available, but users must make individual arrangements with Harwell for licensing. New versions are available of Abaqus, Amber, Gaussian, Mopac, Phoenics and various NAG libraries.
A New National Supercomputer
Brian Davies gave a brief history of events that have led to the allocation of Â£5 million in 1992 and in 1993 for enhancement of the UK academic supercomputer services. Saturation of the machines at the three national centres had led to a submission from ABRC to government and money was now earmarked for supercomputing.
The Supercomputing Management Committee (SMC) had decided that as a first step, a single conventional supercomputer should be obtained and a tender action had been initiated to establish the most effective way to spend the money.
A benchmark suite is being established by the three national centres and the results of this benchmark will carry considerable weight in the decision process. Users were invited to participate by suggesting suitable jobs for the benchmark.
At lunchtime and after the meeting there were demonstrations of the UNICOS 6.0 performance tools and of the Cray distributed application packages MPGS and Unichem. MPGS is an interactive visualisation tool mainly targeted at the engineering community; Unichem is an integration of a highly visual front-end running on a Silicon Graphics workstation with various ab initio and molecular dynamics programs running on a possibly remote Cray supercomputer. MPGS and Unichem will probably become available on the Atlas Centre Cray X-MP during 1992: potential users are invited to contact us for further information.
Following the format of previous meetings, the afternoon was an open discussion: dissemination of "news" proved to be an important topic. Users expressed a desire for notice of service interruptions to appear more rapidly as news items on the front-end machines. The idea of an e-mail distribution list for news or a separate "news id" were discussed as possible improvements.
As well as the problems with data migration there was concern about the /tmp file system filling up, although it was appreciated that restricting the use of /tmp would only worsen turnaround. There was a clear desire to have any new disk space allocated to /tmp in preference to /atlas. Users are encouraged to use $TEMPDIR for working space where possible, since this is released immediately a batch job terminates, whereas the general clean up of /tmp is on a 24 hour basis.
The subject of reserve priority jobs raised mixed feelings. During the summer there had been occasions when more reserve work could have been started, but the opposing concern was that reserve work should never impede high priority work. The current saturation of the machine made it less urgent to review policy in this area but a slight relaxation of the use of reserve priority was generally favoured.
The conjunction of the open User Meeting with the new Atlas Centre User Committee meetings means that it is now appropriate to book the date of the next meeting well in advance. The next Cray User Meeting will be at the Atlas Centre on Thursday 30th April 1992.
Roger Evans, Advanced Research Computing Unit, Central Computing Department, Rutherford Appleton Laboratory
1991 was the year that RAL-GKS was successfully validated.
The National Computer Centre (NCC) provides a GKS testing service for the UK. This ensures that GKS implementations conform to International Standard 7942. In March these tests were completed on RAL-GKS, and NCC issued RAL with a Certificate of Conformance and a Test Report. The certificate is valid until May 1992 and we believe that it makes RAL-GKS the only currently validated GKS in the world.
The test suite was developed by the University of Leicester and Technische Hochschule Darmstadt. It contains 110 tests covering the following areas:
- data consistency
- data structure
- error handling
- output and input
They were run on RAL-GKS 1.34 on a Sun 3/60 with Sun OS 4.0.3, running on a Sun screen, PostScript, Tektronix 4010/4014, GKS Metafiles and WISS.
All the main output primitives (POLYLINE, POLYMARKER, TEXT, FILL AREA and CELL ARRAY) were tested but not the Generalised Drawing Primitive (GDP). All input devices were tested (LOCATOR, STROKE, VALUATOR, CHOICE STRING and PICK).
The Testing Process
The testing process consisted of three stages:
- Pre-Validation Testing
- On-Site Testing
- Interpreting the Results
The pre-validation was performed by RAL and after a lot of bug-fixing (in both RAL-GKS and in the test suite!), there were no errors, except in the metafile tests. All these errors arose from an assumption about metafiles which RAL disputed. NCC accepted all the bug-fixes on the test suite.
The reports, automatically produced by the test suite, were sent to NCC and after these were determined to be without error, the on-site testing was done. For the on-site testing NCC sent a testing specialist, who brought a tape of the test suite with the bug fixes and supervised the mounting of the tape and the compiling, linking and running of it all. This was done in the week beginning on 18th March 1991 and went smoothly.
The results were as expected from pre-validation testing. RAL successfully challenged NCC's interpretation of the GKS standard over metafiles. Hence, the results were interpreted as no deviation has been detected.
The Metafile Dispute
NCC had originally interpreted the GKS standard as saying that all GKS commands should be recorded in the metafile. RAL took the position that, in the interests of economy (shorter metafiles, quicker to produce), only commands that actually affect the displayed graphics, should be recorded.
For example, an application may set the character height (a text attribute) and then produce no text. RAL-GKS would not put the setting of the character height into the metafile. If text were produced later in the program, the most recent setting of the character height would be put in just before the text. NCC originally assumed that all settings of the character height would be put into the metafile immediately, irrespective of whether they were used or not.
RAL initiated an official dispute query to be sent to the official GKS Technical Authority, GMD Bonn, who passed it on to the GKS Testing Services Control Board. The Control Board decided in RAL's favour, thereby enabling RAL-GKS to be validated successfully.
This was the first GKS Validation in the UK. It made sure that RAL-GKS conformed to the ISO standard, and at the same time tested and improved the test suite. The Advisory Group on Computer Graphics (AGOCG) initiated and supported it for both reasons.
Additional workstations drivers have been improved and passed testing using a copy of the test suite; currently these include IBM 3179/3192 G screens and Tektronix 420x terminals.
Karl Palmen, Graphics Group
Colour PostScript Service
We are pleased to announce a new colour hardcopy service, based on a QMS thermal transfer printer. This prints A4 colour output from PostScript files onto white paper, producing excellent results suitable for use in publications, as well as for presentation of results.
This QMS printer has been on self-service trial for several months while essential improvements were made to its driving software, and CCD can now offer a general operated service to users of SERC computers. The printer is attached to VAX VMSFE and it is accessed via Ethernet, using the LPR/LPD system. The name of the printer is PS4CR2 7 A, so the LPR command on the IBM 3090 system is
LPR fn ft fm ( P PS4CR27A @ VMSFE.RL.AC.UK
From systems other than the IBM, it is essential that you supply your distribution code on the CLASS (C) option of LPR. Files sent without a CLASS parameter, and files that are not valid PostScript, will not be printed. Files with an invalid distribution code will be printed but the operators will have no way of knowing where to send your output!
The following command, which would be valid on a Unix system or a VAX running DEC's Ultrix Connection together with software available from CCD VAX Support, specifies a distribution code of post-du, meaning Post to Durham:
lpr -p ps4cr27a -c post-du filename
where the printcap file on the system contains an entry for ps4cr27a indicating that it is hosted on vmsfe.rl.ac.uk. Output will be distributed to users via the methods used for Xerox 4050 output printed in R26.
The paper loaded in the printer is of a size called Special A4 which is the same width as A4 (portrait) but is longer (14in, or 356mm). This is used so that a complete A4 image (minus a 5mm margin, as is common on many PostScript printers) can be printed with the information identifying the job and user above the image. Any output outside the limit of A4 is liable to be clipped and/or overprinted.
At present foils are not available on this printer for operational reasons; it is hoped to remedy this situation soon.