Some facilities of a computer system represent a limited resource which is only capable of accepting up to a certain amount of demand upon it before its use becomes increasingly inefficient and the demand more difficult to meet. Its greater use also may adversely affect the performance of other parts of the system. Budgeting is the means by which the available resources are shared out equitably amongst the users so as to retain the overall demand within reasonable limits. To effect control of jobs running on the machine within the budgeted limits, it is necessary to check the rate of use of the resources and to keep accumulated totals of the usage by each user. It is also required to provide management and user information on total amounts of resource use within various periods and classifications, to check that the levels of budget allocation are reasonable as compared to amounts of use, to provide a basis for assessing probable future demand and to provide a count of costing units for the charging of work run, where applicable. These tasks are performed by the Accounting scheme. Scheduling is the control of jobs as they run on the machine in an attempt to maximise use of the facilities, minimise the time for throughput of work and to provide priority for some special jobs. These aims often prove incompatible, but can usually be suitably balanced, partly by use of information in common with the accounting scheme.
A user should be aware of these matters so as to assess the amounts of budget resource which he will need to work with, to understand the control brought about by accounting and the information it provides him with, and to provide the correct scheduling information to maximise throughput of his own work. This area is necessarily subject to frequent review and change in detail as the amount and type of work presented to the machine changes, and so the exact form of the following description may not always be true in future. However, the general principles continue to apply and the subject may always be viewed within the same framework.
The following resources are those currently controlled by budgets at ACL: time, magnetic tapes, filestore size, number of files. The first of these is under such a coarse level of immediate control that it does not affect most users at the budget level. Further details of the budgetary controls that do apply are given later in I.3.
There is also a special kind of budget called a PRIVILEGE which affects the ability to use certain commands instead of controlling a resource. The holding of a privilege enables the user to use the commands controlled by it. Loss of the privilege implies loss of the ability to use those commands. Note that there are no numerical factors connected with privileges. A user either has a privilege or does not.
For each budget type there is a system budget record associated with each username.
The BUDGETQUERY command (D.5.8) may be used to check the state of any budget held by the user, including whether or not a privilege is held. The command will normally be used as defined:
BUDGETQUERY budgettype
The reply from the command depends on the form of the budgettype parameter, and examples are shown below for each possible budget type.
The BUDGETQUERY command and its reply for this budget type, as relevant to ACL users, would be, for example:
BUDGETQUERY TIME (M) RATION 2000 ALLOWANCE 36000 USED 453
The ration is the amount given for use at the start of each account period. The allowance is the amount available for use in the current period - that is, the ration plus any additional allowance given. The used part contains the amount used so far in the current account period by jobs running under this username. The units of TIME are mill-seconds (not directly related to clock seconds).
A TIME budget represents a notional central processor time, and GEORGE allows for a number of TIME budgets split into sections called URGENCIES'. These are called TIME(A), TIME(B), ........ TIME(M), ........ TIME(Z) respectively. Generally, time used at a higher urgency (A = highest) means greater priority for the job on the machine.
Currently at ACL, TIME(M) is the only urgency used, and most users have an effectively infinite amount. This is achieved by the ACL accounting programs specially updating the allowance of the TIME(M) budget records for each user at the start of each accounting period, which makes the ration figure irrelevant.
Each user has a SPACEMT budget which controls the number of magnetic tapes that he may own. A BUDGETQUERY command and reply for this budget could be:
BUDGETQUERY SPACEMT RATION 6 IN USE 4 CONSUMED 0
The ration here is the maximum number of magnetic tapes that the user may own at any one time. The in use entry shows the number currently owned. The ration remains set (until changed by ACL) as the amount available at any time. If some tapes are freed from use, then the same number within the ration level are immediately available for owning again. There is no account period updating of SPACEMT budgets - the levels may fluctuate at any time.
A SPACEMT budget is used by getting and freed by returning tapes.
Each user has a FSTORL budget and also a FSTORU budget. Example of BUDGETQUERY commands and replies for each of these are:
BUDGETQUERY FSTORU RATION 800 IN USE 0 CONSUMED 600 BUDGETQUERY FSTORL RATION 750 IN USE 0 CONSUMED 500
The rations of these relate to the total size of files owned by the user, entered either in his own directory or a directory inferior to it (a pseudo user). The FSTORL ration is a level of size at which a user is warned about the size that his filestore has reached. The FSTORU ration is a level of size at which a control is applied if the user's filestore holding is found to exceed it. Full details of the control system used are given later in I.3.3.
The amount in use for these budget records is always zero. This is because the budgets are not updated by GEORGE but are used instead by local ACL accounting programs. The consumed entry of the budge records contains the size of filestore owned by the user when the accounting program last checked it.
There are no actual budget record types representing directly the number of filestore files owned by a user. Instead, this factor is controlled relative to the size of filestore budgets mentioned previously. If the rations of the FSTORL and FSTORU budgets are n and m respectively, then the number of files ration levels are currently taken as n/10 and m/10 respectively, these again being warning and control levels. Full details of the control system are again in I.3.3.
An example of the BUDGETQUERY command used to test holding of the CONTEXTA privilege, and its reply is:
BUDGETQUERY CONTEXTA OK
This would mean that CONTEXTA privilege was held by this user. Conversely, a reply of NO would mean that it was not held.
CONTEXTA privilege controls the ability to use those commands which load a core image (D.4), and thus a user's ability to run programs. CONTEXTA privilege must be held to be able to obtain a core image, and the giving and taking of it is the control mechanism used by the filestore accounting system described below (I.3.3), based on the levels in the FSTORL and FSTORU budgets (I.2.2.3).
A new user will be given appropriate levels of TIME(M), FSTORU and FSTORL budgets automatically when his username is first introduced to the system. If he needs to obtain magnetic tapes from the system then he must apply to the ACL tape librarian (tel ext 6640) who will adjust his SPACEMT budget accordingly. If he feels that his ration controlling filestore size and number of files is inadequate (or excessive) for the work that he wishes to run on the machine, then he should apply to the Supervisor, Operational Services (tel ext 6284) in the first instance. The claim will be considered, and may be passed to Resource Management Branch for further negotiation if the required new rations are outside certain limits. Generally, temporary increases of filestore rations for strictly limited periods will be met more readily than permanent increases.
The first of the three levels of accounting (which costs users for resources used) is run at the end of each job. It accumulates totals of the resources used during the job and updates the amounts used of TIME and MONEY budgets. Currently, ACL charges only for TIME used (uniformly for every urgency, if relevant). A section is appended to the end of each job's monitoring file showing the amounts used and left (left = allowance - used) of TIME budgets. The format of these records may well change in future.
The ACL accounting period is currently one week. A period accounting program is run each week (currently about Sunday midnight) which totals into different accumulative periods the weekly figures for each user and provides ACL with records of budget use for each, user, organisation and subject-type. Transient budget amounts used and allowances are reset (I.2.2.1). Further analysis programs may be run at other times summarising totals from a user database, and some of these usage summaries may be made available to various classes of users.
The ACL filestore accounting programs are currently run once a week. The runs may be at any time on any day of the week, but there will always be at least one clear week between subsequent runs. The details of the filestore accounting system are given below.
Each user's filestore size and number of files owned are checked at the time of running the filestore accounting programs. The files checked are all those in the user's directory and any other directories inferior to it (pseudo users). The size and number are then compared with the rations given in, or implied by, the FSTORL and FSTORU budgets (I.2.2.3 and I.2.2.4). If the size and/or number are found to be greater than the lower (warning) levels, but less than the upper (control) levels, then a warning is issued. This will cause a note to be sent to the user, who should treat the warning seriously and attempt to reduce the filestore holding. If the size and/or number are found to exceed the upper (control) level then, provided that the user has been previously sent at least one warning, CONTEXTA privilege (I.2.2) will be taken from that username, thus removing the ability to load programs. There will be a delay between the check being made and the privilege being removed as the latter will be done after some manual checks. If the size and/or number of files are found to be above an upper level but no previous warning of exceeding a lower level has been issued, then a warning will be given and no control action taken at that point.
When a user's holdings are found to be below the lower ration level, a warning sent flag for the username will be removed and subsequent checks will again require at least one warning before applying control.
A user who has lost CONTEXTA privilege should reduce the filestore holding to a point below the condition shown to be at fault on the information sent (or known about previously). Note that jobs can still be run using GEORGE commands to effect filestore changes. Having reduced his holding to a satisfactory level, notification should then be provided to the Supervisor, Operational Services - ext 6284 - in the first instance, whereupon CONTEXTA privilege will be restored subject; to a minimum period of 24 hours between its removal and return.
Alternatively, a user who feels unable to work within prevailing filestore ration limits should negotiate for an increase of ration, as described in I.2.3.
With the filestore accounting system in mind, the LSTR macro (T.1.1) may well be found useful in giving directory listings with totals on size and number of files. Also, the GEORGE LISTDIRECTORY command (D.3.11) will help to keep track of the user's filestore.
Assistance may be sought from ACL in determining which files have been unaccessed for long periods. This may help the user to decide which files can be erased or archived.
Currently, there is no real budget control of TIME that would affect most users at GEORGE or accounting level. However, a manual check will be kept of the rate of use of the machine, and users may well be contacted individually if the need arises. This situation could well change or become more stringent in future.
Magnetic tape usage is controlled permanently by the SPACEMT budget which GEORGE checks when a user attempts to obtain a new magnetic tape from the system.
The total size and number of filestore files is checked and controlled by the filestore accounting system. A user who loses CONTEXTA privilege by control of the system is bound to suffer on future throughput of work. It will not be possible to load programs for at least 24 hours. It is therefore advisable to maintain filestore use within the necessary limits all the time.
A scheduling process is used on jobs flowing through the machine for the following broad reasons:
These general aims would prove incompatible if fully applied individually; for instance, the machine would run quite efficiently if only long processor-bound jobs were run continuously in conjunction with long compatible I/O jobs, but this could only be achieved at the expense of all other shorter jobs being held up indefinitely, which is against objective (2) above.
How these potential clashes are resolved in practice is described below. Users may well achieve faster turn-round for their jobs by being aware of the requirements of the scheduling system described and following its principles.
A background job in the 1906A progresses through various states before it is completed.
On entry to the machine, by a JOB or RUNJOB command, the job becomes an entry in a file (known as the Secure Jobwell) and a monitoring file is created for the job. The job then represents a negligible overhead on the system and in this state is unaffected by system breaks.
The job is eventually started by the High Level Scheduler (HLS). The time when the job is started will depend on its resource requests relative to the current and projected machine loading. These resource requests are made known in the JD parameter to the JOB and RUNJOB commands.
Once started, a job may obey certain commands - involving file manipulation or starting other jobs - without further intervention by the HLS. However, if the job tries to obey a command which loads a core image, it must wait to become fully-started. The delay depends on the state of other programs running in the system; it will normally be only a few seconds.
There are various resources, the requirements of which must be specific in a JD parameter to the RUNJOB or JOB commands. There are currently four classes defined: jobtime (JT), maxsize (MZ), 9-track magnetic tapes (MT) , and 7-track magnetic tapes (7T). The requirements need not all be explicitly specified as there are default settings.
More scheduling classes are likely to be added in the future, although the defaults will always be chosen so that the majority of jobs need not specify their requirements in such classes.
There is normally no point in specifying the use of a resource if it is less than the default value. An exception is jobtime; although the default is two minutes, it is worthwhile specifying a requirement less than this as a quicker turn-round will result (I.4.5).
This is the maximum total accounted time that the job will use from start to finish. The job is abandoned by GEORGE if it attempts to use more time.
The accounted time in a job is a combination of the central processor time used by its object programs (printed in the monitoring file as CLOCKED time when a core image is deleted) and GEORGE time. The GEORGE time arises from job control commands and input-output activity.
If no JT is given, a default of 2 minutes is assumed. Some examples are:
RUNJOB PRMMOP,:NTBE34,MYJOB,JD(JT 60),PARAM(TEST) JOB JOB2,:NTBE34,JD(JT 5MINS) RUNJOB JOB3,MYJOB1,JD(JT 100SECS)
Since the introduction of the JD parameter, the JOBTIME command is almost redundant. It can be used to reduce the jobtime allocation of a job but not to increase it.
This is the maximum size of object program core that a job may have. The parameter is rounded up to the nearest multiple of 1024 (1K). The size of an object program is printed in a SIZE GIVEN message in the monitoring file whenever a program is loaded or it increases its size. If no MZ is given, a default of 100000 is assumed.
If MZ0 (an alternative form is NZ) is used, the job will generally be started immediately but it will not be allowed to load a core image at all. This is useful in obtaining an immediate turn-round for a job that, for example, will simply list a file.
Some examples are:
RUNJOB JOB1,:NTBE34,MYJOB,JD(JT 200,MZ 120K) JOB JOB2,:NTBE34,JD(MZ 250000)
The MAXSIZE command is now almost redundant. It can be used to reduce the allowable MAXSIZE of a job, but not increase it.
A job must declare the maximum number of magnetic tapes of each type, 7-track (7T) and 9-track (MT) which may be used by the job at any one time. Thus a job which uses one 9~track tape, finishes with it and then uses another, will only need MT1 in its JD parameter. Some examples are:
JOB JOB1,:NTBE34,JD(JT 5MINS,MT2) RUNJOB JOB2,:NTBE34,MYJOB,PARAM(RUN),JD(7T 1)
This is now an optional command which enables a job to give the operators more information about potential use of tapes or discs. It is issued before a job becomes fully started (loads its first core image) and may enable the tapes or disc to be loaded while the job is waiting to become fully started. This may reduce later delays, particularly if the job is using more than one tape. Some examples are:
MEDIA TAPE7,1,(700124) MEDIA TAPE9,2,(124,127) MEDIA EDS,1,(027)
There is now no point in using this command without the third parameter.
MOP jobs (those STARTED by a LOGIN command) are restricted in the operations they may perform in that they will not normally be fully started by the High Level Scheduler. This means that fully interactive MOP jobs are not allowed in normal service. The TIME used by MOP jobs when not in fully-started state is given priority in the time-sharing of the system over all jobs. A JOBTIME command may be used in a MOP job, but GEORGE will not terminate the job when JOBTIME expires - instead, a message is sent to the job and more time is allocated. MOP jobs may submit jobs to the background service or to the JOBQ (I.4.6), and for those jobs the normal scheduling rules apply (I.4.5).
Background jobs (started by JOB or RUNJOB) are subject to scheduling by the High Level Scheduler if they attempt to become fully started and should use the appropriate scheduling commands. The potential clashes mentioned in I.4.1 are resolved by a general scheduling strategy for background jobs of giving higher priority to fully-started jobs with the shortest JOBTIME, so that the maximum number of jobs of the least expense to the system are run. However, jobs with longer jobtimes are still given some attention, and they will be run within some reasonable time depending on the state of the workload. Generally, in a heavily loaded machine, jobs with longer jobtimes will be fully started by the High Level Scheduler with less priority than those with shorter jobtimes, but no job will wait indefinitely. Once fully-started, all jobs are time-shared with the same priority.
Therefore, it is in the user's interest for minimum turn-round time of jobs to tend to minimise JT requests. However, care should be taken that the amount requested remains sufficient to complete the job, otherwise the job may be abandoned by GEORGE when JOBTIME expires just before completion and possibly cause waste of the complete run. An adequate but not excessive JOBTIME request should be the aim.
This facility is provided to enable users to input details of a job to be run into a standard file so that the job can be initiated outside of prime shift by the operators. This is for jobs which exceed any prime shift restrictions currently in force or which need special operator attention.
The facility is open to all users who have access to MOP or RJE terminals and may be used at any time when these services are on the air.
An entry to the file :OPERATORS.JOBQ should be made by use of the COPY(APPEND) command (D.3.6) in USER context, and will consist of a RUNJOB command followed by various comment lines. Full details are given in F.3.
WAITFACTOR is a value that is maintained by the High Level Scheduler. It represents the approximate time in minutes that a one-minute job, submitted then, is expected to wait to become fully started. Shorter jobs will be proportionately quicker and longer jobs (provided they do not exceed any restrictions in force if it is prime shift) proportionately slower. Jobs greater than ten minutes will be regarded as ten-minute jobs from the point of view of WAITFACTOR. Jobs with MEDIA requirements (I.4.3.4 and I.4.3.5) may wait longer if the required devices are not available.
When WAITFACTOR is large (greater than 40), guaranteed users will be given priority. The current value of WAITFACTOR may be found by obeying the command SCHED WF (D.5.9). An example of the dialogue on a MOP terminal might be:
SCHED WF DISPLAY: WAITFACTOR=12 END OF MACRO
The estimated wait-time obtained will generally be correct to within ten minutes assuming no hold-ups (such as system breaks) occur.
