Table of Contents Previous Next Index

Section 2 LISTSERV Commands-Job Feature and CJLI Interpreter

Section 2 LISTSERV Commands-Job Feature and CJLI Interpreter
The "Commands-Job" feature of LISTSERV was designed in an attempt to allow for powerful inter-LISTSERV (and more generally, program-to-LISTSERV) command transmission with message redirection and multi-line arguments capability, while still allowing inexpert users to send commands to LISTSERV for execution in a very simple, intuitive" way.
The implementation of Commands-Jobs has therefore been split into two different layers: the 'core' command job language interpreter, with its exacting, powerful but stern control cards syntax, and the 'outer' interface to the user which provides the required default control cards whenever they have been omitted, translating an "intuitive" series of commands to execute into an actual commands-job that can be processed by the 'core' interpreter.
Since this documentation is primarily intended for postmasters and LISTSERV applications programmers, it will be oriented towards a description of the 'core' interpreter. The work of the 'outer interface' will only be mentioned for better understanding. The 'core' interpreter will be referred to as 'Commands Job Language Interpreter' (CJLI) in the following discussion.
Warning: If you are familiar with MVS and JCL, you will probably notice some similarity between command job control cards and JCL. This similarity is purposeful and was intended to make CJLI easier to understand for JCL adepts and to make MVS users more comfortable with CJLI (MVS users who do not have any mailing system such as UCLA mail and have difficulties sending/receiving messages are often forced to use (basic) CJLI control cards to send commands to LISTSERV). However, there are a number of differences between CJLI and JCL and you should not assume that a given JCL card has the same meaning and syntax as its CJLI counterpart. Some JCL features which were deemed to be unnecessary (e.g. dataset concatenation with a blank DD card) have not been implemented, and new features have been implemented whenever required.
2.1 The JOB Entity
As soon as the 'outer' interface detects that a file contains commands (as opposed to a mail or non-mail file intended for redistribution to a list), it passes it to the CJLI for execution. This physical file will contain one or more logical 'JOB's with interspersed comment lines. Each 'JOB' will contain zero or more commands, start with a "// JOB" card and end with a "// EOJ" card. The 'outer' interface will provide these cards if omitted, but this will be detailed later on. Anything before the first "// JOB" card, after the last "// EOJ" or between "// EOJ" and "// JOB" cards is ignored. Notably, mail headers and "Acknowledge-To:" fields (at the bottom of the mail file) would be ignored. A physical job file containing two jobs might look like this:
(any number of header lines, ignored)
//jobname1 JOB options
.
.
.
//jobname1 EOJ
(any number of lines, ignored)
//jobname2 JOB options
.
.
.
//jobname2 EOJ
(any number of lines before EOF, ignored)
 
Each job in the physical file is a completely independent entity which contains commands and the 'dataset' definitions required to execute the commands properly. Jobs are executed in sequential order; if a job does not execute successfully, the other jobs in the physical file are still executed, unless the job has been terminated by a 'global' error (e.g. error reading the physical file), in which case the CJLI terminates after transferring the file to the postmaster. Within a given job, commands are executed in sequential order regardless of the result of the previous commands.
Each job generates a separate 'output', which is sent to its recipients (see below) before the next sequential job is executed. This 'output' consists in a series of messages which are (unless specified otherwise -- see below) sent back as a single mail file.
There are basically three kind of cards in a job stream:
Control Cards – Starts with "//" in column 1 and are interpreted by the CJLI. Control cards are divided into three categories:
Pre-execution control cards, of the form: "//label kwd args". "label" and "args" can be omitted but there must still be a blank between the "//" string and the keyword name, ie "// kwd". If CJLI does not recognize the keyword, it strips off the leading "//" and considers the card as a command-card (see below).
Comments, of the form: "//* any_comment_text". These cards are ignored by CJLI. Note the blank between the asterisk and the first character of the comment text.
Execution-time control cards, of the following format: "//*kwd args". Note that the keyword is concatenated to the "//*" string to differentiate this from a comment. If the keyword is not recognized, the card is treated as a comment control card and ignored. It is otherwise processed by CJLI at execution time, as if it were a normal command card, and in sequence order with the other command cards. These cards are actually commands which are only available from within a command job because they would be irrelevant outside of a job context.
Command Cards – Contains the text of the actual commands to be executed. They are ignored by CJLI which passes them to the main LISTSERV command interpreter for execution. If the card starts with an asterisk, it is treated as a comment and ignored.
Data Cards – These cards are assembled into a dataset under control of a "// DD" control card. They are removed from the job stream by CJLI and are kept in a separate pool for later reference at command execution time.
2.2 Control Cards – General Syntax Rules
All control cards, except comment control cards, follow the same syntactic rules:
1.
2.
Control cards can span any number of physical records: continuation cards can be defined by placing a comma at the end of the first physical line, and having the continuation card start with the string "// " (note the blank) in column 1. This process can be repeated any number of times. No blank is inserted between the end of the first card and the beginning of the continuation card; however, anything before the comma is kept. Examples:

//card1 DD "Some very long text which,
// requires a continuation card"
--> "Some very long text whichrequires a continuation card"

//card1 DD "Some very long text which,
// requires a continuation card"
--> "Some very long text which requires a continuation card"

Since this approach makes it impossible to "cut" a line which ends in a large string of blanks, an alternate method was designed for blanks-sensitive cutting. If the continuation card starts with the string "//+ " in column 1 instead of just "// ", the continuation card is not stripped of leading blanks and data from columns 5-80 is appended to the first card. Example:

//card1 DD "Some very long text which,
//+ requires a continuation card"
--> "Some very long text which requires a continuation card"
3.
Control cards can contain a label of any length starting in column 3. This label is translated to uppercase. If the label is omitted, there must be a blank in column 3. The label can contain any character, except blank and the slash sign ("/").
4.
5.
Arguments can be specified after the "card name", and must be separated from it by at least one blank. Arguments are separated by commas, and there must not be any blank before or after the comma. There are two categories of arguments:
a.
b.
Keywords, of the form "name=data". They can appear in any order and can be freely intermixed with positional keywords. They do not affect the sequence order of positional keywords. Keyword names are translated to uppercase, and can contain any character except blank, comma, double-quote or equal sign.

For example,

//JOB1 JOB XDZ,ECHO=NO,FRECP11,PW=EMERALD
//JOB1 JOB PW=EMERALD,XDZ,FRECP11,ECHO=NO
//JOB1 JOB Echo=NO,pW=EMERALD,XDZ,FRECP11

are three different wordings of the same arguments string.

There are furthermore two different forms of "data" for arguments:

Quoted data: in that case the data is enclosed in double quotes and can contain one or more blank delimited words as well as leading or trailing blanks. Quoted data is case-sensitive, and can contain any character except a double-quote.

Non-quoted data: in that case the data consists of a single word (i.e., blanks cannot appear in the data), which is translated to uppercase. Non-quoted data cannot contain blanks, commas or double-quotes.

In the above example, specifying 'Echo=No' is functionally identical to 'ECHO=NO', while 'Echo="No"' would leave the argument in mixed case. Quoted data is used for list of recipients, full-names, etc.
6.
Comments can be included at the end of the card, and must be separated from the arguments by at least one blank. If you did not specify any argument string, and still want to place comments at the end of the control card, you must specify a null argument string before the comments by putting a ", " before the comment text, e.g.:

//JOB1 JOB , These are comments
//JOB1 JOB XDZ,FRECP11 These are comments too
2.3 The JOB Control Card
The JOB control card indicates the start of a new job and allows you to define several "execution options" for the job. The format of the JOB card is:
//jobname JOB options
'jobname' is the name you want to assign to the job. If you leave it blank, CJLI will default it to be your 'userid'.
Any job options must be comma-delimited with no leading or trailing spaces. For instance, you would find that
//MYJOB JOB ECHO=NO AFTER=19:00
is equivalent to "//MYJOB JOB ECHO=NO". The "AFTER=19:00" option will be treated as a comment (per item #6 in the preceding section, because it is preceded by a space) and ignored. Thus, the correct syntax for the above JOB card would be
//MYJOB JOB ECHO=NO,AFTER=19:00
The following options are available:
The default value (if the keyword is omitted) is YES and indicates that each command must be echoed to the job output before execution. The command is then prefixed with a "> ", and preceded by a blank line on the job output.
The default value is "Sender" and indicates that the output of the job is to be sent to the sender of the job. "None" indicates that no output should be generated, and is used whenever the sender is a server which is not programmed to parse the output of a LISTSERV job. Also it makes sure that no loop can ever occur due to an error in a job. "u@n1 u@n2..." can be used whenever replies are to be sent to a different person/list of persons. These persons will first receive a separate message telling them that they are receiving the output of another person's job.
The default value, "Mail", indicates that the job output is to be sent to its recipients in the form of a mail file. "Message" and "MSG" both indicate that interactive messages are desired. Note that an attempt to send interactive messages to a node which cannot handle them will result in LISTSERV sending a piece of mail containing the text of the various messages.
Determines whether or not server statistics ("Summary of resource utilization") for a given JOB are output in the command response to the invoker. The default is YES. A server-wide default may be set with the JOB_STAT_DEFAULT site configuration variable.
Tells LISTSERV when to run the job, i.e., run it after the time specified. There are three valid formats:
AFTER=date
AFTER=time
AFTER="date time"
The first two formats may also be quoted if desired. The third MUST be quoted since it contains a space. In case 1, the time is 00:00:00; in case 2, the date is today. The only date format that is guaranteed to work is yyyy-mm-dd (other formats may also work, without guarantees). The time is specified in a 24-hour format either with or without seconds: hh:mm or hh:mm:ss.
If the requested date is in the past, the job is executed immediately. Otherwise, it is placed on hold for execution at the exact specified time. If several jobs are submitted for the same execution time, the order in which they are executed is unpredictable. Generally, this would be a bad practice anyway since there is no guarantee that jobs arrive in the same order as you sent them. If on the other hand you schedule your jobs for one second after the previous one in the desired sequence, execution order will be respected, even if the execution of the first job causes the others to be simultaneously eligible for execution.
Is the default password to use on commands where an explicit "PW=" keyword has not been specified. It makes it easier to write jobs performing several maintenance commands on the same distribution list, for example.
All other keywords, as well as positional parameters, are ignored. If an invalid value is specified for a valid keyword, the job is terminated by CJLI but the remaining jobs in the physical file are still executed.
If the JOB card is omitted, the 'outer' interface provides the following default JOB card:
//userid JOB Echo=Yes,Reply-to=Sender,Reply-via=Mail,Stat=Yes,PW=""
2.4 The EOJ Control Card
The EOJ card indicates the end of a job; its syntax is very simple:
//anything EOJ
where 'anything' can be any valid label and is completely ignored. The 'outer' interface provides an EOJ card at the end of the physical job file, as well as before a new JOB card, if none was provided by the user.
2.5 The DD Control Card
The DD control card allows you to define single or multi-line 'datasets' for use by the various commands in the job stream. The syntax of the DD card is the following:
//ddname DD "single-line-constant"
*
*,EOF
*,EOF,Res=Disk
*,EOF,Res=Storage
'ddname' is the name the dataset is to be given. It must follow the general label naming convention, cannot be omitted and cannot appear more than once in the job stream. That is, datasets cannot be concatenated by means of several DD cards.
"single-line-constant" denotes a single-line dataset, whose value is that of the first (quoted) argument. The length of this argument must not exceed 255 bytes.
'*' denotes a multi-line dataset, whose successive lines immediately follow the DD card. Any number of lines can thus be included in a dataset, with a "/*" line indicating the end of the dataset. The JCL "DD DATA,DLM='xxxx'" is not implemented. Note that unlike JCL, CJLI does NOT end the dataset when a control card (i.e. one starting with "//") is encountered; the "/*" must always be specified.
'*,EOF' denotes a multi-line dataset, whose successive lines immediately follow the DD card and end at the end of the PHYSICAL job file. This option is used when transmitting "unknown" data in a dataset that could contain any kind of character string. Needless to say, there can be only one such dataset in the job file, and it must be the last dataset in the last job.
'Res=' indicates whether the dataset is to reside in storage ("Res=Storage") or on disk ("Res=Disk"). In some cases it may be necessary to keep a large dataset on disk to avoid running out of storage and to improve execution speed when a disk-file is to be generated anyway by the command using the dataset. The "Res=" keyword is therefore ignored on all datasets except the '*,EOF' one (if present), and causes a disk file to be generated with the remainder of the input deck. Please note that not all commands will support the "Res=Disk" option: commands which do not expect to receive a large dataset as input will usually expect to find it in storage and report an error when the "Res=Disk" option is used. For example, DISTRIBUTE fully supports "Res=Disk" while DELETE doesn't.
An invalid dataset declaration causes the job to be terminated by CJLI with the remaining jobs in the physical file still being executed.
2.6 The //*MSG Control Card
The "//*MSG" execution-time control card allows you to selectively halt/resume 'typing on the job output' or to discard messages which have been previously output during execution of the job. Note that the latter option has no effect when the output of the job is sent as interactive messages, as it is intended to control mail job output.
The syntax of the "//*MSG" control card is:
//*MSG option1,option2,...
Valid options are: ON, OFF, FLUSH
FLUSH discards all messages previously sent to the job output and leaves the message-receipt status unchanged.
OFF turns message receipt off, as if "Reply-to=None" had been specified in the JOB card.
ON turns message receipt back on. This does NOT override a possible "Reply-to=None" in the JOB card, though.
2.7 Special Considerations
This section contains more information on the trickiest parts of CJLI as well as some useful hints for application programmers.
LISTSERV treats anything mailed to the LISTSERV userid as a set of commands to execute. Thus, as soon as an unknown command is encountered in the job stream, the whole physical job file (not just the current job) is immediately flushed and discarded. This avoids 'executing' hundreds of unknown commands and sending back a huge job output when a regular mail file is sent to the LISTSERV userid by someone who thought it would be distributed to a list. Note that errors from known commands do not cause termination of the job -- only completely unknown commands such as "Hiya!!" would terminate the job.
Although CJLI is based on the network standard 80-characters card images, LISTSERV accepts command jobs in several network formats, including Disk Dump and Netdata. In that case it will accept records of up to 255 characters as input, and you may find this very convenient when sending long commands to the server.
Alternatively, continuation cards can be used to split long commands into several 80-characters cards. In that case you must insert a "// " string before the command text so that CJLI considers it as a control card and performs the required concatenation; it will then realize that the "card name" is unknown and transform the card into a regular command card. If you opt for that method, you will find the "//+" continuation card feature very convenient for a program (but not for a human person). This method is used by LISTSERV when transmitting "DISTRIBUTE" commands to other LISTSERVs.
There is no limit at all on the final size of a control card, i.e., on the number of continuation cards you can specify; however, you must make sure that no line in any of the 'datasets' ever exceeds 255 characters. In particular, if the physical job file is sent in Netdata format, you must make sure that the file lrecl is not higher than 255.
Note for VM servers: Due to internal coding considerations, it is recommended that physical job files be sent to LISTSERV in PUNCH format. This will make it easier for the 'outer' interface to detect the job file for what it is and will save some CPU time to the server, thereby improving job response time. Please keep in mind that DD lines longer than 80 characters cannot be sent in PUNCH format.
For more information on how to send commands to LISTSERV for execution, see LISTSERV MEMO.