Tokens for the language ICPL, Phase 4

Reserved Words

program
procedure
result note: not a token, but rather a predefined identifier.
identifier
begin
end
integer
real
string
boolean
void
if
then
elseif
else
loop
when
exit
get
put
putln
not
and note: not a token
or note: not a token
mod note: not a token

Operators (according to precidence level, highest first)

exp_op ^
mult_op * | /
add_op + | -
rel_op = | < | > | <= | >= | #
bool_op and | or

Symbol tokens

; , := ( )

Constants

let char-d be the set of all ASCII characters except a double quote.
let char-s be the set of all ASCII characters except a single quote.
let digit be the set of digits from 0 to 9.

str_const = ' char-s* ' | " char-d* " note: a string constant may not extend beyond a line. Thus if a string constant that is started and not terminated prior to the end of a line is considered to be in error.
int_const = digit digit* note: while a digit may be of any length, if it causes an overflow it will be considered to be in error. An int_const will be terminated with an operator, a symbol or a space. It may not be terminated with a letter.
real_const = digit digit* '.' digit digit* note: a real constant must contain a decimal point. Its value will be the normal real value of the constant. A real_const will be terminated with an operator, a symbol other than a period, or a space. It may not be terminated with a letter.

Identifiers

let letter be the 52 letters of the alphabet (upper or lower case letters will be considered distinct)
let digit be the 10 digits

identifier = letter (letter | digit)* note: an identifier can have any length. However, only the first 63 are to be significant. An identifier will be terminated with an operator, a symbol or a space.
true and false are predefined identifiers that are not reserved words. They are each of boolean type and have the expected meaning.

Comments and Spaces

Blank characters, tab characters, and end of line characters will all be considered to be spaces. Two or more consecutive spaces are to be considered as a single space. Spaces are not tokens, but may separate tokens. No token may contain an embedded space, thus a space will terminate any token.

A comment may take on either of the following forms:

line comment: '--' will start a line comment. The end of the line will terminate the comment.
block comment: '{' will start a block comment. '}' or the end of the file will end a block comment. Block comments may extend over any number of lines. Block comments may not be nested.

A comment, either line or block, will be treated as a single space.