A.2 Tokens

Bold-faced non-terminals in this grammar are the tokens in the full grammar of A.1. As usual, tokens are defined as the longest possible sequence of characters that are in the language defined by the grammar given below. The meta-notations "one of "..."", "any but x," and "x..y" are used to concisely list a range of alternative characters. space, tab, and newline stand for the corresponding characters.

name	::=	letter {letter | digit} [id_cont]
	|	"_" {"_"} op_name	the first underscore is not part of the msg name
op_name	::=	punct {punct} [id_cont]
	|	"_" {"_"} name	the first underscore is not part of the msg name
id_cont	::=	"_" {"_"} [name | op_name]

integer	::=	[radix] hex_digits	a leading "-" is considered a unary operator
radix	::=	digits "_"
hex_digits	::=	hex_digit {hex_digit}
hex_digit	::=	digit | one of "a..fA..F"

float	::=	integer "." hex_digits [exponent]
	|	integer exponent
exponent	::=	"^" ["+" | "-"] digits
digits	::=	digit {digit}

character	::=	"'" char "'"
string	::=	""" { char | line_break } """
char	::=	any | "\" escape_char
escape_char	::=	one of "abfnrtv'"\?0"
 	|	["o"|"d"] digit [digit [digit]]
	|	["o"] digit [digit [digit]]
	|	"x" hex_digit [hex_digit]
line_break	::=	"\" {whitespace} new_line {whitespace} "\"
			characters between \'s are not part of the string
brace_balanced_chars 	::=
		{any but "{"} ["{" brace_balanced_chars "}" {any but "}"}]

letter	::=	one of "a..zA..Z"
digit	::=	one of "0..9"
punct	::=	one of "!#$%^&*-+=<>/?~\|"