(This is the older material - click here - for current specification content)

2. Lexical Analysis (1)

A 'Lexicon' is collection of terms related to a specific subject.

For example you can have a lexicon of embroidery terms or a lexicon of web design terms.

It is just a collection of terms that someone in that specialist area would understand and use in its correct context. For example the word 'frame' means one thing for a web designer but another thing entirely in embroidery circles.

In terms of a compiler, a lexicon is a set of terms, called 'keywords' or 'reserved words' that the compiler is designed to understand.

For example, a keyword might be 'byte' or '=' or '3' (You would expect any compiler to understand numbers and the standard maths signs!).

A compiler of course, is designed to understand the source code of the computer language it is designed for.

The role of the lexical analysis section of a compiler is to convert the incoming character stream (source code) into fixed length binary items (1s and 0s) called 'tokens'.