Lexical analysis#Token

A token (article: " the "; Pl: " token " ) is a character string which is assigned by a grammar a guy. The token is the basic unit of lexical parser. A token corresponds to the grammar of the parser a terminal symbol.

In the trivial case, tokens are the characters of the original input: For example, the letter A is recognized as a token of type A. Multiple characters can be summarized in the course of lexical analysis to tokens. The string token underlying is called lexeme. Examples: