Programming, Compilers, Interpreters, Programming Languages.
its type (is it a keyword? an identifier? an operator? …) and the position (line and/or column number) in the source code where it appears. Sequence of lexemes generated during lexical analysis If the compiler encounters a string of characters for which it cannot create a token, it will stop its