I’ve been looking at Clang and they define lexer tokens in a way that I thought was clever.
The challenge is: how do you keep a single list of language tokens but use them as both an enum and a list of strings?
If you just #include this file, the preprocessor defines TOK(X) as “” (nothing), so the whole thing becomes an empty file.
Now if they have a token, they can say TokNames[token.kind] to get the string name of that token. It lets them use the token types efficiently, print them out nicely for debugging, and not have to maintain multiple lists of tokens.
This entry was posted Friday, August 24th, 2012 at 2:52 pm	and is filed under Programming. You can skip to the end and leave a response. Pinging is currently not allowed.