title: Preserving comments in `Text.Parsec.Token` tokenizers
body: <p>I'm writing a source-to-source transformation using <a href="http://hackage.haskell.org/package/parsec" rel="nofollow">parsec</a>, So I have a <code>LanguageDef</code> for my language and I build a <code>TokenParser</code> for it using <code>Text.Parsec.Token.makeTokenParser</code>:</p>
  
  <pre><code>myLanguage = LanguageDef { ...
    commentStart = "/*"
    , commentEnd = "*/"
    ...
  }
  
  -- defines 'stringLiteral', 'identifier', etc...
  TokenParser {..} = makeTokenParser myLanguage
  </code></pre>
  
  <p>Unfortunately since I defined <code>commentStart</code> and <code>commentEnd</code>, each of the parser combinators in the <code>TokenParser</code> is a <em>lexeme</em> parser implemented in terms of <code>whiteSpace</code>, and <code>whiteSpace</code> eats spaces as well as comments.</p>
  
  <p>What is the right way to preserve comments in this situation?</p>
  
  <p>Approaches I can think of:</p>
  
  <ol>
  <li>Don't define <code>commentStart</code> and <code>commentEnd</code>.  Wrap each of the lexeme parsers in another combinator that grabs comments before parsing each token.</li>
  <li>Implement my own version of <code>makeTokenParser</code> (or perhaps use some library that generalizes <code>Text.Parsec.Token</code>; if so, which library?)</li>
  </ol>
  
  <p>What's the done thing in this situation?</p>