title: String tokenization in prolog
body: <p>I have the following context free grammar in a text file 'grammar.txt'</p>
  
  <pre><code>S ::= a S b
  S ::= []
  </code></pre>
  
  <p>I'm opening this file and able to read each line in prolog.
  Now i want to tokenize each line and generate a list such as</p>
  
  <pre><code>L=[['S','::=','a','S','b'],['S','::=','#']]  ('#' represents empty)
  </code></pre>
  
  <p>How can i do this?</p>