title: Why can't I access the first token returned from Java's StringTokenizer?
body: <p>I'm using Buffered Reader to pass individual lines of a file to Java's <code>StringTokenizer</code>. The file is structurd as follows: </p>
  
  <pre><code>"2,0";"foo";"foo.doc";"12345"
  "2,4";"foo";"foo.doc";"34567";"foo7";"foo7.doc";"45678";"foo6";"foo6.doc";"56789";"foo5";"foo5.doc";"67890";"foo4";"foo4.doc";"23456"   
  "3,0";"foo7";"foo7.doc";"34567"
  "3,0";"foo6";"foo6.doc";"45678"
  "3,0";"foo5";"foo5.doc";"56789"
  "3,0";"foo4";"foo4.doc";"67890"
  </code></pre>
  
  <p>Here's the code I'm using.</p>
  
  <pre><code>public class parse {
    public static void main(String args[]) {
      FileInputStream inputStream = new FileInputStream("whidata0.txt");
      BufferedReader br = new BufferedReader(new InputStreamReader(inputStream)); 
      while((scrubbedInput=br.readLine())!=null) {
        StringTokenizer strTok = new StringTokenizer(scrubbedInput, ";", false);
        int tokens = strTok.countTokens();
        while (strTok.hasMoreTokens()) {
          tok01 = strTok.nextToken();
        }
        System.out.println("  scrubbed: " + scrubbedInput);
        System.out.println("    tokens: " + tokens);
        System.out.println("     tok01: " + tok01);
      }
    }
  }
  </code></pre>
  
  <p>which yields this result.</p>
  
  <pre><code>scrubbed: "2,0";"foo";"foo.doc";"12345" 
    tokens: 4
     tok01: 12345  scrubbed: "2,4";"foo";"foo.doc";"34567";"foo7";"foo7.doc";"45678";"foo6";"foo6.doc";"56789";"foo5";"foo5.doc";"67890";"foo4";"foo4.doc";"23456"    
    tokens: 16
     tok01: 23456
  scrubbed: "3,0";"foo7";"foo7.doc";"34567"
    tokens: 4
     tok01: 34567
  scrubbed: "3,0";"foo6";"foo6.doc";"45678"
    tokens: 4
     tok01: 45678
  scrubbed: "3,0";"foo5";"foo5.doc";"56789"
    tokens: 4
     tok01: 56789
  scrubbed: "3,0";"foo4";"foo4.doc";"67890"               
    tokens: 4
     tok01: 67890
  </code></pre>
  
  <p>When using <code>nextToken()</code> what is the starting token supposed to be? It appears as though <code>StringTokenizer</code> starts with token <code>0</code>, so that the <code>nextToken()</code> is actually token <code>1</code> -- the second physical token. I did not see a <code>firstToken()</code> method in Java documentation, nor did I see a way to assign specific tokens to specific variables (e.g., <code>String myToken = strTok.tokenNumber(0)</code> etc.). What do I need to do to access the first physical token in my String? </p>