title: How do access specific tokens with Java's StringTokenizer?
body: <p>I'm using Buffered Reader to pass individual lines of a file to Java's <code>StringTokenizer</code>. The file is structurd as follows: </p>
  
  <pre><code>"2,0";"12345";"foo";"foo.doc"
  "2,4";"23456";"foo";"foo.doc";"34567";"foo7";"foo7.doc";"45678";"foo6";"foo6.doc";"56789";"foo5";"foo5.doc";"67890";"foo4";"foo4.doc"   
  "3,0";"34567";"foo7";"foo7.doc"
  "3,0";"45678";"foo6";"foo6.doc"
  "3,0";"56789";"foo5";"foo5.doc"
  "3,0";"67890";"foo4";"foo4.doc"
  </code></pre>
  
  <p>Here's the code I'm using--so far.</p>
  
  <pre><code>public class parse {
    public static void main(String args[]) {
      FileInputStream inputStream = new FileInputStream("whidata0.txt");
      BufferedReader br = new BufferedReader(new InputStreamReader(inputStream)); 
      while((scrubbedInput=br.readLine())!=null) {
        StringTokenizer strTok = new StringTokenizer(scrubbedInput, ";", false);
        int tokens = strTok.countTokens();
        while (strTok.hasMoreTokens()) {
          tok01 = strTok.nextToken();
        }
        System.out.println("  scrubbed: " + scrubbedInput);
        System.out.println("    tokens: " + tokens);
        System.out.println("     tok01: " + tok01);
      }
    }
  }
  </code></pre>
  
  <p>I need to be able to assign each token in a string to a variable to do additional manipulation. However, if I assign those variable in my <code>while</code> loop, the iteration will overwrite my variables, and they will all return with the same value.</p>
  
  <p>I'm trying to devide a way to do the following:</p>
  
  <pre><code>String token01 = strTok.tokenNumber(0);
  String token02 = strTok.tokenNumber(1);
  String token03 = strTok.tokenNumber(2);
  String token04 = strTok.tokenNumber(3);
  etc.
  </code></pre>
  
  <p>but cannot find any methods in the String Tokenizer documentation that will allow that. I can certainly write each line to a String array of <code>thisLineOfTokens[]</code> and use a for loop to create <code>String tokenN = thisLineOfTokens[n]</code>, but is there a more direct way to access specific tokens?</p>
  
  <p>I'm kinda lost about the best way to reference a SPECIFIC token from my string.</p>