title: Designing a smart custom csv-like parser in Java
body: <h1>Problem description</h1>
  
  <p>I want to refactor a parser for a flexible csv like format which describes the columns in the first line. Depending on this information I want the parser to build objects which have simple attributes but also complex ones like a <code>List&lt;String&gt;</code> (space separated), for example <code>Thing</code>s:</p>
  
  <h2>Example data type</h2>
  
  <pre><code>import java.util.List;
  
  public class Thing {
      protected int           foo;
      protected String        bar
      protected List&lt;String&gt;  baz;
  
      public Thing(int foo, String bar, List&lt;String&gt; baz) {
          this.foo = foo;
          this.bar = bar;
          this.baz = baz;
      }
  
      public String toString() {
          return "foo: " + foo + ", bar: " + bar + ", baz: " + baz;
      }
  }
  </code></pre>
  
  <p>The parser's input will be text files with a column line (comma separated) in the first line and the data in the <code>n</code> next lines (comma separated). To simplify testing, I will use <code>Iterator&lt;String&gt;</code> for input lines. This simple test should illustrate what I want to build:</p>
  
  <h2>JUnit test</h2>
  
  <pre><code>// prepare example string iterator
  List&lt;String&gt; lines = new ArrayList&lt;String&gt;();
  lines.add("bar,baz,foo");
  lines.add("yay,quux quuux,17");
  lines.add("hey,qaax qaaax,42");
  
  // test parsed things
  List&lt;Thing&gt; things = ThingBuilder.buildThings(lines.iterator());
  assertNotNull(things);
  assertEquals(2, things.size());
  assertEquals("foo: 17, bar: yay, baz: [quux, quuux]", things.get(0).toString());
  assertEquals("foo: 42, bar: hey, baz: [qaax, qaaax]", things.get(1).toString());
  </code></pre>
  
  <h2>Easiest approach</h2>
  
  <ol>
  <li>read the first line and split it in column names</li>
  <li>read all other lines and do the following with them:
  <ul>
  <li>split the line in tokens</li>
  <li>loop over them:
  <ul>
  <li>for token <code>i</code> do a big <code>switch</code>/<code>else if</code> on column name <code>i</code> to</li>
  <li>transform token <code>i</code></li>
  <li>store the extracted value somewhere</li>
  </ul></li>
  <li>collect everything and build a <code>Thing</code></li>
  </ul></li>
  <li>done.</li>
  </ol>
  
  <p>My problem with this approach is the inner switch. After processing the first line, it should be clear how lines are parsed.</p>
  
  <h2>What I would like</h2>
  
  <p>In a language with closures, I would try the following:</p>
  
  <ol>
  <li>read the first line and split it in column names</li>
  <li>for each column name create a closure which sets the right value for a given token and add it to an <strong>array of parser closures</strong></li>
  <li>read all other lines and do the following with them:
  <ul>
  <li>split the line in tokens</li>
  <li>loop over them:
  <ul>
  <li><strong>call parser closure <code>i</code> with token <code>i</code></strong></li>
  </ul></li>
  <li>collect everything and build a <code>Thing</code></li>
  </ul></li>
  <li>done.</li>
  </ol>
  
  <h1>What I tried</h1>
  
  <p>I have a simple interface for all three token parsers. They are supposed to get a token and inject the generated value in the given <code>ThingBuilder</code>'s cache:</p>
  
  <pre><code>public interface TokenParser {
      public void parse(String token, ThingBuilder builder);
  }
  
  public class FooParser implements TokenParser {
      @Override public void parse(String token, ThingBuilder builder) {
          builder.setFoo(Integer.parseInt(token));
      }
  }
  
  public class BarParser implements TokenParser {
      @Override public void parse(String token, ThingBuilder builder) {
          builder.setBar(token);
      }
  }
  
  import java.util.ArrayList;
  import java.util.List;
  public class BazParser implements TokenParser {
      @Override public void parse(String token, ThingBuilder builder) {
          List&lt;String&gt; baz = new ArrayList&lt;String&gt;();
          for (String s : token.split(" ")) baz.add(s);
          builder.setBaz(baz);
      }
  }
  </code></pre>
  
  <p>My <code>ThingBuilder</code>'s <code>buildThings</code> method is static and creates a <code>ThingBuilder</code> object internally, the constructor gets the first (columns) line. This is also the place where the token parser list is filled. After this the hidden <code>ThingBuilder</code> object is ready and with the following input lines the <code>buildThing</code> method is called repeatedly to create a list of <code>Thing</code>s:</p>
  
  <pre><code>import java.util.ArrayList;
  import java.util.List;
  import java.util.Iterator;
  
  public class ThingBuilder {
  
      // single column parsers
      protected List&lt;TokenParser&gt; columnParsers;
  
      // thing attribute cache
      protected int           fooCache;
      protected String        barCache;
      protected List&lt;String&gt;  bazCache;
  
      // thing attribute cache setter
      public void setFoo(int          foo) { fooCache = foo; }
      public void setBar(String       bar) { barCache = bar; }
      public void setBaz(List&lt;String&gt; baz) { bazCache = baz; }
  
      // cleanup helper method
      protected void cleanup() {
          setFoo(0); setBar(null); setBaz(null);
      }
  
      // statically build a list of things from given lines
      public static List&lt;Thing&gt; buildThings(Iterator&lt;String&gt; lines) {
  
          // prepare builder with the first line
          ThingBuilder builder = new ThingBuilder(lines.next());
  
          // parse things
          List&lt;Thing&gt; things = new ArrayList&lt;Thing&gt;();
          while (lines.hasNext()) {
              things.add(builder.buildThing(lines.next()));
          }
          return things;
      }
  
      // prepares a builder to parse thing lines
      protected ThingBuilder(String columnLine) {
  
          // split line into columns
          String[] columns = columnLine.split(",");
  
          // prepare a parser for each column
          columnParsers = new ArrayList&lt;TokenParser&gt;();
          for (String column : columns) {
              TokenParser parser;
              if      (column.equals("foo")) parser = new FooParser();
              else if (column.equals("bar")) parser = new BarParser();
              else if (column.equals("baz")) parser = new BazParser();
              else throw new RuntimeException("unknown column: " + column);
              columnParsers.add(parser);
          }
      }
  
      // builds a thing from a string
      protected Thing buildThing(String line) {
  
          // split the line in tokens
          String[] tokens = line.split(",");
  
          // let the parsers do the work
          for (int i = 0; i &lt; tokens.length; i++) {
              columnParsers.get(i).parse(tokens[i], this);
          }
  
          // hopefully they're done
          Thing thing = new Thing(fooCache, barCache, bazCache);
          cleanup();
          return thing;
      }
  }
  </code></pre>
  
  <p>This works, but:</p>
  
  <h2>What I don't like about my solution</h2>
  
  <ul>
  <li><strong>It feels complicated</strong>!</li>
  <li><strong>The public cache setter thing</strong>. Only <code>TokenParser</code>s should be allowed to fill the builder cache.</li>
  <li><strong>What if I have more than one column with <code>int</code>'s?</strong> Do I have to build a single parser class for each column or is it possible to use an IntegerParser class more the once? The problem here is, that the parser has to call the right cache setter method.</li>
  </ul>
  
  <p>Thanks in advance for your hints!</p>