title: Concatenate Streams or Iterables: StackOverflow
body: <p>Recently on my job I had to process a bunch of xml files sequentially. In this case I wrote a sort of tree walk, so each xml file became an Iterator&lt;SomeXmlElement&gt;. </p>
  
  <p>Later on the program didn't care about which SomeXmlElement object came from which file, so I wanted to concatenate all the Iterators into one.</p>
  
  <p>This is roughly how the concat'ing was done (using String instead of SomeXmlElement):</p>
  
  <pre class="lang-java prettyprint-override"><code>Stream&lt;String&gt; s = Stream.empty();
  for (int i = 0; i &lt; 100000; i++) {
    s = Stream.concat(s, Arrays.asList("1", "2", "3").stream());
  }
  s.findFirst().ifPresent(System.out::println);
  </code></pre>
  
  <p>It turns out that this never prints anything, it just hangs for a while and eventually you get heap error or stack overflow. So I tried again, this time using guava:</p>
  
  <pre class="lang-java prettyprint-override"><code>Iterable&lt;String&gt; s = Collections.emptyList();
  for (int i = 0; i &lt; 100000; i++) {
    s = Iterables.concat(s, Arrays.asList("1", "2", "3"));
  }
  System.out.println(Iterables.getFirst(s, null));
  </code></pre>
  
  <p>Somewhat surprisingly, this also throws a StackOverflow. In the end I had to do the concat'ing manually, by implementing Iterator, and that finally worked as expected.</p>
  
  <p>Why do the concatenate methods of these standard libraries fail when there's enough data? Streams and Iterables are designed to handle even infinite input, after all. Is there an easy alternative, apart from the "hard way" of implementing Iterator?</p>