title: wicket: how to stream a resource from a database
body: <p>I'm trying to generate a sitemap dynamically for a large web site with thousands of pages. </p>
  
  <p>Yes, I have considered generating the sitemap file offline and simply serving it statically, and I might end up doing exactly that. But I think this is a generally useful question: </p>
  
  <p><strong>How can I stream large data from a DB in Wicket?</strong></p>
  
  <p>I followed the instructions at the <a href="https://cwiki.apache.org/WICKET/seo-search-engine-optimization.html" rel="nofollow">Wicket SEO page</a>, and was able to get a dynamic sitemap implementation working using a DataProvider. But it doesn't scale- it runs out of memory when it calls my DataProvider's <code>iterator()</code> method with a <code>count</code> arg equal to the total number of objects I'm returning, rather than iterating over them in chunks.</p>
  
  <p>I think the solution lies somewhere with <code>WebResource/ResourceStreamingRequestTarget</code>. But those classes expect an <code>IResourceStream</code>, which ultimately boils down to providing an <code>InputStream</code> implementation, which deals in bytes, rather than DB records. I wouldn't know how to implement the <code>length()</code> method in such a case, as that would require visiting every record ahead of time to compute the overall length.</p>