title: JSoup - Java OutOfMemoryError - using jsoup parser across huge files
body: <p>I am using <a href="http://jsoup.org/" rel="nofollow">jsoup</a> parser to read HTML files and creating XML file with data extracted. I will be running this code/script across a directory which has several html files (~250k - 300k files) and some of the files are large.</p>
  
  <p>I run into a</p>
  
  <pre class="lang-none prettyprint-override"><code>java.lang.OutOfMemoryError: Requested array size exceeds VM limit" or Java heap space
  </code></pre>
  
  <p>error. I have tried running the VM with different <code>-Xmx</code> and <code>-Xms</code> values, but I keep hitting the same error. I have attached my code snippet.
  I am assuming the error occurs when reading a large file. Any thoughts on how to resolve this?</p>
  
  <pre><code>String target_dir_output = "/test/";
  File dir = new File(target_dir);
  File[] files = dir.listFiles();
  for (File f : files) {
      if(f.isFile()) {
          String fileName = f.getName();
          String testValue = null;
          try {
              Document doc = Jsoup.parse(f, "UTF-8", "");
              Elements metalinks = doc.select("meta[name=testValue]");
              testValue = metalinks.first().attr("content");
              String output = "&lt;data&gt;" + "\n";
              output += "&lt;testValue&gt;" + testValue + "&lt;/testValue&gt;" ;
              output += "&lt;/data&gt;";
              FileOutputStream out = new FileOutputStream(fileName + ".xml");
              out.write(output.getBytes());
              out.close();
          } catch (IOException e) {
              e.printStackTrace();
          }
  </code></pre>