title: How to avoid running out of memory reading a complex PDF via iText7?
body: <p>I'm using iText7 and Java to read PDFs that are not very large (10-30MB), but they contain a massive number of objects, causing <code>OutOfMemoryError</code> problems when creating and using a <code>PdfDocument</code>. (The internal <code>xref</code> table and <code>Map/Tree/Pdf[Dict/Array]</code> objects are in the millions.)</p>
  
  <p>For example, a single PDF might only be 33MB but it contains a single table with a million rows spanning 800 pages, and the bookkeeping inside of <code>PdfDocument</code> is blowing up to 400MB. Here's the sample code and heap dump:</p>
  
  <pre><code>public static void main(String[] args) throws Exception {
  
      // PDF file is 33MB on disk (one big table over 800 pages)
      File pdf = new File("big.pdf"); // Also tried InputStream
  
      PdfReader reader = new PdfReader(pdf); // 35MB heap
  
      PdfDocument document = new PdfDocument(reader); // 400MB+ heap
  
      // do stuff ... assuming we didn't get an OOM above
  }
  </code></pre>
  
  <p><img src="https://i.stack.imgur.com/Q9LKq.jpg" alt="Image of heap dump from a complex PdfDocument"></p>
  
  <p>We added more memory to the JVM, but we don't know how big/complex some of these PDFs might be, so a long-term solution is needed, ideally one that lets us read contents in pieces or in an event-like callback manner (like XML+STAX/SAX).</p>
  
  <p>Is there a more efficient way to either stream the PDF or break it up into sub <code>PdfDocuments</code> given a file or <code>InputStream</code>? We want to locate and extract objects like forms, tables, tooltips, etc.</p>
  
  <p><strong>Update:</strong> I got in contact with the IText team and IText7 doesn't allow partial readings of PDFs. So there isn't much I can do in this case except add more RAM or pre-parse the PDF mysql and look for "too much data" (a lot of work). I also checked <code>PDFBox</code> and it suffers from the same problem.</p>