title: Extracting tables from a pdf
body: <p>I'm trying to get the data from the tables in this <a href="https://www.dropbox.com/s/y3nivxhjvvzva7d/test1.pdf?dl=0" rel="nofollow noreferrer">PDF</a>. I've tried pdfminer and pypdf with a little luck but I can't really get the data from the tables.</p>
  
  <p>This is what one of the tables looks like:
  <img src="https://i.stack.imgur.com/3kgtx.png" alt="enter image description here"></p>
  
  <p>As you can see, some columns are marked with an 'x'. I'm trying to this table into a list of objects. </p>
  
  <p>This is the code so far, I'm using pdfminer now. </p>
  
  <pre><code># pdfminer test
  from pdfminer.pdfdocument import PDFDocument
  from pdfminer.pdfparser import PDFParser
  from pdfminer.pdfinterp import PDFResourceManager, PDFPageInterpreter
  from pdfminer.pdfdevice import PDFDevice, TagExtractor
  from pdfminer.pdfpage import PDFPage, PDFTextExtractionNotAllowed
  from pdfminer.converter import XMLConverter, HTMLConverter, TextConverter, PDFPageAggregator
  from pdfminer.cmapdb import CMapDB
  from pdfminer.layout import LAParams, LTTextBox, LTTextLine, LTFigure, LTImage
  from pdfminer.image import ImageWriter
  from cStringIO import StringIO
  import sys
  import os
  
  
  def pdfToText(path):
      rsrcmgr = PDFResourceManager()
      retstr = StringIO()
      codec = 'utf-8'
      laparams = LAParams()
      device = TextConverter(rsrcmgr, retstr, codec=codec, laparams=laparams)
      fp = file(path, 'rb')
      interpreter = PDFPageInterpreter(rsrcmgr, device)
      password = ''
      maxpages = 0
      caching = True
      pagenos = set()
  
      records = []
      i = 1
      for page in PDFPage.get_pages(fp, pagenos, maxpages=maxpages, password=password,
                                    caching=caching, check_extractable=True):
          # process page
          interpreter.process_page(page)
  
          # only select lines from the line containing 'Tool' to the line containing "1 The 'All'"
          lines = retstr.getvalue().splitlines()
  
          idx = containsSubString(lines, 'Tool')
          lines = lines[idx+1:]
          idx = containsSubString(lines, "1 The 'All'")
          lines = lines[:idx]
  
          for line in lines:
              records.append(line)
          i += 1
  
      fp.close()
      device.close()
      retstr.close()
  
      return records
  
  
  def containsSubString(list, substring):
      # find a substring in a list item
      for i, s in enumerate(list):
          if substring in s:
              return i
      return -1
  
  
  # process pdf
  fn = '../test1.pdf'
  ft = 'test.txt'
  
  text = pdfToText(fn)
  outFile = open(ft, 'w')
  for i in range(0, len(text)):
      outFile.write(text[i])
  outFile.close()
  </code></pre>
  
  <p>That produces a text file and it gets all of the text but, the x's don't have the spacing preserved. The output looks like this:
  <img src="https://i.stack.imgur.com/pmV2O.png" alt="enter image description here"></p>
  
  <p>The x's are just single spaced in the text document</p>
  
  <p>Right now, I'm just producing text output but my goal is to produce an html document with the data from the tables. I've been searching for OCR examples, and most of them seem confusing or incomplete. I'm open to using C# or any other language that might produce the results I'm looking for.</p>
  
  <p><strong>EDIT:</strong> There will be multiple pdfs like this that I need to get the table data from. The headers will be the same for all pdfs (s far as I know).</p>