title: Extract text information from PDF files with different layouts - machine learning
body: <p>I need assistance with a ML project I am currently trying to create.</p>
  
  <p>I receive a lot of invoices from a lot of different suppliers - all in their own unique layout. I need to extract <strong>3</strong> key elements from the invoices. These <strong>3</strong> elements are all located in a table/line items for all the invoices.</p>
  
  <p>The <strong>3</strong> elements are: </p>
  
  <ul>
  <li><strong>1</strong>: Tariff number (digit)</li>
  <li><strong>2</strong>: Quantity (always a digit)</li>
  <li><strong>3</strong>: Total line amount (monetary value)</li>
  </ul>
  
  <p>Please refer to below screenshot, where I have marked these field on a sample invoice.</p>
  
  <p><a href="https://i.stack.imgur.com/HYmGZ.png" rel="noreferrer"><img src="https://i.stack.imgur.com/HYmGZ.png" alt="enter image description here"></a></p>
  
  <p>I started this project with a template approach, based on <em>regular expressions</em>. This, however, was not scaleable at all and I ended up with tons of different rules.</p>
  
  <p>I am hoping that machine learning can help me here - or maybe a hybrid solution?</p>
  
  <h1>The common denominator</h1>
  
  <p>In <strong>all</strong> of my invoices, despite of the different layouts, each line item will <strong>always</strong> consist of one <strong>tariff number</strong>. This tariff number is always 8 digits, and is always formatted in one the ways like below:</p>
  
  <ul>
  <li>xxxxxxxx</li>
  <li>xxxx.xxxx</li>
  <li>xx.xx.xx.xx</li>
  </ul>
  
  <p>(Where "x" is a digit from 0 - 9).</p>
  
  <p><strong>Further</strong>, as you can see on the invoice there is both a Unit Price and a Total Amount per line. The amount I will need is <strong>always</strong> the highest for each line.</p>
  
  <h1>The output</h1>
  
  <p>For each invoice like the one above, I need the output for each line. This could for example be something like this:</p>
  
  <pre><code>{
      "line":"0",
      "tariff":"85444290",
      "quantity":"3",
      "amount":"258.93"
  },
  {
      "line":"1",
      "tariff":"85444290",
      "quantity":"4",
      "amount":"548.32"
  },
  {
      "line":"2",
      "tariff":"76109090",
      "quantity":"5",
      "amount":"412.30"
  }
  </code></pre>
  
  <h1>Where to go from here?</h1>
  
  <p>I am not sure of what I am looking to do falls under machine learning and if so, under which category. Is it computer vision? NLP? Named Entity Recognition?</p>
  
  <p>My initial thought was to:</p>
  
  <ol>
  <li>Convert the invoice to text. (The invoices are all in textable PDFs, so I can use something like <code>pdftotext</code> to get the exact textual values)</li>
  <li>Create custom <strong>named entities</strong> for <code>quantity</code>, <code>tariff</code> and <code>amount</code></li>
  <li>Export the found entities.</li>
  </ol>
  
  <p>However, I feel like I might be missing something. </p>
  
  <p><strong>Can anyone assist me in the right direction?</strong></p>
  
  <h1>Edit:</h1>
  
  <p>Please see below for a few more examples of how an invoice table section can look like:</p>
  
  <p>Sample invoice #2
  <a href="https://i.stack.imgur.com/0dvIC.png" rel="noreferrer"><img src="https://i.stack.imgur.com/0dvIC.png" alt="enter image description here"></a></p>
  
  <p>Sample invoice #3
  <a href="https://i.stack.imgur.com/8Wh6f.png" rel="noreferrer"><img src="https://i.stack.imgur.com/8Wh6f.png" alt="enter image description here"></a></p>
  
  <h1>Edit 2:</h1>
  
  <p>Please see below for the three sample images, <strong>without</strong> the borders/bounding boxes:</p>
  
  <p>Image 1:
  <a href="https://i.stack.imgur.com/5Iln5.png" rel="noreferrer"><img src="https://i.stack.imgur.com/5Iln5.png" alt="Sample 1 without bbox"></a></p>
  
  <p>Image 2:
  <a href="https://i.stack.imgur.com/cEabY.png" rel="noreferrer"><img src="https://i.stack.imgur.com/cEabY.png" alt="Sample 2 without bbox"></a></p>
  
  <p>Image 3:
  <a href="https://i.stack.imgur.com/cSHzh.png" rel="noreferrer"><img src="https://i.stack.imgur.com/cSHzh.png" alt="Sample 3 without bbox"></a></p>