title: Python Data Extraction from an Encrypted PDF
body: <p>I am an recent graduate in pure mathematics who only has taken few basic programming courses. I am doing an internship and I have an internal data analysis project. I have to analyze the internal PDFs of the last years. The PDFs are "secured." In other words, they are encrypted. We do not have PDF passwords, even more, we are not sure if passwords exist. But, we have all these documents and we can read them manually. We can print them as well. The goal is to read them with Python because is the language that we have some idea. </p>
  
  <p>First, I tried to read the PDFs with some Python libraries. However, the Python libraries that I found do not read encrypted PDFs. At that time, I could not export the information using Adobe Reader either. </p>
  
  <p>Second, I decided to decrypt the PDFs. I was successful using the Python library pykepdf. <strong><em>Pykepdf</em></strong> works very well! However, the decrypted PDFs cannot be read as well with the Python libraries of the previous point (<strong>PyPDF2</strong> and <strong><em>Tabula</em></strong>). At this time, we have made some improvement because using Adobe Reader I can export the information from the decrypted PDFs, but the goal is to do everything with Python.</p>
  
  <p>The code that I am showing works perfectly with unencrypted PDFs, but not with encrypted PDFs. It is not working with the decrypted PDFs that were gotten with pykepdf as well.</p>
  
  <p>I did not write the code. I found it in the documentation of the Python libraries <strong><em>Pykepdf</em></strong> and <strong><em>Tabula</em></strong>. The PyPDF2 solution was written by <strong><em>Al Sweigart</em></strong> in his book, "<strong><em>Automate the Boring Stuff with Python</em></strong>," that I highly recommend. I also checked that the code is working fine, with the limitations that I explained before.</p>
  
  <p><strong><em>First question, 
  why I cannot read the decrypted files, if the programs work with files that never have been encrypted?</em></strong></p>
  
  <p><strong><em>Second question,
  Can we read with Python the decrypted files somehow? Which library can do it or is impossible? Are all decrypted PDFs extractable?</em></strong></p>
  
  <p>Thank you for your time and help!!!</p>
  
  <p>I found these results using Python 3.7, Windows 10, Jupiter Notebooks, and Anaconda 2019.07.</p>
  
  <pre><code>Python
  
  import pikepdf
  with pikepdf.open("encrypted.pdf") as pdf:
    num_pages = len(pdf.pages)
    del pdf.pages[-1]
    pdf.save("decrypted.pdf")
  
  import tabula
  tabula.read_pdf("decrypted.pdf", stream=True)
  
  import PyPDF2
  pdfFileObj=open("decrypted.pdf", "rb")
  pdfReader=PyPDF2.PdfFileReader(pdfFileObj)
  pdfReader.numPages
  pageObj=pdfReader.getPage(0)
  pageObj.extractText()
  </code></pre>
  
  <p>With Tabula, I am getting the message "the output file is empty."</p>
  
  <p>With PyPDF2, I am getting only '/n'</p>
  
  <p><strong><em>UPDATE 10/3/2019 Pdfminer.six (Version November 2018)</em></strong></p>
  
  <p>I got better results using the solution posted by <strong><em>DuckPuncher</em></strong>. For the decrypted file, I got the labels, but not the data. Same happens with the encrypted file. For the file that has never been encrypted works perfect. <strong><em>As I need the data and the labels of encrypted or decrypted files, this code does not work for me.</em></strong> For that analysis, I used <strong><em>pdfminer.six</em></strong> that is Python library that was released in November 2018. Pdfminer.six includes a library pycryptodome. According to their documentation "<strong><em>PyCryptodome</em></strong> is a self-contained Python package of low-level cryptographic primitives.."</p>
  
  <p>The code is in the stack exchange question: 
  <a href="https://stackoverflow.com/questions/26494211/extracting-text-from-a-pdf-file-using-pdfminer-in-python">Extracting text from a PDF file using PDFMiner in python?</a></p>
  
  <p><strong><em>I would love if you want to repeat my experiment. Here is the description:</em></strong></p>
  
  <p>1) Run the codes mention in this question with any PDF that never has been encrypted.</p>
  
  <p>2) Do the same with a PDF "Secure" (this is a term that Adobe uses), I am calling it the encrypted PDF. Use a generic form that you can find using Google. After you download it, you need to fill the fields. Otherwise, you would be checking for labels, but not fields. The data is in the fields.</p>
  
  <p>3) Decrypt the encrypted PDF using Pykepdf. This will be the decrypted PDF.</p>
  
  <p>4) Run the codes again using the decrypted PDF.</p>
  
  <p><strong><em>UPDATE 10/4/2019 Camelot (Version July 2019)</em></strong></p>
  
  <p>I found the Python library Camelot. Be careful that you need <strong><em>camelot-py 0.7.3.</em></strong></p>
  
  <p>It is very powerful, and works with Python 3.7. Also, it is very easy to use. First, you need also to install <strong><em>Ghostscript</em></strong>. Otherwise, it will not work.
  You need also to install <strong><em>Pandas</em></strong>.  <strong><em>Do not use pip install camelot-py</em></strong>. Instead use <strong><em>pip install camelot-py[cv]</em></strong></p>
  
  <p>The author of the program is Vinayak Mehta. Frank Du shares this code in a YouTube video "Extract tabular data from PDF with Camelot Using Python." </p>
  
  <p>I checked the code and it is working with unencrypted files. <strong><em>However, it does not work with encrypted and decrypted files, and that is my goal</em></strong>.</p>
  
  <p>Camelot is oriented to get tables from PDFs.</p>
  
  <p>Here is the code:</p>
  
  <pre><code>Python
  
  import camelot
  import pandas
  name_table = camelot.read_pdf("uncrypted.pdf")
  type(name_table)
  
  #This is a Pandas dataframe
  name_table[0]
  
  first_table = name_table[0]   
  
  #Translate camelot table object to a pandas dataframe
  first_table.df
  
  first_table.to_excel("unencrypted.xlsx")
  #This creates an excel file.
  #Same can be done with csv, json, html, or sqlite.
  
  #To get all the tables of the pdf you need to use this code.
  for table in name_table:
     print(table.df)
  </code></pre>
  
  <p><strong><em>UPDATE 10/7/2019</em></strong>
  I found one trick. If I open the secured pdf with Adobe Reader, and I print it using Microsoft to PDF, and I save it as a PDF, I can extract the data using that copy. I also can convert the PDF file to JSON, Excel, SQLite, CSV, HTML, and another formats. <strong><em>This is a possible solution to my question. However, I am still looking for an option to do it without that trick because the goal is to do it 100% with Python.</em></strong> I am also concerned that if a better method of encryption is used the trick maybe would not work. Sometimes you need to use Adobe Reader several times to get an extractable copy.</p>
  
  <p><strong><em>UPDATE 10/8/2019. Third question.</em></strong>
  I have now a third question. Do all secured/encrypted pdf are password protected? Why pikepdf is not working? My guess is that the current version of pikepdf can break some type of encryptions but not all of them. 
  @constt mentioned that PyPDF2 can break some type of protection. However, I replied to him that I found an article that PyPDF2 can break encryptions made with Adobe Acrobat Pro 6.0, but no with posterior versions.</p>