title: Python extract json strings from large textfile
body: <p>I am working on a project that requires a large data set. I found a sufficiently large data set ( editions dump at <a href="https://openlibrary.org/developers/dumps" rel="nofollow noreferrer">https://openlibrary.org/developers/dumps</a>, about 5GB) which is formatted</p>
  
  <pre><code>/type/edition   /books/OL10000135M  4   2010-04-24T17:54:01.503315  {"publishers": ["Bernan Press"], "physical_format": "Hardcover", "subtitle": "9th November - 3rd December, 1992", "key": "/books/OL10000135M", "title": "Parliamentary Debates, House of Lords, Bound Volumes, 1992-93", "identifiers": {"goodreads": ["6850240"]}, "isbn_13": ["9780107805401"], "languages": [{"key": "/languages/eng"}], "number_of_pages": 64, "isbn_10": ["0107805405"], "publish_date": "December 1993", "last_modified": {"type": "/type/datetime", "value": "2010-04-24T17:54:01.503315"}, "authors": [{"key": "/authors/OL2645777A"}], "latest_revision": 4, "works": [{"key": "/works/OL7925046W"}], "type": {"key": "/type/edition"}, "subjects": ["Government - Comparative", "Politics / Current Events"], "revision": 4} 
  /type/edition   /books/OL10000179M  4   2010-04-24T17:54:01.503315  {"publishers": ["Stationery Office"], "physical_format": "Hardcover", "subtitle": "26 January - 4 February 1998", "title": "Parliamentary Debates, House of Lords, 1997-98", "isbn_10": ["0107805855"], "identifiers": {"goodreads": ["2862283"]}, "isbn_13": ["9780107805852"], "edition_name": "5th edition", "languages": [{"key": "/languages/eng"}], "number_of_pages": 124, "last_modified": {"type": "/type/datetime", "value": "2010-04-24T17:54:01.503315"}, "latest_revision": 4, "key": "/books/OL10000179M", "authors": [{"key": "/authors/OL2645811A"}], "publish_date": "January 1999", "works": [{"key": "/works/OL7925994W"}], "type": {"key": "/type/edition"}, "subjects": ["Bibliographies, catalogues, discographies", "POLITICS &amp; GOVERNMENT", "Reference works", "Bibliographies &amp; Indexes", "Reference"], "revision": 4}
   etc...
  </code></pre>
  
  <p>and I wan to extract the JSON part (the fifth field).</p>
  
  <p>I am trying to use str.replace() (on a 50 line subset of the big file) but it is being finicky.
  I would think something like this would work, but it is not (nothing is changed/replaced)</p>
  
  <pre><code> with fileinput.input(files=("testData.txt"), inplace=True, backup='.bak') as file:
      for line in file:
              print(line.replace(".*({.*})$", "\1"), end="")
  </code></pre>
  
  <p>I tried to parse it column by column (a regex that identifies each column) but then I ran into something that baffled me. The following code</p>
  
  <pre><code> with fileinput.input(files=("testData.txt"), inplace=True, backup='.bak') as file:
      for line in file:
              print(line.replace("/type/edition\t/books/", "WORK PLZ"), end="")
  </code></pre>
  
  <p>yields</p>
  
  <pre><code> WORK PLZOL10000135M    4   2010-04-24T17:54:01.503315  {"publishers": ["Bernan Press"], "physical_format": "Hardcover", "subtitle": "9th November - 3rd December, 1992", "key": "/books/OL10000135M", "title": "Parliamentary Debates, House of Lords, Bound Volumes, 1992-93", "identifiers": {"goodreads": ["6850240"]}, "isbn_13": ["9780107805401"], "languages": [{"key": "/languages/eng"}], "number_of_pages": 64, "isbn_10": ["0107805405"], "publish_date": "December 1993", "last_modified": {"type": "/type/datetime", "value": "2010-04-24T17:54:01.503315"}, "authors": [{"key": "/authors/OL2645777A"}], "latest_revision": 4, "works": [{"key": "/works/OL7925046W"}], "type": {"key": "/type/edition"}, "subjects": ["Government - Comparative", "Politics / Current Events"], "revision": 4}
   WORK PLZOL10000179M    4   2010-04-24T17:54:01.503315  {"publishers": ["Stationery Office"], "physical_format": "Hardcover", "subtitle": "26 January - 4 February 1998", "title": "Parliamentary Debates, House of Lords, 1997-98", "isbn_10": ["0107805855"], "identifiers": {"goodreads": ["2862283"]}, "isbn_13": ["9780107805852"], "edition_name": "5th edition", "languages": [{"key": "/languages/eng"}], "number_of_pages": 124, "last_modified": {"type": "/type/datetime", "value": "2010-04-24T17:54:01.503315"}, "latest_revision": 4, "key": "/books/OL10000179M", "authors": [{"key": "/authors/OL2645811A"}], "publish_date": "January 1999", "works": [{"key": "/works/OL7925994W"}], "type": {"key": "/type/edition"}, "subjects": ["Bibliographies, catalogues, discographies", "POLITICS &amp; GOVERNMENT", "Reference works", "Bibliographies &amp; Indexes", "Reference"], "revision": 4}
  </code></pre>
  
  <p>but</p>
  
  <pre><code> with fileinput.input(files=("testData.txt"), inplace=True, backup='.bak') as file:
      for line in file:
              print(line.replace("/type/edition\t/books/\w+", "WORK PLZ"), end="")
  </code></pre>
  
  <p>does not do anything. It seems like the \w+ is not picking up the alphanumeric string after the /books/.</p>
  
  <p>Am I doing something wring with my regular expressions? Is there a better way to go about this?</p>