title: Clever ways to read a text file into a pandas dataframe with regex
body: <p>I have a text file that gives me a list of "blobs," each with an index, and then the area of the blobs at each time step that they exist. </p>
  
  <p>I want to get:
  a) the duration, in time steps, of each blob 
  b) the average area of each blob across all times steps for which it exists</p>
  
  <p>The text file looks like this: </p>
  
  <p><a href="https://i.stack.imgur.com/dg2s7.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/dg2s7.png" alt="screen shot of text file I want to get into a data frame"></a></p>
  
  <p>Technically, the duration of each blob is already given in parentheses after the Blob index, but there are 60,000 blobs so I need a way to extract this data and figured if I could get it into a pandas data frame, I could just do a "count" on the areas to get the duration. </p>
  
  <p>However, I am having trouble figuring out the best way to parse the text file into a usable pandas data frame. I think that the Blob X labels need to be an index, somehow, and I'm sure I need to use regex to do that. However, I have never worked with regular expressions and would love some advice.  </p>
  
  <p>So far, I have the following (not that impressive, I know):</p>
  
  <pre><code>import pandas as pd
  
  
  df = pd.read_csv(f, sep="\t")
  df.columns = df.columns.str.strip().str.lower().str.replace(' ', '_')
  print(df)
  </code></pre>
  
  <p>But the only "column" that pandas is finding is the very first Blob 1 (1). Any ideas on how to parse the text file to get it into a usable data frame format? </p>