title: Extracting Data from Multiple TXT Files and Creating a Summary CSV File in Python
body: <p>I have a folder with about 50 .txt files containing data in the following format.</p>
  
  <pre><code>=== Predictions on test data ===
  
   inst#     actual  predicted error distribution (OFTd1_OF_Latency)
       1        1:S        2:R   +   0.125,*0.875 (73.84)
  </code></pre>
  
  <p>I need to write a program that combines the following: my index number (i), the letter of the true class (R or S), the letter of the predicted class, and each of the distribution predictions (the decimals less than 1.0).</p>
  
  <p>I would like it to look like the following when finished, but preferably as a .csv file. </p>
  
  <pre><code>ID   True   Pred   S      R
  1    S      R      0.125  0.875
  2    R      R      0.105  0.895
  3    S      S      0.945  0.055
  .    .      .      .      .
  .    .      .      .      .
  .    .      .      .      .
  n    S      S      0.900  0.100
  </code></pre>
  
  <p>I'm a beginner and a bit fuzzy on how to get all of that parsed and then concatenated and appended. Here's what I was thinking, but feel free to suggest another direction if that would be easier. </p>
  
  <pre><code>for i in range(1, n):
     s = str(i)
     readin = open('mydata/output/output'+s+'out','r')
     #The files are all named the same but with different numbers associated
     output = open("mydata/summary.csv", "a")
     storage = []
     for line in readin:
       #data extraction/concatenation here
       if line.startswith('1'):
          id = i
          true = # split at the ':' and take the letter after it
          pred = # split at the second ':' and take the letter after it
           #some have error '+'s and some don't so I'm not exactly sure what to do to get the distributions
          ds = # split at the ',' and take the string of 5 digits before it
          if pred == 'R':
             dr = #skip the character after the comma but take the have characters after
          else: 
             #take the five characters after the comma
          lineholder = id+' , '+true+' , '+pred+' , '+ds+' , '+dr
       else: continue
     output.write(lineholder)
  </code></pre>
  
  <p>I think using the indexes would be another option, but it might complicate things if the spacing is off in any of the files and I haven't checked this for sure. </p>
  
  <p>Thank you for your help!</p>