title: How do I Import a csv file containing repeated lines of information (blocks) whereby for example a row beginning with 'T' refers to time etc
body: <p>I have a very long csv file with repeated blocks of information, however it's not perfectly regular:</p>
  
  <pre><code>T,2002,12,03,09,22,54
  B,35,77,27,34,190,400,341,3447,940.3,303.5
  G,3229987,41014,25,3447,1784033,21787,16,3447,940.3,303.5
  R,3273751,46609,6452,3447,1810631,45933,6382,3447,940.3,303.5
  D,NBXX,31,4.267,6.833,6.646,2.270,9.975,3.987
  Y,194669,940.3,303.5,298.4,11.6,12.9,5.8,7,0000
  T,2002,12,03,09,27,56
  B,3520252,76702,297,3447,1906319,39865,305,3447,940.4,303.6
  G,3231611,40449,13,3447,1785214,21650,25,3447,940.4,303.6
  R,3273277,46425,6431,3447,1813279,45613,6425,3447,940.4,303.6
  D,NBXX,28,-6.813,4.314,5.826,1.527,2.997,-9.648
  Y,194767,940.4,303.6,298.4,11.4,12.9,5.8,9,0000
  Z,2.782e-5,1.512e-5,1.195e-5,1.415e-5,8.290e-6,1.232e-5,2.319e-5
  T,2002,12,03,09,32,59
  .
  .
  .  
  </code></pre>
  
  <p>the information isn't completely regular and some of the 'D' lines contain or less the normal number of elements e.g. most if not all 'D' lines contain 9 elements - </p>
  
  <pre><code>['D', 'ZBXX', '110', '2.590e-5', '1.393e-5', '1.032e-5e-6']
  ['D', 'ZBXX', '118', '2.641e-5', '1.402e-5', '1.027e-5', '1.237e-5', 
  '6.553e-6', '9.466', '290.9', '6.1', '12.0', '6.2', '7', '0000']
  ['D', 'ZBXX', '110', '2.590e-5', '1.393e-5', '1.032e-5e-6']
  ['D', 'ZBXX', '118', '2.641e-5', '1.402e-5', '1.027e-5', '1.237e-5', 
  '6.553e-6', '9.466', '290.9', '6.1', '12.0', '6.2', '7', '0000']  
  </code></pre>
  
  <p>And I want it to look like:</p>
  
  <pre><code>Time [yy-mm-dd-hh-ss]     D[3]       D[4]   D[5]    D[6]    D[7]    D[8]    Y[4] Y[[5]  
  2002-12-03-09-22-54     4.267       6.833   6.646   2.270   9.975   3.987   303.5   
  2002-12-03-09-27-56     -6.813      4.314   5.826
  
  2002-12-03-09-32-59                             
  </code></pre>
  
  <p>This is the code I have thus far:</p>
  
  <pre><code>year_i=np.array(1999) # Start year
  
  dataframe_rows = []
  for x in range(1,6): # we have 5 files
      # Create the name of file that will change within the loop
      year_str='nef'+str(year_i)
      start='C:\\Users\\'
      end=".dat"
      name_file=start+year_str+end # concat strings
      file_ = open(name_file, 'r+').readlines()
  
      rows = ""    
      for i in range(len(file_)):
          if (file_[i].startswith('Z')): #ignore lines starting with 'Z'
              continue
          string = file_[i]
          if (file_[i].startswith('B')): #ignore lines starting with 'B'
              continue
          string = file_[i]
          if (file_[i].startswith('G')): #ignore lines starting with 'G'
          continue
          string = file_[i]
          if (file_[i].startswith('R')):  #ignore lines starting with 'R'
          continue
          string = file_[i]
          if "T," in string:  
              if len(rows) &gt; 0:
              dataframe_rows.append(rows[:-1])
              rows = ""
          string = file_[i].replace("\n","").replace("\r","")
          string = string[2:].replace(",","-")
          rows += string + ","
  
          #if "D," in string:
          # I want to select certain the last 6 elements and convert them into columns
  
          #if (file_[i].startswith('Y')):
          # I want to select the 3rd, 5th, 6th and last elements and convert them into columns
  
      else:
          string = file_[i].replace("\n","").replace("\r","")
          aux_row += string[2:] + ","
  
  year_i+=1 # counter
  
  fixed_rows = []
  for row in (dataframe_rows):
      if (len(row.split(","))) == 18:
          fixed_rows.append(row)
  
  df = pd.read_csv(io.StringIO('\n'.join(fixed_rows)))
  </code></pre>