title: How to optimize creating of datasets in Pandas?
body: <p>I read the file and group all lines by count of columns to different dataframes:</p>
  <pre><code>dict_of_cols = {}
  dict_of_dfs = {}
  
  with open(filename, 'r', encoding=&quot;utf-8&quot;) as f:
      reader = csv.reader(f, delimiter=data_file_delimiter)
      data= list(reader)
      
  with open(filename, 'r', encoding=&quot;utf-8&quot;) as f:
      while (line := f.readline().strip()):
          cls = line.split(data_file_delimiter)
          if len(cls) in dict_of_cols:
              dict_of_cols[len(cls)].append(cols)
          else:    
              dict_of_cols[len(cls)] = [cls]
          
  for colsKey, colsValue in dict_of_cols.items():
      print(colsValue)
      dict_of_dfs['dt_'+str(colsKey)] = pd.DataFrame(colsValue)
  </code></pre>
  <p>As result I can use any dataframe:</p>
  <p><code>dict_of_dfs['dt_12'].head()</code> this is dataframe based lines with 12 columns</p>
  <p>I got the error when I call prev command:</p>
  <pre><code>IOPub data rate exceeded.
  The notebook server will temporarily stop sending output
  to the client in order to avoid crashing it.
  To change this limit, set the config variable
  `--NotebookApp.iopub_data_rate_limit`.
  
  Current values:
  NotebookApp.iopub_data_rate_limit=1000000.0 (bytes/sec)
  NotebookApp.rate_limit_window=3.0 (secs)
  </code></pre>
  <p>How to optimize this (loops)?</p>