title: Is there a more elegant way to read in CSV columns and merge with record IDs?
body: <p>Forgive me if this simple. I'm new to Python and self-taught.</p>
  
  <p>I have a folder full of CSV files. Each file represent one record and contains one column (among 5 total columns in each file with no headers) that I want to aggregate into a single dataframe containing all records. In the final dataframe I want the first column to be the record numbers and then the remaining columns to be the values from the files. Some records will have more or fewer values, so most records will have NaN in some columns.</p>
  
  <p>I've solved the problem, but I am curious if there is a more elegant solution. In particular, the way I create a list of dfs, append dfs to the list, concatenate them, transpose them, and then concatenate that with the record numbers seems a little clunky. Here is the code I have:</p>
  
  <pre><code>import pandas as pd
  import numpy as np
  
  start_record = 64090
  end_record = 64092
  record_range = pd.Series(data = np.arange(start_record, end_record + 1))
  record_df = pd.DataFrame(data = record_range, columns = ["record_num"])
  
  lst = []
  for record in record_range:
      profile_df = pd.read_csv("record-" + str(record) + ".csv", names = ["second","val1","val2","val3","val4"])
      lst.append(profile_df["val1"])
  
  profiles_df = pd.concat(lst,axis = 1,)
  profiles_df_t = profiles_df.T
  profiles_df_t = profiles_df_t.reset_index(drop=True)
  total = pd.concat([record_df,profiles_df_t], axis = 1)
  </code></pre>
  
  <p>How would you handle this?</p>