title: open, apply function and merge different files from a folder?
body: <p>so here's my problem:</p>
  <p>a have the following architecture:</p>
  <pre><code>Repository: 
        -Sample A folder:
                         -file_A_quant.csv
                         -file_A_isdb.tsv
                         -WORKSPACE_A folder:
                                      -file_A_sir.tsv
        -Sample B folder:
                         -file_B_quant.csv
                         -file_B_isdb.tsv
                         -WORKSPACE_B folder:
                                      -file_B_sir.tsv
         -Sample N folder:
                         -file_N_quant.csv
                         -file_N_isdb.tsv
                         -WORKSPACE_N:
                                      -file_N_sir.tsv
  </code></pre>
  <p>I am able to apply a particular function to certain of the files in each folder, they're recognized by a particular suffix. i do this using this kind of function:</p>
  <p>def ind_quant_table(repository_path, quant_suffix):</p>
  <pre><code>for r, d, f in os.walk(repository_path):
      for file in (f for f in f if f.endswith(quant_suffix)):
              
              complete_file_path =r+'/'+file 
              df = pd.read_csv(complete_file_path)
  
              df.rename(columns = lambda x: x.replace(' Peak area', ''),inplace=True)
              df.rename(columns = lambda x: x.replace(file_extention, ''),inplace=True)
              df.drop(list(df.filter(regex = 'Unnamed:')), axis = 1, inplace = True)
              df.sort_index(axis=1, inplace=True)
  
              prefix = 'treated_'
              df.to_csv(r+'/'+prefix+file, sep =',')
  </code></pre>
  <p>basically, with the function, I process the data in the way I want and then save it as a new file.</p>
  <p>Now, in the same way i treat the other files, and i get individual files.</p>
  <p>HOW can i, <strong>for each sample folder</strong>, treat the individual files and merge them and then save one unique file?. The files shared a common column that can be used to merge the information.</p>
  <p>so far i tried something like this, but it does not work:</p>
  <p>def ind_quant_table_f(repository_path, quant_suffix, isdb_suffix, sir_suffix):</p>
  <pre><code>for r, d, f in os.walk(repository_path):
      for file in (f for f in f if f.endswith(quant_table_suffix)):
              
              complete_file_path =r+'/'+file 
              df = pd.read_csv(complete_file_path)
              'do something to the df'
              df 
  
   for r, d, f in os.walk(repository_path):
        for file in (f for f in f if f.endswith(isdb_suffix)):
                                  
               complete_file_path =r+'/'+file 
               dfisdb = pd.read_csv(complete_file_path)          
               'do something to the dfisdb'
               df = pd.merge (df, dfisdb)
    
   for r, d, f in os.walk(repository_path):
        for file in (f for f in f if f.endswith(sir_suffix)):
                                  
               complete_file_path =r+'/'+file 
               dfsii = pd.read_csv(complete_file_path)          
               'do something to the dfsir'
               df = pd.merge (df, dfsir)
                
              prefix = 'treated_'
              df.to_csv(r+'/'+prefix+file, sep =',')
  </code></pre>
  <p>i would appreciate your ideas!</p>
  <p>Luis</p>