title: pandas group by sum of column size
body: <p>I have multiple huge tsv files which I'm trying to process using pandas. I want to group by 'col3' and 'col5'. I've tried this:</p>
  
  <pre><code>import pandas as pd
  df = pd.read_csv('filename.txt', sep = "\t")
  g1 = df.groupby(['col3', 'col5']).size()
  </code></pre>
  
  <p>It works fine so far and prints an output like this:</p>
  
  <pre><code>yes AB12 1
      FG14 1
  no  nn18 1
      pp76 1
  </code></pre>
  
  <p>I want my output to be like:</p>
  
  <pre><code>yes 2
  no  2
  </code></pre>
  
  <p>I'd like to be able to aggregate the output from multiple files, i.e., to be able to group by these two columns in all the files at once and print one common output with total number of occurrences of 'yes' or 'no' or whatever that attribute could be. </p>
  
  <p>PS- Say I have duplicates in col3 for a given value in col5., I'd like them to be merged into one and not counted twice. In other words, if AB12 has two rows and both the rows have yes in col5, I'd like the script to count it as one and not two. </p>
  
  <p>UPDATE:</p>
  
  <p>I got the duplicates removed by doing:</p>
  
  <pre><code>g2 = df.drop_duplicates(['col3', 'col5'])
  g3 = g2.groupby(['col3', 'col5']).size().sum(level=0)
  print g3
  </code></pre>
  
  <p>I'd now like to use groupby on multiple files at once. And if a file doesn't have one of these columns, it should be skipped and we should go to the next file.  </p>