title: Pandas / Numpy - How do I get and compare the counts of every column with every column and write to csv?
body: <p>I was trying to figure out how to compare the count of every data frame column to every data frame column. There can be over 300 columns, so there would be over 300 X 300 = over 90,000 comparisons. This is some sample data:</p>
  
  <p><a href="https://i.stack.imgur.com/lHD8S.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/lHD8S.png" alt="enter image description here"></a></p>
  
  <p>I was trying to figure out how many 'AGE' 55 were Female, how many 'AGE' 48 were male, how many Male were 'NP' D, etc.</p>
  
  <p>This is part of the code I used:</p>
  
  <pre><code>df_search1 = pd.read_csv("file1.txt",sep='\t', 
  dtype=str,engine='c',header=0,encoding='latin-1')
  
  items = ['AGE','NP','GENDER']
  
  headr = True
  for item1 in items:
      headr = True
      for item2 in items:
          # appends
          df_search1[(df_search1['%s'%item1].notnull()) &amp; (df_search1['%s'%item2].notnull())] 
          [['%s'%item2]].count().to_csv('search_out.txt',header=headr,sep='\t',mode='a')
          headr = False
  </code></pre>
  
  <p>The result I get are similar to this:</p>
  
  <p><a href="https://i.stack.imgur.com/DyjEs.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/DyjEs.png" alt="enter image description here"></a></p>
  
  <p>The results I was looking for were similar to this:
  <a href="https://i.stack.imgur.com/r1j01.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/r1j01.png" alt="enter image description here"></a></p>
  
  <p>Where the first 'AGE', 'NP', 'GENDER' are on one row (just the counts with no column names); the second 'AGE', 'NP', 'GENDER' to be on the next row (just the counts with no column names); and the third 'AGE', 'NP', 'GENDER' to be on one row (just the counts with no column names).</p>
  
  <p>Any help would be greatly appreciated.
  Thanks</p>