title: Pandas: DataFrame.sum() or DataFrame().as_matrix.sum()
body: <p>I am writing a function that computes conditional probability all columns in a pd.DataFrame that has ~800 columns. I wrote a few versions of the function and found a very big difference in compute time over two primary options:</p>
  
  <pre><code>col_sums = data.sum()   #Simple Column Sum over 800 x 800 DataFrame
  </code></pre>
  
  <p><strong>Option #1:</strong> 
  {'col_sums' and 'data' are a Series and DataFrame respectively}</p>
  
  <p>[This is contained within a loop over index1 and index2 to get all combinations]</p>
  
  <pre><code>joint_occurance = data[index1] * data[index2]
  sum_joint_occurance = joint_occurance.sum()
  max_single_occurance = max(col_sum[index1], col_sum[index2])
  cond_prob = sum_joint_occurance / max_single_occurance #Symmetric Conditional Prob
  results[index1][index2] = cond_prob
  </code></pre>
  
  <p>Vs. </p>
  
  <p><strong>Option #2:</strong> [While looping over index1 and index2 to get get all combinations]
  Only Difference is instead of using DataFrame I exported the data_matrix to a np.array prior to looping</p>
  
  <pre><code>new_data = data.T.as_matrix() [Type: np.array]
  </code></pre>
  
  <p>Option #1 Runtime is ~1700 sec
  Option #2 Runtime is ~122 sec</p>
  
  <p><strong>Questions:</strong></p>
  
  <ol>
  <li>Is converting the contents of DataFrames to np.array's best for computational tasks? </li>
  <li>Is the .sum() routine in pandas significantly different to to .sum() routine in NumPy or is the difference in speed due to the label access to data? </li>
  <li>Why are these runtimes so different?</li>
  </ol>