title: Make nested loops more efficient?
body: <p>I'm analyzing large sets of data using the following script:</p>
  
  <pre><code>M &lt;- c_alignment 
  c_check &lt;- function(x){
      if (x == c_1) {
          1
      }else{
          0
      }
  }
  both_c_check &lt;- function(x){
      if (x[res_1] == c_1 &amp;&amp; x[res_2] == c_1) {
          1
      }else{
          0
      }
  }
  variance_function &lt;- function(x,y){
      sqrt(x*(1-x))*sqrt(y*(1-y))
  }
  frames_total &lt;- nrow(M)
  cols &lt;- ncol(M)
  c_vector &lt;- apply(M, 2, max)
  freq_vector &lt;- matrix(nrow = sum(c_vector))
  co_freq_matrix &lt;- matrix(nrow = sum(c_vector), ncol = sum(c_vector))
  insertion &lt;- 0
  res_1_insertion &lt;- 0
  for (res_1 in 1:cols){
      for (c_1 in 1:conf_vector[res_1]){
          res_1_insertion &lt;- res_1_insertion + 1
          insertion &lt;- insertion + 1
          res_1_subset &lt;- sapply(M[,res_1], c_check)
          freq_vector[insertion] &lt;- sum(res_1_subset)/frames_total
          res_2_insertion &lt;- 0
          for (res_2 in 1:cols){
              if (is.na(co_freq_matrix[res_1_insertion, res_2_insertion + 1])){
                  for (c_2 in 1:max(c_vector[res_2])){
                      res_2_insertion &lt;- res_2_insertion + 1
                      both_res_subset &lt;- apply(M, 1, both_c_check)
                      co_freq_matrix[res_1_insertion, res_2_insertion] &lt;- sum(both_res_subset)/frames_total
                      co_freq_matrix[res_2_insertion, res_1_insertion] &lt;- sum(both_res_subset)/frames_total
                  }
              }
          }
      }
  }
  covariance_matrix &lt;- (co_freq_matrix - crossprod(t(freq_vector)))
  variance_matrix &lt;- matrix(outer(freq_vector, freq_vector, variance_function), ncol = length(freq_vector))
  correlation_coefficient_matrix &lt;- covariance_matrix/variance_matrix
  </code></pre>
  
  <p>A model input would be something like this:</p>
  
  <pre><code>1 2 1 4 3
  1 3 4 2 1
  2 3 3 3 1
  1 1 2 1 2
  2 3 4 4 2
  </code></pre>
  
  <p>What I'm calculating is the binomial covariance for each state found in <code>M[,i]</code> with each state found in <code>M[,j]</code>. Each row is the state found for that trial, and I want to see how the state of the columns co-vary. </p>
  
  <p>Clarification: I'm finding the covariance of two multinomial distributions, but I'm doing it  through binomial comparisons.</p>
  
  <p>The input is a 4200 x 510 matrix, and the c value for each column is about 15 on average. I know <code>for</code> loops are terribly slow in R, but I'm not sure how I can use the <code>apply</code> function here. If anyone has a suggestion as to properly using <code>apply</code> here, I'd really appreciate it. Right now the script takes several hours. Thanks!</p>