text: user3766: I am working on a script to create a correlation matrix and mutual_rank matrix from RNAseq gene counts for 57992 genes in 7027 samples. I have already tested this script for a smaller file (for 10,000 genes) and it is working, however, when I am running this script on my final input file, it is giving the following error:
   corrupt matrix -- dims not not match length (this error comes after gene to 
   gene correlation step)
  
  Script which i am running is:
  E <- read.table("mdat_human_test.txt", header = TRUE, row.names=1)
  
  # gene-to-gene correlation
  C <- cor (t(E))
  write.table(C, "human_correlation_genes_mdat.txt", quote=FALSE, sep="\t", 
  row.names=TRUE)
  
  # correlation rank matrix
  R <- t(apply(1-C, 1, rank))
  write.table(R, "human_rank_genes_mdat.txt", quote=FALSE, sep="\t", 
  row.names=TRUE)
  
  # mutual rank matrix
  M <- sqrt(R * t(R))
  write.table(M, "human_mutual_rank_genes_mdat.txt", quote=FALSE, sep="\t", 
  row.names=TRUE)
  
  q()
  
  
  user77: You can't compute a correlation matrix when you have more than ~46000 rows, since a standard R matrix can have a maximum of 2^31-1 values. Have a look at packages like bigcor. Alternatively, consider if having 3+ billion correlation values is really useful.
  
original_text: 3766: <p>I am working on a script to create a correlation matrix and mutual_rank matrix from RNAseq gene counts for 57992 genes in 7027 samples. I have already tested this script for a smaller file (for 10,000 genes) and it is working, however, when I am running this script on my final input file, it is giving the following error:</p>
  
  <pre><code> corrupt matrix -- dims not not match length (this error comes after gene to 
   gene correlation step)
  </code></pre>
  
  <p>Script which i am running is:</p>
  
  <pre><code>E &lt;- read.table("mdat_human_test.txt", header = TRUE, row.names=1)
  
  # gene-to-gene correlation
  C &lt;- cor (t(E))
  write.table(C, "human_correlation_genes_mdat.txt", quote=FALSE, sep="\t", 
  row.names=TRUE)
  
  # correlation rank matrix
  R &lt;- t(apply(1-C, 1, rank))
  write.table(R, "human_rank_genes_mdat.txt", quote=FALSE, sep="\t", 
  row.names=TRUE)
  
  # mutual rank matrix
  M &lt;- sqrt(R * t(R))
  write.table(M, "human_mutual_rank_genes_mdat.txt", quote=FALSE, sep="\t", 
  row.names=TRUE)
  
  q()
  </code></pre>
   77: <p>You can't compute a correlation matrix when you have more than ~46000 rows, since a standard R matrix can have a maximum of 2^31-1 values. Have a look at packages like <a href="https://www.rdocumentation.org/packages/propagate/versions/1.0-6/topics/bigcor" rel="nofollow noreferrer">bigcor</a>. Alternatively, consider if having 3+ billion correlation values is really useful.</p>
  