title: Large distance matrix in clustering
body: <p>I am running R 3.2.3 on a machine with 16 GB RAM. I have a large matrix of 3,00,000 rows x 12 columns. I wanna use a hierarchical clustering algorithm in R, so before I do that, I am trying to create a distance matrix. Since data is of mixed type, I use different matrices for different type. I get an error about memory allocation:</p>
  
  <pre><code>df &lt;- as.data.frame(matrix(rnorm(36*10^5), nrow = 3*10^5))
  d1=as.dist(distm(df[,c(1:2)])/10^5)
  d2=dist(df[,c(3:8)], method = "euclidean") 
  d3= hamming.distance(df[,c(9:12)]%&gt;%as.matrix(.))%&gt;%as.dist(.)
  </code></pre>
  
  <p>I get the following error </p>
  
  <pre><code>&gt; d1=as.dist(distm(df1[,c(1:2)])/10^5)
  Error: cannot allocate vector of size 670.6 Gb
  In addition: Warning messages:
  1: In matrix(0, ncol = n, nrow = n) :
  Reached total allocation of 16070Mb: see help(memory.size)
  2: In matrix(0, ncol = n, nrow = n) :
  Reached total allocation of 16070Mb: see help(memory.size)
  3: In matrix(0, ncol = n, nrow = n) :
  Reached total allocation of 16070Mb: see help(memory.size)
  4: In matrix(0, ncol = n, nrow = n) :
  Reached total allocation of 16070Mb: see help(memory.size)
  &gt; d2=dist(df1[,c(3:8)], method = "euclidean") 
  Error: cannot allocate vector of size 335.3 Gb
  In addition: Warning messages:
  1: In dist(df1[, c(3:8)], method = "euclidean") :
   Reached total allocation of 16070Mb: see help(memory.size)
  2: In dist(df1[, c(3:8)], method = "euclidean") :
  Reached total allocation of 16070Mb: see help(memory.size)
  3: In dist(df1[, c(3:8)], method = "euclidean") :
  Reached total allocation of 16070Mb: see help(memory.size)
  4: In dist(df1[, c(3:8)], method = "euclidean") :
  Reached total allocation of 16070Mb: see help(memory.size)
  &gt; d3= hamming.distance(df1[,c(9:12)]%&gt;%as.matrix(.))%&gt;%as.dist(.)
  Error: cannot allocate vector of size 670.6 Gb
  In addition: Warning messages:
  1: In matrix(0, nrow = nrow(x), ncol = nrow(x)) :
  Reached total allocation of 16070Mb: see help(memory.size)
  2: In matrix(0, nrow = nrow(x), ncol = nrow(x)) :
  Reached total allocation of 16070Mb: see help(memory.size)
  3: In matrix(0, nrow = nrow(x), ncol = nrow(x)) :
  Reached total allocation of 16070Mb: see help(memory.size)
  4: In matrix(0, nrow = nrow(x), ncol = nrow(x)) :
  Reached total allocation of 16070Mb: see help(memory.size)
  </code></pre>