title: Split a large dataframe into a list of data frames based on common value in column
body: <p>I have a data frame with 10 columns, collecting actions of "users", where one of the columns contains an ID (not unique, identifying user)(column 10). the length of the data frame is about 750000 rows. I am trying to extract individual data frames (so getting a list or vector of data frames) split by the column containing the "user" identifier, to isolate the actions of a single actor.</p>
  
  <pre><code>ID | Data1 | Data2 | ... | UserID
  1  | aaa   | bbb   | ... | u_001
  2  | aab   | bb2   | ... | u_001
  3  | aac   | bb3   | ... | u_001
  4  | aad   | bb4   | ... | u_002
  </code></pre>
  
  <p>resulting into </p>
  
  <pre><code>list(
  ID | Data1 | Data2 | ... | UserID
  1  | aaa   | bbb   | ... | u_001
  2  | aab   | bb2   | ... | u_001
  3  | aac   | bb3   | ... | u_001
  ,
  4  | aad   | bb4   | ... | u_002
  ...)
  </code></pre>
  
  <p>The following works very well for me on a small sample (1000 rows):</p>
  
  <pre><code>paths = by(smallsampleMat, smallsampleMat[,"userID"], function(x) x)
  </code></pre>
  
  <p>and then accessing the element I want by paths[1] for instance.</p>
  
  <p>When applying on the original large data frame or even a matrix representation, this chokes my machine ( 4GB RAM, MacOSX 10.6, R 2.15) and never completes (I know that a newer R version exists, but I believe this is not the main problem). </p>
  
  <p>It seems that split is more performant and after a long time completes, but I do not know ( inferior R knowledge) how to piece the resulting list of vectors into a vector of matrices.</p>
  
  <pre><code>path = split(smallsampleMat, smallsampleMat[,10]) 
  </code></pre>
  
  <p>I have considered also using <code>big.matrix</code> etc, but without much success that would speed up the process.</p>