<p>If you've got 6GB RAM you've got a 64bit machine, so the easiest solution is probably to just up your RAM.</p>

<p>Otherwise, crosspost of this: <a href="https://scicomp.stackexchange.com/questions/1681/what-is-the-fastest-way-to-calculate-the-largest-eigenvalue-of-a-general-matrix/7487#7487">https://scicomp.stackexchange.com/questions/1681/what-is-the-fastest-way-to-calculate-the-largest-eigenvalue-of-a-general-matrix/7487#7487</a></p>

<p>There has been some good research on this recently. The new approaches use "randomized algorithms" which only require a few reads of your matrix to get good accuracy on the largest eigenvalues. This is in contrast to power iterations which require several matrix-vector multiplications to reach high accuracy.</p>

<p>You can read more about the new research here:</p>

<p><a href="http://math.berkeley.edu/~strain/273.F10/martinsson.tygert.rokhlin.randomized.decomposition.pdf" rel="nofollow noreferrer">http://math.berkeley.edu/~strain/273.F10/martinsson.tygert.rokhlin.randomized.decomposition.pdf</a></p>

<p><a href="http://arxiv.org/abs/0909.4061" rel="nofollow noreferrer">http://arxiv.org/abs/0909.4061</a></p>

<p>This code will do it for you:</p>

<p><a href="http://cims.nyu.edu/~tygert/software.html" rel="nofollow noreferrer">http://cims.nyu.edu/~tygert/software.html</a></p>

<p><a href="https://bitbucket.org/rcompton/pca_hgdp/raw/be45a1d9a7077b60219f7017af0130c7f43d7b52/pca.m" rel="nofollow noreferrer">https://bitbucket.org/rcompton/pca_hgdp/raw/be45a1d9a7077b60219f7017af0130c7f43d7b52/pca.m</a></p>

<p><a href="http://code.google.com/p/redsvd/" rel="nofollow noreferrer">http://code.google.com/p/redsvd/</a></p>

<p><a href="https://cwiki.apache.org/MAHOUT/stochastic-singular-value-decomposition.html" rel="nofollow noreferrer">https://cwiki.apache.org/MAHOUT/stochastic-singular-value-decomposition.html</a></p>

<p>If your language of choice isn't in there you can roll your own randomized SVD pretty easily; it only requires a matrix vector multiplication followed by a call to an off-the-shelf SVD.</p>
Performing PCA on a large dataset
<p>I've got a document classification problem with only 2 classes and my training dataset matrix size, after the CountVectorizer becomes (40,845 X 218,904) (unigram). In the case of considering trigrams, it can reach up to (40845 X 3,931,789). Is there a way to perform PCA on such dataset without getting memory or sparse dataset errors. I'm using python sklearn on an 6GB machine.</p>