title: clustering on very large sparse matrix?
body: <p>I am trying to do some (k-means) clustering on a very large matrix.</p>
  
  <p>The matrix is approximately 500000 rows x 4000 cols yet very sparse (only a couple of "1" values per row). I want to get around 2000 clusters.</p>
  
  <p>I got two questions:
  - Can someone recommend an open source platform or tool for doing that (maybe using k-means, maybe with something better)?
  - How can I best estimate the time the algorithm will need to finish? I tried weka once, but aborted the job after a couple of days because I couldn't tell how much time it would take.</p>
  
  <p>Thanks!</p>