title: How to get bag of words from textual data?
body: <p>I am working on prediction problem using a large textual dataset. I am implementing <a href="http://en.wikipedia.org/wiki/Bag-of-words_model">Bag of Words Model.</a></p>
  
  <p>What should be the best way to get the bag of words? Right now, I have <a href="http://en.wikipedia.org/wiki/Tf%E2%80%93idf">tf-idf</a> of the various words and the number of words is too large to use it for further assignments. If I use tf-idf criteria, what should be the tf-idf threshold for getting bag of words? Or should I use some other algorithms. I am using python.</p>