title: Algorithms or libraries for textual analysis, specifically: dominant words, phrases across text, and collection of text
body: <p>I'm working on a project where I need to analyze a page of text and collections of pages of text to determine dominant words.   I'd like to know if there is a library (prefer c# or java) that will handle the heavy lifting for me.  If not, is there an algorithm or multiple that would achieve my goals below.  </p>
  
  <p>What I want to do is similar to word clouds built from a url or rss feed that you find on the web, except I don't want the visualization.  They are used all the time for analyzing the presidential candidate speeches to see what the theme or most used words are.  </p>
  
  <p>The complication, is that I need to do this on thousands of short documents, and then collections or categories of these documents.  </p>
  
  <p>My initial plan was to parse the document out, then filter common words - of, the, he, she, etc..  Then count the number of times the remaining words show up in the text (and overall collection/category).  </p>
  
  <p>The problem is that in the future, I would like to handle stemming, plural forms, etc..   I would also like to see if there is a way to identify important phrases. (Instead of a count of a word, the count of a phrase being 2-3 words together)</p>
  
  <p>Any guidance on a strategy, libraries or algorithms that would help are appreciated.  </p>