title: NLP with Python - how to build a corpus, which classifier to use?
body: <p>I’m trying to figure out which direction to take my Python NLP project in, and I’d be very grateful to the SO community for any advice.</p>
  
  <p><strong>Problem:</strong></p>
  
  <p>Let’s say I have 100 .txt files that contain the minutes of 100 meetings held by a decisionmaking body. I also have 100 .txt files of corresponding meeting outcomes, which contain the resolutions passed by this body. The outcomes fall into one of seven categories – 1 – take no action, 2 – take soft action, 3 – take stronger action, 4 – take strongest action, 5 – cancel soft action previously taken, 6 – cancel stronger action previously taken, 7 – cancel strongest action previously taken. Alternatively, this can be presented on a scale from -3 to +3, with 0 signifying no action, +1 signifying soft action, -1 signifying cancellation of soft action previously taken, and so on. </p>
  
  <p><strong>Based on the text of the inputs, I’m interested in predicting which of these seven outcomes will occur.</strong> </p>
  
  <p>I’m thinking of treating this as a form of sentiment analysis, since the decision to take a certain kind of action is basically a sentiment. However, all the sentiment analysis examples I’ve found have focused on positive/negative dichotomies, sometimes adding in neutral sentiment as a category. I haven’t found any examples with more than 3 possible classifications of outcomes – not sure whether this is because I haven’t looked in the right places, because it just isn’t really an approach of interest for whatever reason, or because this approach is a silly idea for some reason of which I’m not yet quite sure. </p>
  
  <p><strong>Question 1.</strong> Should be I approaching this as a form of sentiment analysis, or is there some other approach that would work better? Should I instead treat this as a kind of categorization matter, similar to classifying news articles by topic and training the model to recognize the "topic" (outcome)?  </p>
  
  <p><strong>Corpus:</strong></p>
  
  <p>I understand that I will need to build a corpus for training/test data, and it looks like I have two immediately evident options: </p>
  
  <p>1 – hand-code a CSV file for training data that would contain some key phrases from each input text and list the value of the corresponding outcome on a 7-point scale, similar to what’s been done here: <a href="http://help.sentiment140.com/for-students" rel="nofollow">http://help.sentiment140.com/for-students</a></p>
  
  <p>2 – use the approach Pang and Lee used (<a href="http://www.cs.cornell.edu/people/pabo/movie-review-data/" rel="nofollow">http://www.cs.cornell.edu/people/pabo/movie-review-data/</a>) and put each of my .txt files of inputs into one of seven folders based on outcomes, since the outcomes (what kind of action was taken) are known based on historical data.</p>
  
  <p>The downside to the first option is that it would be very subjective – I would determine which keywords/phrases I think are the most important to include, and I may not necessarily be the best arbiter. The downside to the second option is that it might have less predictive power because the texts are pretty long, contain lots of extraneous words/phrases, and are often stylistically similar (policy speeches tend to use policy words). I looked at Pang and Lee’s data, though, and it seems like that may not be a huge problem, since the reviews they’re using are also not very varied in terms of style. I’m leaning towards the Pang and Lee approach, but I’m not sure if it would even work with more than two types of outcomes.</p>
  
  <p><strong>Question 2.</strong> Am I correct in assuming that these are my two general options for building the corpus? Am I missing some other (better) option? </p>
  
  <p><strong>Question 3.</strong> Given all of the above, which classifier should I be using? I’m thinking maximum entropy would work best; I’ve also looked into random forests, but I have no experience with the latter and really have no idea what I’m doing (yet) when it comes to them. </p>
  
  <p>Thank you very much in advance :)</p>