title: Methods for automated synonym detection
body: <p>I am currently working on a neural network based approach to short document classification, and since the corpuses I am working with are usually around ten words, the standard statistical document classification methods are of limited use. Due to this fact I am attempting to implement some form of automated synonym detection for the matches provided in the training. My question more specifically is about resolving a situation as follows:</p>
  
  <p>Say I have classifications of "Involving Food", and one of "Involving Spheres" and a data set as follows:</p>
  
  <pre><code>"Eating Apples"(Food);"Eating Marbles"(Spheres); "Eating Oranges"(Food, Spheres);
  "Throwing Baseballs(Spheres)";"Throwing Apples(Food)";"Throwing Balls(Spheres)";
  "Spinning Apples"(Food);"Spinning Baseballs";
  </code></pre>
  
  <p>I am looking for an incremental method that would move towards the following linkages:</p>
  
  <pre><code>Eating --&gt; Food
  Apples --&gt; Food
  Marbles --&gt; Spheres
  Oranges --&gt; Food, Spheres
  Throwing --&gt; Spheres
  Baseballs --&gt; Spheres
  Balls --&gt; Spheres
  Spinning --&gt; Neutral
  Involving --&gt; Neutral
  </code></pre>
  
  <p>I do realize that in this specific case these might be slightly suspect matches, but it illustrates the problems I am having. My general thoughts were that if I incremented a word for appearing opposite the words in a category, but in that case I would end up incidentally linking everything to the word "Involving", I then thought that I would simply decrement a word for appearing in conjunction with multiple synonyms, or with non-synonyms, but then I would lose the link between "Eating" and "Food". Does anyone have any clue as to how I would put together an algorithm that would move me in the directions indicated above? </p>