title: Entity Extraction/Recognition with free tools while feeding Lucene Index
body: <p>I'm currently investigating the options to extract person names, locations, tech words and categories from text (a lot articles from the web) which will then feeded into a Lucene/ElasticSearch index. The additional information is then added as metadata and should increase precision of the search. </p>
  
  <p>E.g. when someone queries 'wicket' he should be able to decide whether he means the cricket sport or the Apache project. I tried to implement this on my own with minor success so far. Now I found a lot tools, but I'm not sure if they are suited for this task and which of them integrates good with Lucene or if precision of entity extraction is high enough.</p>
  
  <ul>
  <li><a href="http://dbpedia.org/spotlight" rel="nofollow noreferrer">Dbpedia Spotlight</a>, the <a href="http://spotlight.dbpedia.org/demo/index.xhtml" rel="nofollow noreferrer">demo</a> looks very promising</li>
  <li><a href="http://incubator.apache.org/opennlp/index.html" rel="nofollow noreferrer">OpenNLP</a> requires <a href="https://stackoverflow.com/questions/6952512/how-i-train-an-named-entity-reconigzer-identifier-in-opennlp">training</a>. Which training data to use?</li>
  <li><a href="http://opennlp.sourceforge.net/projects.html" rel="nofollow noreferrer">OpenNLP tools</a></li>
  <li><a href="http://incubator.apache.org/stanbol/" rel="nofollow noreferrer">Stanbol</a></li>
  <li><a href="http://www.nltk.org/download" rel="nofollow noreferrer">NLTK</a></li>
  <li><a href="http://balie.sourceforge.net/" rel="nofollow noreferrer">balie</a></li>
  <li><a href="http://uima.apache.org/" rel="nofollow noreferrer">UIMA</a></li>
  <li><a href="http://gate.ac.uk/" rel="nofollow noreferrer">GATE</a> -> <a href="http://gate.ac.uk/wiki/code-repository/" rel="nofollow noreferrer">example code</a></li>
  <li><a href="http://mahout.apache.org/" rel="nofollow noreferrer">Apache Mahout</a></li>
  <li><a href="http://nlp.stanford.edu/software/CRF-NER.shtml" rel="nofollow noreferrer">Stanford CRF-NER</a></li>
  <li><a href="http://code.google.com/p/maui-indexer" rel="nofollow noreferrer">maui-indexer</a></li>
  <li><a href="http://mallet.cs.umass.edu/" rel="nofollow noreferrer">Mallet</a></li>
  <li><a href="http://cogcomp.cs.illinois.edu/page/software_view/4" rel="nofollow noreferrer">Illinois Named Entity Tagger</a> Not open source but free</li>
  <li><a href="http://code.google.com/p/wikipedianerdata/source/browse/trunk/src/main/entityExtractor/EntityExtractionManager.java?r=2" rel="nofollow noreferrer">wikipedianer data</a></li>
  </ul>
  
  <p><strong>My questions:</strong></p>
  
  <ul>
  <li>Does anyone have experience with some of the listed tools above and its precision/recall? Or if there is training data required + available.</li>
  <li>Are there articles or tutorials where I can get started with entity extraction(NER) for each and every tool?</li>
  <li>How can they be integrated with Lucene?</li>
  </ul>
  
  <p>Here are some questions related to that subject:</p>
  
  <ul>
  <li><a href="https://stackoverflow.com/questions/5544475/does-an-algorithm-exist-to-help-detect-the-primary-topic-of-an-english-sentence">Does an algorithm exist to help detect the &quot;primary topic&quot; of an English sentence?</a></li>
  <li><a href="https://stackoverflow.com/questions/188176/named-entity-recognition-libraries-for-java">Named Entity Recognition Libraries for Java</a></li>
  <li><a href="https://stackoverflow.com/questions/5571519/named-entity-recognition-with-java">Named entity recognition with Java</a></li>
  </ul>