title: How to extract key phrases from a given text with OpenNLP?
body: <p>I'm using Apache OpenNLP and i'd like to extract the Keyphrases of a given text. I'm already gathering entities - but i would like to have Keyphrases.</p>
  
  <p>The problem i have is that i can't use TF-IDF cause i don't have models for that and i only have a single text (not multiple documents)</p>
  
  <p>Here is some code (prototyped - not so clean)</p>
  
  <pre><code> public List&lt;KeywordsModel&gt; extractKeywords(String text, NLPProvider pipeline) {
  
          SentenceDetectorME sentenceDetector = new SentenceDetectorME(pipeline.getSentencedetecto("en"));
          TokenizerME tokenizer = new TokenizerME(pipeline.getTokenizer("en"));
          POSTaggerME posTagger = new POSTaggerME(pipeline.getPosmodel("en"));
          ChunkerME chunker = new ChunkerME(pipeline.getChunker("en"));
  
          ArrayList&lt;String&gt; stopwords = pipeline.getStopwords("en");
  
          Span[] sentSpans = sentenceDetector.sentPosDetect(text);
          Map&lt;String, Float&gt; results = new LinkedHashMap&lt;&gt;();
          SortedMap&lt;String, Float&gt; sortedData = new TreeMap(new MapSort.FloatValueComparer(results));
  
          float sentenceCounter = sentSpans.length;
          float prominenceVal = 0;
          int sentences = sentSpans.length;
          for (Span sentSpan : sentSpans) {
              prominenceVal = sentenceCounter / sentences;
              sentenceCounter--;
              String sentence = sentSpan.getCoveredText(text).toString();
              int start = sentSpan.getStart();
              Span[] tokSpans = tokenizer.tokenizePos(sentence);
              String[] tokens = new String[tokSpans.length];
              for (int i = 0; i &lt; tokens.length; i++) {
                  tokens[i] = tokSpans[i].getCoveredText(sentence).toString();
              }
              String[] tags = posTagger.tag(tokens);
              Span[] chunks = chunker.chunkAsSpans(tokens, tags);
              for (Span chunk : chunks) {
                  if ("NP".equals(chunk.getType())) {
                      int npstart = start + tokSpans[chunk.getStart()].getStart();
                      int npend = start + tokSpans[chunk.getEnd() - 1].getEnd();
                      String potentialKey = text.substring(npstart, npend);
                      if (!results.containsKey(potentialKey)) {
                          boolean hasStopWord = false;
                          String[] pKeys = potentialKey.split("\\s+");
                          if (pKeys.length &lt; 3) {
                              for (String pKey : pKeys) {
                                  for (String stopword : stopwords) {
                                      if (pKey.toLowerCase().matches(stopword)) {
                                          hasStopWord = true;
                                          break;
                                      }
                                  }
                                  if (hasStopWord == true) {
                                      break;
                                  }
                              }
                          }else{
                              hasStopWord=true;
                          }
                          if (hasStopWord == false) {
                              int count = StringUtils.countMatches(text, potentialKey);
                              results.put(potentialKey, (float) (Math.log(count) / 100) + (float)(prominenceVal/5));
                          }
                      }
                  }
              }
          }
          sortedData.putAll(results);
          System.out.println(sortedData);
          return null;
      }
  </code></pre>
  
  <p>What it basically does is giving me the Nouns back and sorting them by prominence value (where is it in the text?) and counts. </p>
  
  <p>But honestly - this doesn't work soo good. </p>
  
  <p>I also tried it with lucene analyzer but the results were also not so good.</p>
  
  <p>So - how can i achieve what i want to do? I already know of KEA/Maui-indexer etc (but i'm afraid i can't use them because of GPL :( )</p>
  
  <hr>
  
  <p>Also interesting? Which other algorithms can i use instead of TF-IDF?</p>
  
  <p>Example:</p>
  
  <p>This text: <a href="http://techcrunch.com/2015/09/04/etsys-pulling-the-plug-on-grand-st-at-the-end-of-this-month/" rel="nofollow">http://techcrunch.com/2015/09/04/etsys-pulling-the-plug-on-grand-st-at-the-end-of-this-month/</a></p>
  
  <p>Good output in my opinion: Etsy, Grand St., solar chargers, maker marketplace, tech hardware</p>