title: How do you highlight span_near queries in Elasticsearch?
body: <p>I am attempting to highlight a document based on a <code>span_near</code> query in Elasticsearch <strong>2.1.1</strong>, and ES is incorrectly highlighting a term that is not actually a hit because it is outside the span in question.</p>
  
  <p>The steps I'm executing are:</p>
  
  <p><strong>Create the index</strong></p>
  
  <pre><code>curl -XPUT 'http://localhost:9200/twitter/' -d '{
      "mappings": {
          "tweet": {
              "properties": {
                  "message": {
                      "type": "string", 
                      "term_vector": "with_positions_offsets", 
                      "store": true
                  }
              }
          }
      }
  }'
  </code></pre>
  
  <p><strong>Index the document</strong></p>
  
  <pre><code>curl -XPUT 'localhost:9200/twitter/tweet/1?refresh=true' -d '{
      "message" : "A new bonsai tree in the office. Bonsai!"
  }'
  </code></pre>
  
  <p><strong>Search</strong></p>
  
  <pre><code>curl -XGET 'http://localhost:9200/twitter/tweet/_search?pretty' -d '{
      "query" : {
          "span_near" : {
              "clauses" : [
                  {"span_term": {"message": "new"}}, 
                  {"span_term": {"message": "bonsai"}}
              ], 
              "slop": 1, 
              "in_order": false
          }
      }, 
      "highlight": {"fields": {"message": {"type": "plain"}}}
  }'
  </code></pre>
  
  <p>The search above is returning:</p>
  
  <pre><code>{
    "took" : 7,
    "timed_out" : false,
    "_shards" : {
      "total" : 5,
      "successful" : 5,
      "failed" : 0
    },
    "hits" : {
      "total" : 1,
      "max_score" : 0.13561106,
      "hits" : [ {
        "_index" : "twitter",
        "_type" : "tweet",
        "_id" : "1",
        "_score" : 0.13561106,
        "_source":{"message" : "A new bonsai tree in the office. Bonsai!"},
        "highlight" : {
          "message" : [ "A &lt;em&gt;new&lt;/em&gt; &lt;em&gt;bonsai&lt;/em&gt; tree in the office. &lt;em&gt;Bonsai&lt;/em&gt;!" ]
        }
      } ]
    }
  }
  </code></pre>
  
  <p>As you can see, it is incorrectly highlighting the occurrence of "Bonsai" at the end of the field that is not within 1 word of "new". A couple of things to note:</p>
  
  <ol>
  <li>This same exact set of steps produces <strong>correct</strong> highlighting results against Elasticsearch <strong>1.5.2</strong>.</li>
  <li>There is an open bug for <code>span_near</code> queries not working using the Fast Vector Highlighter (FVH) -- <a href="https://github.com/elastic/elasticsearch/issues/5496" rel="nofollow">https://github.com/elastic/elasticsearch/issues/5496</a> -- which is why I'm attempting to use the <code>plain</code> highlighter above</li>
  </ol>
  
  <p>Is there something I'm missing in order to get highlighting to work with <code>span_near</code> queries?</p>