title: How to use standard tokenizer with preserve_original?
body: <p>I created 2 custom analyzers as shown below but both doesnt work as I wanted. 
  here is what i want in my inverted index 
  for example; for the word <code>reb-tn2000xxxl</code> i need to have
  reb, tn2000xxl, reb-tn2000xxxl in my inverted index.</p>
  
  <pre><code>{  
     "analysis":{  
        "filter":{  
           "my_word_delimiter":{  
              "split_on_numerics":"true",
              "generate_word_parts":"true",
              "preserve_original":"true",
              "generate_number_parts":"true",
              "catenate_all":"true",
              "split_on_case_change":"true",
              "type":"word_delimiter"
           }
        },
        "analyzer":{  
           "my_analyzer":{  
              "filter":[  
                 "standard",
                 "lowercase",
                 "my_word_delimiter"
              ],
              "type":"custom",
              "tokenizer":"whitespace"
           },
           "standard_caseinsensitive":{  
              "filter":[  
                 "standard",
                 "lowercase"
              ],
              "type":"custom",
              "tokenizer":"keyword"
           },
           "my_delimiter":{  
              "filter":[  
                 "lowercase",
                 "my_word_delimiter"
              ],
              "type":"custom",
              "tokenizer":"standard"
           }
        }
     }
  }
  </code></pre>
  
  <p>if I use <code>my_analyzer</code> which implements <code>whitespace</code> tokenizer, results looks like below if i check with curl</p>
  
  <pre><code>  curl -XGET "index/_analyze?analyzer=my_analyzer&amp;pretty=true" -d "reb-tn2000xxxl"
  {
    "tokens" : [ {
      "token" : "reb-tn2000xxxl",
      "start_offset" : 0,
      "end_offset" : 14,
      "type" : "word",
      "position" : 0
    }, {
      "token" : "reb",
      "start_offset" : 0,
      "end_offset" : 3,
      "type" : "word",
      "position" : 0
    }, {
      "token" : "rebtn2000xxxl",
      "start_offset" : 0,
      "end_offset" : 14,
      "type" : "word",
      "position" : 0
    }, {
      "token" : "tn",
      "start_offset" : 4,
      "end_offset" : 6,
      "type" : "word",
      "position" : 1
    }, {
      "token" : "2000",
      "start_offset" : 6,
      "end_offset" : 10,
      "type" : "word",
      "position" : 2
    }, {
      "token" : "xxxl",
      "start_offset" : 10,
      "end_offset" : 14,
      "type" : "word",
      "position" : 3
    } ]
  }
  </code></pre>
  
  <p>so here I am missing <code>tn2000xxxl</code>split which can be obtained if I use <code>standard</code> tokenizer instead of <code>whitespace</code> but problem is once I use standard like  <code>my_delimiter</code> custom analyzer is using. I dont have original value in the inverted index. It seems that <code>standard</code> tokinezer and <code>preserve_original</code> filter together doesnt work. I read somewhere that because standard tokenizer already splits on original before filter is applied, that's why original is no longer is the same. but How can I achieve this task to prevent original while splitting like standard tokenizer? </p>
  
  <pre><code>curl -XGET "index/_analyze?analyzer=my_delimiter&amp;pretty=true" -d "reb-tn2000xxxl"
  {  
     "tokens":[  
        {  
           "token":"reb",
           "start_offset":0,
           "end_offset":3,
           "type":"&lt;ALPHANUM&gt;",
           "position":0
        },
        {  
           "token":"tn2000xxxl",
           "start_offset":4,
           "end_offset":14,
           "type":"&lt;ALPHANUM&gt;",
           "position":1
        },
        {  
           "token":"tn",
           "start_offset":4,
           "end_offset":6,
           "type":"&lt;ALPHANUM&gt;",
           "position":1
        },
        {  
           "token":"tn2000xxxl",
           "start_offset":4,
           "end_offset":14,
           "type":"&lt;ALPHANUM&gt;",
           "position":1
        },
        {  
           "token":"2000",
           "start_offset":6,
           "end_offset":10,
           "type":"&lt;ALPHANUM&gt;",
           "position":2
        },
        {  
           "token":"xxxl",
           "start_offset":10,
           "end_offset":14,
           "type":"&lt;ALPHANUM&gt;",
           "position":3
        }
     ]
  }
  </code></pre>