Take the 2-minute tour ×

I'm trying to cause Solr to extract only the second 7 digit portion of a ticket formatted like n-nnnnnnn

Originally I hoped to keep the full ticket together. According to documentation digits with numbers should be kept together but after hammering away a this problem for some time and looking at the code I don't think that's the case. Solr always generates two terms. So rather than large numbers of matches for the first digit of n- I'm thinking I can get better query results from just the second portion. Substituting an A for a dash:

    <charFilter class="solr.PatternReplaceCharFilterFactory"
      pattern="\b\d[A](\d\d\d\d\d\d\d)\b" replacement="$1" replace="all" 

will parse 1A1234567 fine But -\b" replacement="$1" replace="all" maxBlockChars="20000"/>

will not parse 1-1234567

So it looks like just a problem with the hyphen. I've tried -(escaped) and [-] and \u002D and \x{45} and \x045 without success.

I've tried putting char filters around it:

   <charFilter class="solr.MappingCharFilterFactory" mapping="mapping.txt"/>
    <charFilter class="solr.PatternReplaceCharFilterFactory"
      pattern="\b\d[-](\d\d\d\d\d\d\d)\b" replacement="$1" replace="all" maxBlockChars="20000"/>
    <charFilter class="solr.MappingCharFilterFactory" mapping="mapping2.txt"/>

with mappings:

"-" => "z"

and then

"z" => "-"

I looks like the hyphen is eaten up in the Flex tokenization and isn't even available to the char filter.

Has anyone had more success with hyphen/dash in Solr/Lucene? Thanks

share|improve this question
Generally, Filters are applied after the analyzer and tokenizer, so yes, those hyphens are already gone. My question would be: does this field need to be analyzed and tokenized like that? If it's just a ticket number, then I would say make use a KeywordAnalyzer, or perhaps make the field a StrField, or something else along those lines. –  femtoRgon Nov 26 '12 at 19:38
Sounds like it might be easier to do in the application building the solr document. –  d whelan Nov 27 '12 at 3:02
add comment

1 Answer

up vote 1 down vote accepted

If your Solr is using a recent Lucene (3.x+ I think), you will want to use a ClassicAnalyzer rather than a StandardAnalyzer, as StandardAnalyzer now always treats hyphens as a delimiter.

share|improve this answer
Thanks for your help. We're going to also try just using the unanalyzed base string type. –  user1840253 Nov 27 '12 at 16:04
Understanding that is good. However, we have a number of indexed fields and for most of them we would need to use a StandardAnalyzer. We then group them together into one default field for our query portal. Thus the user entered query would still split off the first portion and give lots on unwanted results. I'm still perplexed as to why the charFilters don't work since they are prior to a tokenizer. Adding them still looks necessary. –  user1840253 Nov 27 '12 at 20:58
add comment

Your Answer


