question: private void flushUnigram ( ) { clearAttributes ( ) ; char termBuffer [ ] = termAtt . resizeBuffer ( 2 ) ; int len = Character . toChars ( buffer [ index ] , termBuffer , 0 ) ; termAtt . setLength ( len ) ; offsetAtt . setOffset ( startOffset [ index ] , endOffset [ index ] ) ; typeAtt . setType ( SINGLE_TYPE ) ; index ++ ; }
target:  Flushes a unigram token to output from our buffer. This happens when we encounter isolated CJK characters, either the whole CJK string is a single character, or we encounter a CJK character surrounded  by space, punctuation, english, etc, but not beside any other CJK.