public static void normalizeTokens(final List<List<Token>> tokens,
      final String lang) {
    for (final List<Token> sentence : tokens) {
      Normalizer.convertNonCanonicalStrings(sentence, lang);
      Normalizer.normalizeQuotes(sentence, lang);
      Normalizer.normalizeDoubleQuotes(sentence, lang);
    }
  }
public static void normalizeTokens ( final List < List < Token > > tokens , final String lang ) { for ( final List < Token > sentence : tokens ) { Normalizer . convertNonCanonicalStrings ( sentence , lang ) ; Normalizer . normalizeQuotes ( sentence , lang ) ; Normalizer . normalizeDoubleQuotes ( sentence , lang ) ; } }
Set as value of the token its normalized counterpart. Normalization is done
following languages and corpora (Penn TreeBank, Ancora, Tiger, Tutpenn,
etc.) conventions.

@param tokens
the tokens
@param lang
the language