original_docstring: Removes end characters from word token list.
docstring: Removes end characters from word token list.
docstring_tokens: Removes end characters from word token list .
code: def remove_end_characters(text):
      """Removes end characters from word token list."""
  
      # If text is empty, return None.
      if not text: return None
      # If text is not tokenize, tokenize text.
      if is_tokenized(text):
          was_tokenized = True
          normalized_text = merge_tokens(text)
      else:
          was_tokenized = False
          normalized_text = text
      normalized_text += ' '
      # Replace stopwords with spaces.
      normalized_text = sub(get_end_characters_pattern(), r' ', normalized_text)
      # Then remove multiple adjacent spaces.
      normalized_text = sub(' +', ' ', normalized_text)
      # Then strip text.
      normalized_text = normalized_text.strip()
      if normalized_text[-1] == r'.':
          normalized_text = normalized_text[:-1]
      # If text was tokenized, then re-tokenize.
      if was_tokenized:
          normalized_text = word_tokenize(normalized_text)
      # Return normalized text.
      return normalized_text
code_tokens: def remove_end_characters ( text ) : if not text : return None if is_tokenized ( text ) : was_tokenized = True normalized_text = merge_tokens ( text ) else : was_tokenized = False normalized_text = text normalized_text += ' ' normalized_text = sub ( get_end_characters_pattern ( ) , r' ' , normalized_text ) normalized_text = sub ( ' +' , ' ' , normalized_text ) normalized_text = normalized_text . strip ( ) if normalized_text [ - 1 ] == r'.' : normalized_text = normalized_text [ : - 1 ] if was_tokenized : normalized_text = word_tokenize ( normalized_text ) return normalized_text
short_docstring: Removes end characters from word token list.
short_docstring_tokens: Removes end characters from word token list .
comment: """Removes end characters from word token list."""$SEP$# If text is empty, return None.$SEP$# If text is not tokenize, tokenize text.$SEP$# Replace stopwords with spaces.$SEP$# Then remove multiple adjacent spaces.$SEP$# Then strip text.$SEP$# If text was tokenized, then re-tokenize.$SEP$# Return normalized text.