human
With respect to tokenization, what is the difference between a token and a word?
assistant
A token is a single character in a string. A word is a sequence of tokens. For example, the word “cat” is a sequence of three tokens: “c”, “a”, and “t”.