text: By default, it is used to count tokens for all LLMs.You can also use it to count tokens when splitting documents with from langchain.text_splitter import CharacterTextSplitterCharacterTextSplitter.from_huggingface_tokenizer(...)For a more detailed walkthrough of this, see this notebookDatasetsâ€‹The Hugging Face Hub has lots of great datasets that can be used to evaluate your LLM chains.For a detailed walkthrough of how to use them to do so, see this notebookPreviousHologresNextiFixitInstallation and SetupWrappersLLMEmbeddingsTokenizerDatasetsCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.