---
title: Semanticist Tokenizer
emoji: 🐠
colorFrom: yellow
colorTo: yellow
sdk: gradio
sdk_version: 5.20.1
app_file: tok_demo.py
pinned: false
license: mit
short_description: Tokenization demo for arxiv.org/abs/2503.08685
---
demo for arxiv.org/abs/2503.08685

Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference