Marathi Tokenizer Demo
A web interface for the Marathi BPE tokenizer. This demo allows you to:
- Enter Marathi text
- See the encoded tokens
- Verify the decoding
- Check token counts
Usage
Simply enter Marathi text in the input box and see the tokenization results!
