tf-tpu
/

unigram-tokenizer-wikitext

Model card Files Files and versions Community

carted-ml commited on Jun 10, 2022

Commit

3a74166

·

1 Parent(s): 99b7b7b

Create README.md

Files changed (1) hide show

README.md +1 -0

README.md ADDED Viewed

	@@ -0,0 +1 @@


1	+ This is a [Unigram tokenizer](https://huggingface.co/course/chapter6/7?fw=pt) trained on the [Wikitext dataset](https://huggingface.co/datasets/wikitext). Refer to the `train_unigram.py` script within this repository to know how it was trained.