Hugging Face's logo

mehdie
/
BPE-tokenizer

Model card Files Community
morten-j commited on
Commit
fac3fd5
·
verified ·
1 Parent(s): 03ff453

Create README.md

Browse files
Files changed (1) hide show
  1. README.md +2 -0
README.md ADDED
@@ -0,0 +1,2 @@
 
 
 
1
+ BPE based tokenizer used for the MEHDIE project and the training of a bilingual BERT model.
2
+ Vocab size of 52000.