Hugging Face's logo

RWKV
/
v5-Eagle-7B-pth

Model card Files Community
7

size of the vocabulary

#1
by yuimo - opened
Discussion
yuimo

what is the size of the vocabulary? and how to train the tokenizer? BPE or wordpiece?

BlinkDL
 RWKV org

65536
It's a greedy tokenizer
see https://github.com/BlinkDL/ChatRWKV/tree/main/tokenizer

Your need to confirm your account before you can post a new comment.

Sign up or log in to comment