Where is the tokenizer?
#5
by
bobby-nakamoto
- opened
I can't find the tokenizer for this model. Can someone direct me to it?
This comment has been hidden
I haven't found the tokenizer yet.
I haven't found the tokenizer yet.
give me solution
I have the same problem...
I ended up using a tokenizer from a model that was trained off of this one.
I forget which one, but search for models that were trained off of this one and go in and take that model's tokenizer from their files.