What tokenizer is used ?
#3
by
Aruno
- opened
Hello,
Thank you for your model. I could use it for inference and fine-tuning without problem.
However, could you provide more information about tokenizer used ?
It is a GPTNeoXTokenizer, but how did you train it ?
Have a nice day