Upload tokenizer.json
#2
by
ysdede
- opened
Please add default tokenizer.json to make ct2 conversion easier as we copy tokenizer while creating ct2 (faster-whisper) versions, eg:
ct2-transformers-converter --model {asr_model['name']} --output_dir {output_dir} --copy_files preprocessor_config.json tokenizer.json --quantization float16 --force
selimc
changed pull request status to
merged