Upload tokenizer.json
Browse filesPlease add default tokenizer.json to make ct2 conversion easier as we copy tokenizer while creating ct2 (faster-whisper) versions, eg:
```python
ct2-transformers-converter --model {asr_model['name']} --output_dir {output_dir} --copy_files preprocessor_config.json tokenizer.json --quantization float16 --force
```
- tokenizer.json +0 -0
tokenizer.json
ADDED
The diff for this file is too large to render.
See raw diff
|
|