ysdede commited on
Commit
24bcb35
·
verified ·
1 Parent(s): 7df2ce3

Upload tokenizer.json

Browse files

Please add default tokenizer.json to make ct2 conversion easier as we copy tokenizer while creating ct2 (faster-whisper) versions, eg:

```python
ct2-transformers-converter --model {asr_model['name']} --output_dir {output_dir} --copy_files preprocessor_config.json tokenizer.json --quantization float16 --force
```

Files changed (1) hide show
  1. tokenizer.json +0 -0
tokenizer.json ADDED
The diff for this file is too large to render. See raw diff