Rename tokenizer/tokenizer_vocab.json to tokenizer/vocab.json 84d5806 verified Saurabh4509 commited on May 16, 2024
Rename tokenizer/tokenizer_tokenizer_config.json to tokenizer/tokenizer_config.json 01d815f verified Saurabh4509 commited on May 16, 2024
Rename tokenizer/tokenizer_special_tokens_map.json to tokenizer/special_tokens_map.json 033aefe verified Saurabh4509 commited on May 16, 2024
Rename tokenizer/tokenizer_merges.txt to tokenizer/merges.txt a191f6c verified Saurabh4509 commited on May 16, 2024