another llama

#620
by mangojesus - opened

i checked the file tree and couldn't see a tokenizer, however, it's a meta-llama-3.1-8b-abliterated.... maybe im ignorant, but my thought is tokenziers arent unique to each individual model are they? given the included files, can this model be quantized?

https://huggingface.co/vedant-jumle/meta-llama3.1-8b-abliterated-hacking/tree/main

The tokenizer is absolutely required, and yes, there isn't one in that model. Very likely it is the same as the llama-3.1 one, but that kind of doctoring normally goes beyond what I am willing to do - it's too big a change, too little of the creator intent.

Looks like it's time for you to step into the spotlight as the ultimate model fixer! Time to clone that masterpiece and integrate those elusive tokenizer files from llama-3.1-8b. Let’s unleash some serious AI power!

(disclaimer: this reply was written with the help of an ai).

Sign up or log in to comment