File size: 666 Bytes
1d58d2f 62f25f0 41acd1a 3cd34b0 41acd1a |
1 2 3 4 5 6 7 8 9 10 11 |
---
license: apache-2.0
---
4bit GGUF quantization of [TinyLlama-1.1B-intermediate-step-955k-token-2T](https://huggingface.co/TinyLlama/TinyLlama-1.1B-intermediate-step-955k-token-2T)
I used [this script](https://github.com/ggerganov/llama.cpp/blob/master/examples/make-ggml.py) to generate the file using this command:
`python make-ggml.py ~/ooba/models/TinyLlama_TinyLlama-1.1B-intermediate-step-955k-token-2T/ --model_type=llama --quants=Q4_K_M`
The original model is so small that there is only one safetensors file named `model.safetensors`, so I had to change that filename to `model-00001-of-00001.safetensors` to make the script load the model properly. |