eachadea

/

legacy-ggml-vicuna-13b-4bit

like 207

First, ensure your SHA256 checks out:

c6eb3a970b687584b16987e0aedc8513e885ec368b9f4a51e8cd69de5740cb7b

Also ensure that your llama.cpp is up to date. The conversion script I used is not backwards compatible.

Lastly, this error can also appear if there is no model at the specified path - double check that it's there and that all the paths are correct.

Everything seems to be correct:

❯ sha256sum models/ggml-vicuna-13b-4bit.bin
c6eb3a970b687584b16987e0aedc8513e885ec368b9f4a51e8cd69de5740cb7b  models/ggml-vicuna-13b-4bit.bin
❯ git pull
Already up to date.
❯ ./main -m ./models/ggml-vicuna-13b-4bit.bin  -t 8 -n 256 --repeat_penalty 1.0 -ins
main: seed = 1680590608
llama_model_load: loading model from './models/ggml-vicuna-13b-4bit.bin' - please wait ...
llama_model_load: invalid model file './models/ggml-vicuna-13b-4bit.bin' (bad magic)
llama_init_from_file: failed to load model
main: error: failed to load model './models/ggml-vicuna-13b-4bit.bin'

May it be related to the command I use to start llama.cpp ?

It seems to be up to date, but did you compile the binaries with the latest code?

Wow I am so stupid... This is the result that we almost only code in languages that don't compile...

So sorry for the inconvenience, it works!
Thanks again for your time and work!

I actually have the same issue. I just installed dalai on my M2 MacBook according to these instructions: https://github.com/cocktailpeanut/dalai. What exactly do you mean by “compile the binaries with the latest code”? Do I need to compile the models first?

@rachedamer

I actually have the same issue. I just installed dalai on my M2 MacBook according to these instructions: https://github.com/cocktailpeanut/dalai. What exactly do you mean by “compile the binaries with the latest code”? Do I need to compile the models first?

The GGML format is evolving regularly at the moment, you should try with the latest version: https://huggingface.co/eachadea/ggml-vicuna-13b-1.1

But I wasn't trying to use Vicuna but Alpaca. Can I get Vicuna to work with dalai?

Every ggml models compatible with https://github.com/ggerganov/llama.cpp should works with dalai

Every ggml models compatible with https://github.com/ggerganov/llama.cpp should works with dalai

then why i am getting this error? llama_model_load: loading model from './models/ggml-gpt4all-j-v1.3-groovy.bin' - please wait ...
llama_init_from_file: failed to load model