smpanaro

/

Llama-3.2-1B-Instruct-CoreML

like 7

Two ways:

• Run this CLI but like this: swift run LLMCLI --repo-id smpanaro/Llama-3.2-1B-Instruct-CoreML. It will download the model into ~/Documents/huggingface and also run it.
• Clone this repo directly from the three dot menu:

Hi first I followed the following way to download the model (Run this CLI but like this: swift run LLMCLI --repo-id smpanaro/Llama-3.2-1B-Instruct-CoreML. It will download the model into ~/Documents/huggingface and also run it.).

Then, I tried to run the model with: swift run LLMCLI --repo-id smpanaro/Llama-3.2-1B-Instruct-CoreML

And this is the output:

Building for debugging...
[1/1] Write swift-version--58304C5D6DBC3306.txt
Build of product 'LLMCLI' complete! (0.22s)
ModelPipeline Llama-3.2-1B-Instruct (6 chunks)
Error: The data couldn’t be read because it isn’t in the correct format.

Do you happen to know why I get this error ? Thanks!

Can you delete the downloaded files and try again? That has helped other people with similar errors.

Hi! Thank you for the reply. I replaced the config.json, tokenizer_config.json and tokenizer.json with the actual files from the meta model and it now outputted the below. Does this mean that it runs successfully now ? If yes, how can I ask it questions ? Thanks in advance!

coreml-llm-cli % swift run LLMCLI --repo-id smpanaro/Llama-3.2-1B-Instruct-CoreML

Building for debugging...
[1/1] Write swift-version--58304C5D6DBC2226.txt
Build of product 'LLMCLI' complete! (0.22s)
ModelPipeline Llama-3.2-1B-Instruct (6 chunks)
Compiling models: ******
Loading models : ******
128000 35862 9606 315 14362 392 315 279 64677 76233 12791 8356 279 63494 505 902 11033 374 28532 13 578 1403 1925 9606 54453 67166 527 76233 12791 649 24453 6347 320 14804 82979 8 323 76233 12791 52412 3074 320 7098 370 3074 570 578 10989 3074 9606 374 6646 311 387 315 5190 4367 323 374 810 11646 1109 279 4997 82979 9606 13 578 10989 3074 9606 374 1101 810 16614 311 9499 323 38193 11 902 3727 433 810 5107 311 67632 13 578 10989 3074 9606

<|begin_of_text|>Several species of shrub of the genus Coffea produce the berries from which coffee is extracted. The two main species commercially cultivated are Coffea canephora (Robusta) and Coffea arabica (Arabica). The Arabica species is considered to be of higher quality and is more expensive than the Robusta species. The Arabica species is also more sensitive to temperature and humidity, which makes it more difficult to cultivate. The Arabica species

Compile + Load: 0,87 sec
Generate : 65,65 +/- 37,56 ms / token
16,22 +/- 1,78 token / sec