| language: | |
| - en | |
| tags: | |
| - llama.cpp | |
| - gguf | |
| - aya | |
| - cohere | |
| - quantized | |
| library_name: llama.cpp | |
| pipeline_tag: text-generation | |
| license: apache-2.0 | |
| # Aya Sl Biz 8B | |
| This is a GGUF format quantized version of a fine-tuned CohereForAI/aya-23-8B model. | |
| ## Model Details | |
| - **Original Model:** CohereForAI/aya-23-8B | |
| - **Quantization Type:** Q4_K_M | |
| - **Format:** GGUF | |
| - **Conversion Date:** 2024-10-31 | |
| - **Framework:** llama.cpp | |
| ## Usage | |
| This model can be used with [llama.cpp](https://github.com/ggerganov/llama.cpp). Here's how to use it: | |
| ```bash | |
| # Basic usage | |
| ./llama-cli -m path_to_model.gguf -n 512 --prompt "Your prompt here" | |
| # Chat format | |
| ./llama-cli -m path_to_model.gguf --temp 0.7 --repeat-penalty 1.2 -n 512 --prompt "<|START_OF_TURN_TOKEN|><|SYSTEM_TOKEN|>You are Command-R, a helpful AI assistant.<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|USER_TOKEN|>Your prompt here<|END_OF_TURN_TOKEN|><|START_OF_TURN_TOKEN|><|CHATBOT_TOKEN|>" | |
| ``` | |
| ## Quantization Details | |
| This model was quantized using the Q4_K_M format, which offers a good balance between model size and performance. The quantization was performed using llama.cpp's quantization tools. | |
| Original model size: ~16GB | |
| Quantized model size: ~4.7GB | |
| ## License | |
| This model is released under the Apache 2.0 license. | |