|
|
--- |
|
|
license: apache-2.0 |
|
|
tags: |
|
|
- openvino |
|
|
- openvino-export |
|
|
pipeline_tag: text-generation |
|
|
base_model: OuteAI/Lite-Oute-1-300M-Instruct |
|
|
--- |
|
|
|
|
|
This model was converted to OpenVINO from [`OuteAI/Lite-Oute-1-300M-Instruct`](https://huggingface.co/OuteAI/Lite-Oute-1-300M-Instruct) using [optimum-intel](https://github.com/huggingface/optimum-intel) |
|
|
via the [export](https://huggingface.co/spaces/echarlaix/openvino-export) space. |
|
|
|
|
|
# Lite-Oute-1-300M-Instruct |
|
|
|
|
|
Lite-Oute-1-300M-Instruct is a Lite series model based on the Mistral architecture, comprising approximately 300 million parameters. <br> |
|
|
This model aims to improve upon our previous 150M version by increasing size and training on a more refined dataset. The primary goal of this 300 million parameter model is to offer enhanced performance while still maintaining efficiency for deployment on a variety of devices. <br> |
|
|
With its larger size, it should provide improved context retention and coherence, however users should note that as a compact model, it still have limitations compared to larger language models. <br> |
|
|
The model was trained on 30 billion tokens with a context length of 4096. |
|
|
|
|
|
## Available versions: |
|
|
<a href="https://huggingface.co/OuteAI/Lite-Oute-1-300M-Instruct">Lite-Oute-1-300M-Instruct</a> <br> |
|
|
<a href="https://huggingface.co/OuteAI/Lite-Oute-1-300M-Instruct-GGUF">Lite-Oute-1-300M-Instruct-GGUF</a> <br> |
|
|
<a href="https://huggingface.co/OuteAI/Lite-Oute-1-300M">Lite-Oute-1-300M</a> <br> |
|
|
<a href="https://huggingface.co/OuteAI/Lite-Oute-1-300M-GGUF">Lite-Oute-1-300M-GGUF</a> <br> |
|
|
|
|
|
## Chat format |
|
|
> [!IMPORTANT] |
|
|
> This model uses **ChatML** template. Ensure you use the correct template: |
|
|
|
|
|
``` |
|
|
<|im_start|>system |
|
|
[System message]<|im_end|> |
|
|
<|im_start|>user |
|
|
[Your question or message]<|im_end|> |
|
|
<|im_start|>assistant |
|
|
[The model's response]<|im_end|> |
|
|
``` |
|
|
|
|
|
## Benchmarks: |
|
|
<table style="text-align: left;"> |
|
|
<tr> |
|
|
<th>Benchmark</th> |
|
|
<th>5-shot</th> |
|
|
<th>0-shot</th> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>ARC Challenge</td> |
|
|
<td>26.37</td> |
|
|
<td>26.02</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>ARC Easy</td> |
|
|
<td>51.43</td> |
|
|
<td>49.79</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>CommonsenseQA</td> |
|
|
<td>20.72</td> |
|
|
<td>20.31</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>HellaSWAG</td> |
|
|
<td>34.93</td> |
|
|
<td>34.50</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>MMLU</td> |
|
|
<td>25.87</td> |
|
|
<td>24.00</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>OpenBookQA</td> |
|
|
<td>31.40</td> |
|
|
<td>32.20</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>PIQA</td> |
|
|
<td>65.07</td> |
|
|
<td>65.40</td> |
|
|
</tr> |
|
|
<tr> |
|
|
<td>Winogrande</td> |
|
|
<td>52.01</td> |
|
|
<td>53.75</td> |
|
|
</tr> |
|
|
</table> |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
First make sure you have optimum-intel installed: |
|
|
|
|
|
```bash |
|
|
pip install optimum[openvino] |
|
|
``` |
|
|
|
|
|
To load your model you can do as follows: |
|
|
|
|
|
```python |
|
|
from optimum.intel import OVModelForCausalLM |
|
|
|
|
|
model_id = "FM-1976/Lite-Oute-1-300M-Instruct-openvino" |
|
|
model = OVModelForCausalLM.from_pretrained(model_id) |
|
|
``` |
|
|
|
