Quantization made by Richard Erkhov.

vinallama-12.5b-chat-DUS - GGUF

Model creator: https://huggingface.co/vilm/
Original model: https://huggingface.co/vilm/vinallama-12.5b-chat-DUS/

Name	Quant method	Size
vinallama-12.5b-chat-DUS.Q2_K.gguf	Q2_K	4.36GB
vinallama-12.5b-chat-DUS.IQ3_XS.gguf	IQ3_XS	4.82GB
vinallama-12.5b-chat-DUS.IQ3_S.gguf	IQ3_S	5.08GB
vinallama-12.5b-chat-DUS.Q3_K_S.gguf	Q3_K_S	5.08GB
vinallama-12.5b-chat-DUS.IQ3_M.gguf	IQ3_M	5.37GB
vinallama-12.5b-chat-DUS.Q3_K.gguf	Q3_K	5.69GB
vinallama-12.5b-chat-DUS.Q3_K_M.gguf	Q3_K_M	5.69GB
vinallama-12.5b-chat-DUS.Q3_K_L.gguf	Q3_K_L	6.22GB
vinallama-12.5b-chat-DUS.IQ4_XS.gguf	IQ4_XS	6.3GB
vinallama-12.5b-chat-DUS.Q4_0.gguf	Q4_0	6.61GB
vinallama-12.5b-chat-DUS.IQ4_NL.gguf	IQ4_NL	6.64GB
vinallama-12.5b-chat-DUS.Q4_K_S.gguf	Q4_K_S	6.65GB
vinallama-12.5b-chat-DUS.Q4_K.gguf	Q4_K	7.05GB
vinallama-12.5b-chat-DUS.Q4_K_M.gguf	Q4_K_M	7.05GB
vinallama-12.5b-chat-DUS.Q4_1.gguf	Q4_1	7.33GB
vinallama-12.5b-chat-DUS.Q5_0.gguf	Q5_0	8.04GB
vinallama-12.5b-chat-DUS.Q5_K_S.gguf	Q5_K_S	8.04GB
vinallama-12.5b-chat-DUS.Q5_K.gguf	Q5_K	8.27GB
vinallama-12.5b-chat-DUS.Q5_K_M.gguf	Q5_K_M	8.27GB
vinallama-12.5b-chat-DUS.Q5_1.gguf	Q5_1	8.76GB
vinallama-12.5b-chat-DUS.Q6_K.gguf	Q6_K	9.57GB
vinallama-12.5b-chat-DUS.Q8_0.gguf	Q8_0	12.39GB

Original model description:

license: apache-2.0 tags: - merge - mergekit - vilm/vinallama-7b-chat

VinaLLaMA - State-of-the-art Vietnamese LLMs

Read our Paper

Prompt Format (ChatML):

<|im_start|>system
Bạn là một trợ lí AI hữu ích. Hãy trả lời người dùng một cách chính xác.
<|im_end|>
<|im_start|>user
Hello world!<|im_end|>
<|im_start|>assistant

Evaluation

This table is copied from VBD-Llama2 with updated results from VinaLLaMA-12.5B-chat-DUS

Model	Model size	arc_vi (acc)	hellaswag_vi (acc)	mmlu_vi (acc)	truthfulqa_vi (acc)	Average
URA-LLaMA-13B	13B	0,3752	0,4830	0,3973	0,4574	0,4282
BLOOMZ-7B	7B	0,3205	0,4930	0,3975	0,4523	0,4158
PhoGPT-7B5-Instruct	7B	0,2470	0,2578	0,2413	0,4759	0,3055
SeaLLM-7B-chat	7B	0,3607	0,5112	0,3339	0,4948	0,4252
Vietcuna-7b-v3	7B	0,3419	0,4939	0,3354	0,4807	0,4130
VinaLLaMA-2.7B-chat	7B	0,3273	0,4814	0,3051	0,4972	0,4028
VinaLLaMA-7B-chat	7B	0,4239	0,5407	0,3932	0,5251	0,4707
VBD-LLaMA2-7B-50b	7B	0,3222	0,5195	0,2964	0,4614	0,3999
VBD-LLaMA2-7B-50b-Chat	7B	0,3585	0,5207	0,3444	0,5179	0,4354
VinaLLaMA-12.5B-chat-DUS	12.5B	0,4325	0,5816	0,3875	0,5850	0,4967

Merging Methods

This model is a merge of the following models made with LazyMergekit:

vilm/vinallama-7b-chat

🧩 Configuration

slices:
  - sources:
    - model: vilm/vinallama-7b-chat
      layer_range: [0, 16]
  - sources:
    - model: vilm/vinallama-7b-chat
      layer_range: [8, 16]
  - sources:
    - model: vilm/vinallama-7b-chat
      layer_range: [8, 16]      
  - sources:
    - model: vilm/vinallama-7b-chat
      layer_range: [16, 24]
  - sources:
    - model: vilm/vinallama-7b-chat
      layer_range: [16, 24]
  - sources:
    - model: vilm/vinallama-7b-chat
      layer_range: [24, 28]
  - sources:
    - model: vilm/vinallama-7b-chat
      layer_range: [24, 28]
  - sources:
    - model: vilm/vinallama-7b-chat
      layer_range: [28, 32]
merge_method: passthrough
dtype: bfloat16