llama-3-stinky-v2-8B

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the Model Stock merge method using flammenai/Mahou-1.1-llama3-8B as a base.

Models Merged

The following models were included in the merge:

• mlabonne/ChimeraLlama-3-8B-v2
• grimjim/llama-3-merge-pp-instruct-8B
• grimjim/llama-3-merge-virt-req-8B
• uygarkurt/llama-3-merged-linear
• jeiku/Orthocopter_8B
• grimjim/llama-3-nvidia-ChatQA-1.5-8B
• openlynn/Llama-3-Soliloquy-8B-v2
• VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct
• nbeerbower/llama-3-stella-8B
• cloudyu/Meta-Llama-3-8B-Instruct-DPO
• NeverSleep/Llama-3-Lumimaid-8B-v0.1-OAS
• flammenai/Mahou-1.0-llama3-8B

Configuration

The following YAML configuration was used to produce this model:

models:
  - model: mlabonne/ChimeraLlama-3-8B-v2
  - model: cloudyu/Meta-Llama-3-8B-Instruct-DPO
  - model: nbeerbower/llama-3-stella-8B
  - model: VAGOsolutions/Llama-3-SauerkrautLM-8b-Instruct
  - model: uygarkurt/llama-3-merged-linear
  - model: openlynn/Llama-3-Soliloquy-8B-v2
  - model: grimjim/llama-3-merge-pp-instruct-8B
  - model: NeverSleep/Llama-3-Lumimaid-8B-v0.1-OAS
  - model: grimjim/llama-3-merge-virt-req-8B
  - model: jeiku/Orthocopter_8B
  - model: grimjim/llama-3-nvidia-ChatQA-1.5-8B
  - model: flammenai/Mahou-1.0-llama3-8B
merge_method: model_stock
base_model: flammenai/Mahou-1.1-llama3-8B
dtype: bfloat16

Open LLM Leaderboard Evaluation Results

Detailed results can be found here

Metric	Value
Avg.	70.27
AI2 Reasoning Challenge (25-Shot)	66.98
HellaSwag (10-Shot)	83.20
MMLU (5-Shot)	68.33
TruthfulQA (0-shot)	55.83
Winogrande (5-shot)	77.51
GSM8k (5-shot)	69.75