metadata
base_model:
- concedo/KobbleTinyV2-1.1B
- MysteriousAI/Mia-1B
- mrcuddle/tiny-darkllama-dpo
library_name: transformers
tags:
- mergekit
- merge
language:
- en
- es
datasets:
- Nitral-AI/Reddit-NSFW-Writing_Prompts_ShareGPT
- OEvortex/vortex-mini
Kobold-1.1B
Merge Method
This model was merged using the Model Stock merge method using concedo/KobbleTinyV2-1.1B as a base.
Models Merged
The following models were included in the merge:
Configuration
The following YAML configuration was used to produce this model:
merge_method: model_stock
models:
- model: mrcuddle/tiny-darkllama-dpo
- model: MysteriousAI/Mia-1B
base_model: concedo/KobbleTinyV2-1.1B
dtype: bfloat16
parameters:
t:
- filter: self_attn
value: [0, 0.5, 0.3, 0.7, 1]
- filter: mlp
value: [1, 0.5, 0.7, 0.3, 0]
- value: 0.5