---
base_model:
- concedo/KobbleTinyV2-1.1B
- MysteriousAI/Mia-1B
- mrcuddle/tiny-darkllama-dpo
library_name: transformers
tags:
- mergekit
- merge
language:
- en
- es
datasets:
- Nitral-AI/Reddit-NSFW-Writing_Prompts_ShareGPT
- OEvortex/vortex-mini
---
# Kobold-1.1B

<center>
<img src="https://images.squarespace-cdn.com/content/v1/58814a7abf629aeb03ec15b1/1549562098197-CDZPLVRU5C4CW0DQRFB0/Kobold+Colony.jpg" alt="IMG-20250313-213616" border="0">
</center>

---

### Merge Method

This model was merged using the [Model Stock](https://arxiv.org/abs/2403.19522) merge method using [concedo/KobbleTinyV2-1.1B](https://huggingface.co/concedo/KobbleTinyV2-1.1B) as a base.

### Models Merged

The following models were included in the merge:
* [MysteriousAI/Mia-1B](https://huggingface.co/MysteriousAI/Mia-1B)
* [mrcuddle/tiny-darkllama-dpo](https://huggingface.co/mrcuddle/tiny-darkllama-dpo)

### Configuration

The following YAML configuration was used to produce this model:

```yaml
merge_method: model_stock
models:
- model: mrcuddle/tiny-darkllama-dpo
- model: MysteriousAI/Mia-1B
base_model: concedo/KobbleTinyV2-1.1B
dtype: bfloat16
parameters:
  t:
    - filter: self_attn
      value: [0, 0.5, 0.3, 0.7, 1]
    - filter: mlp
      value: [1, 0.5, 0.7, 0.3, 0]
    - value: 0.5
```