Qwen2.5-0.5B-Cliff / README.md
Novaciano's picture
Update README.md
aec3060 verified
metadata
base_model:
  - CoolSpring/Qwen2-0.5B-Abyme-merge3
  - >-
    FlofloB/100k_fineweb_continued_pretraining_Qwen2.5-0.5B-Instruct_Unsloth_merged_16bit
library_name: transformers
tags:
  - mergekit
  - merge
  - rp
  - roleplay
language:
  - es
  - en
datasets:
  - HuggingFaceFW/fineweb

merge

This is a merge of pre-trained language models created using mergekit.

Merge Details

The Brothers

Abyss & Cliff

IMG-20250311-031904

Both is the combination of the best models QWEN2.5-0.5B of the Open LLM Leaderscore.

Merge Method

This model was merged using the Arcee Fusion merge method using FlofloB/100k_fineweb_continued_pretraining_Qwen2.5-0.5B-Instruct_Unsloth_merged_16bit as a base.

Models Merged

The following models were included in the merge:

Configuration

The following YAML configuration was used to produce this model:

models:
  - model: FlofloB/100k_fineweb_continued_pretraining_Qwen2.5-0.5B-Instruct_Unsloth_merged_16bit
  - model: CoolSpring/Qwen2-0.5B-Abyme-merge3
    parameters:
      density: 0.53
      weight: 0.6
merge_method: arcee_fusion
base_model: FlofloB/100k_fineweb_continued_pretraining_Qwen2.5-0.5B-Instruct_Unsloth_merged_16bit
tokenizer_source: union
parameters:
  int8_mask: true
dtype: bfloat16
random_seed: 0