Meta-Llama-3.1-8B-Instruct-abliterated-companionv4-20240927_v4c

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the linear merge method.

Models Merged

The following models were included in the merge:

• /home/jovyan/llm-training-1tb/thanhtuan/models/Meta-Llama-3.1-8B-Instruct-abliterated-companionv4-20240927_v4c-e1
• /home/jovyan/llm-training-1tb/thanhtuan/models/Meta-Llama-3.1-8B-Instruct-abliterated-companionv4-20240927_v4c-e2
• /home/jovyan/llm-training-1tb/thanhtuan/models/Meta-Llama-3.1-8B-Instruct-abliterated-companionv4-20240927_v4c-e3

Configuration

The following YAML configuration was used to produce this model:

models:
  - model: /home/jovyan/llm-training-1tb/thanhtuan/models/Meta-Llama-3.1-8B-Instruct-abliterated-companionv4-20240927_v4c-e1
    parameters:
      weight: 1.0
  - model: /home/jovyan/llm-training-1tb/thanhtuan/models/Meta-Llama-3.1-8B-Instruct-abliterated-companionv4-20240927_v4c-e2
    parameters:
      weight: 1.0
  - model: /home/jovyan/llm-training-1tb/thanhtuan/models/Meta-Llama-3.1-8B-Instruct-abliterated-companionv4-20240927_v4c-e3
    parameters:
      weight: 1.0
merge_method: linear
dtype: bfloat16
parameters:
  normalize: true