Hugging Face's logo

VongolaChouko
/
Starcannon-Unleashed-12B-v1.0

Text Generation
Transformers
Safetensors
mistral
mergekit
Merge
text-generation-inference
Inference Endpoints
Model card Files Community
2
Starcannon-Unleashed-12B-v1.0 / README.md
VongolaChouko's picture
VongolaChouko
Update README.md
5d81a58 verified
preview code
|
raw
history blame
4.23 kB
metadata
base_model:
  - nothingiisreal/MN-12B-Starcannon-v3
  - MarinaraSpaghetti/NemoMix-Unleashed-12B
library_name: transformers
tags:
  - mergekit
  - merge
license: cc-by-nc-4.0

image/png

Starcannon Unleashed 12B v1.0

Quantized

GGUF: VongolaChouko/Starcannon-Unleashed-12B-v1.0-GGUF

I was only able to test the model using Q6_K with 24576 context at most due to PC limitations, so please let me know how it faired for you. Hopefully it still works well with higher context!

Sample Output

image/jpeg

Introduction

Instruct

Both ChatML and Mistral should work fine. Personally, I tested this using ChatML. I found that I like the model's responses better when I use this format. Try to test it out and observe which one you like best. :D

Settings

I recommend using these setings: Starcannon-Unleashed-12B-v1.0-ST-Formatting-2024-10-29.json

IMPORTANT: Open Silly Tavern and use "Master Import", which can be found under "A" tab — Advanced Formatting. Replace the "INSERT WORLD HERE" placeholders with the world/universe in which your charcater belongs to. If not applicable, just remove that part.

image/png

Temperature 1.15 - 1.25 is good, but lower should also work well, as long as you also tweak the Min P and XTC to ensure the model won't choke. Play around with it to see what suits your taste.

Both are modified version of MarinaraSpaghetti's Mistral-Small-Correct.json, transformed into ChatML.

You can find the original version here: MarinaraSpaghetti/SillyTavern-Settings

Tips

  • Examples of Dialogue and First Message are very important. The model will copy the style you wrote in these sections, so for example, if you want short outputs, make Examples of Dialogue and First Message short, and if you want longer outputs, make sure your examples have full paragraphs, composed of several sentences.
  • Utilize Author's Note In-chat @ Depth 2 as System if you want the instruction to have greater impact on the next response. If you want something exciting and spontaneous, you can try out this note I used when I tested out the model: "Scenario: Spontaneous. {{char}} has full autonomy to do anything they wish and progress the interaction in any way they like."

Credits

A very huge thank you to MarinaraSpaghetti and Nothing is Real!! (灬^ω^灬)ノ～ ♡ (´｡• ᵕ •｡`) ♡ I really fell in love with your models and it inspired me to learn how to make this one, and boi was it worth it! °˖✧◝(TT▿TT)◜✧˖°

Merge Details

This is a merge of pre-trained language models created using mergekit.

Merge Method

This model was merged using the della_linear merge method using G:\text-generation-webui\models\MarinaraSpaghetti_NemoMix-Unleashed-12B as a base.

Models Merged

The following models were included in the merge:

  • G:\text-generation-webui\models\Nothingiisreal_MN-12B-Starcannon-v3

Configuration

The following YAML configuration was used to produce this model:

base_model: G:\text-generation-webui\models\MarinaraSpaghetti_NemoMix-Unleashed-12B
dtype: bfloat16
merge_method: della_linear
parameters:
  epsilon: 0.05
  int8_mask: 1.0
  lambda: 1.0
slices:
- sources:
  - layer_range: [0, 40]
    model: G:\text-generation-webui\models\MarinaraSpaghetti_NemoMix-Unleashed-12B
    parameters:
      density: 0.65
      weight: 0.4
  - layer_range: [0, 40]
    model: G:\text-generation-webui\models\Nothingiisreal_MN-12B-Starcannon-v3
    parameters:
      density: 0.55
      weight: 0.6