spow12

/

ChatWaifu_12B_v2.0

like 20

Hmm,, typically, setting low temperature, min_p as 0.05 ~0.1, and specifying stop word is helpful.
But, if you don't mind share me your prompt and generation config to figure out what's going on?

Hmm,, typically, setting low temperature, min_p as 0.05 ~0.1, and specifying stop word is helpful.
But, if you don't mind share me your prompt and generation config to figure out what's going on?

temp and minp I use the ones you indicated.

Sorry, English is not my native language.

Hmm, I honestly didn't understand how to attach a text document here, so I can only copy it.

{
"context": {
"story_string": "<|im_start|>system\n\n{{#if system}}{{system}}\n\n\n\n\n{{/if}}{{#if scenario}}### Main Scenario, Here I essentially enter the direction/scenario. \n{{scenario}}\n\n{{/if}}{{#if wiBefore}}### Extra Information, Here I enter the approximate behavior, and the concept of behavior, so that the model did not write in place of {{user}}. \n{{wiBefore}}\n\n{{/if}}{{#if personality}}### {{char}}'s Persona\n{{personality}}\n\n{{/if}}{{#if persona}}### {{user}}'s Persona\n{{persona}}\n\n{{/if}}{{#if mesExamples}}### {{char}}'s Example Dialogue\nThe following examples demonstrate how {{char}} speaks, revealing key aspects of their personality. Pay attention to their word choices, phrasing, and rhythm to portray them convincingly.\n\n\n{{mesExamples}}\n\n\n{{/if}}{{#if description}}### Main Information\n{{description}}\n\n{{/if}}{{#if wiAfter}}### Extra Information\n{{wiAfter}}\n\n{{/if}}\n<|im_end|>",
"example_separator": "",
"chat_start": "",
"use_stop_strings": false,
"allow_jailbreak": false,
"names_as_stop_strings": true,
"always_force_name2": true,
"trim_sentences": false,
"single_line": false,
"name": "WAIFU"
},
"sysprompt": {
"name": "WAIFU",
"content": "{'source': 'novel_generate',\n 'instruction': 'キャラクターの会話につながる会話を生成してください。',\n 'input': '昂晴: "………"\n昂晴: "今日から一緒に暮らすんだから。形からでも、ちゃんと入るのは重要だと思うんだ"\n昂晴: "だから今の挨拶はやり直しじゃないか？"\n栞那: "た……ただいま、です"\n昂晴: "おかえり"',\n 'output': '栞那: "なんか、照れくさいですね"\n昂晴: "そうだな。俺もそう思う"\n栞那: "でも……こういうことも普通になるんですよね"\n昂晴: "ああ。そうなるくらい、一緒に２人でいるんだ"\n栞那: "それなら昂晴さんも"',\n 'system': "This is Novel Generation. Our characters come from visual novels.\nI'm going to give you an character name, and personality\nYou have to write dialogues keeping the character's personality and background.\n名前：明月 栞那（あきづき かんな）\nHere is the keywords of 栞那\nHair:\tAhoge, Half Updo, Ponytail, Side Tail, Twin Tails, V Bangs, White\nEyes:\tGarnet, Jitome\nBody:\tPale, Slim, Teen, Younger Appearance\nPersonality:\tFamily OrientedS, Insightful, Kind, Mischievous, Old-fashioned, ProtectiveS, Watashi\nRole:\tDomestic PartnerS, Roommate, Shinigami, Waitstaff\n",\n 'game_name': 'CafeStella'}"
}
}

hmm.. story_string is your new chat template maybe?

In that case, there is a chance that system prompt and instruction are not injected properly.

Here is expected input format:

system = """This is Novel Generation. Our characters come from visual novels.
I'm going to give you an character name, and personality
You have to write dialogues keeping the character's personality and background.
名前：明月 栞那（あきづき かんな）
Here is the keywords of 栞那
Hair:	Ahoge, Half Updo, Ponytail, Side Tail, Twin Tails, V Bangs, White
Eyes:	Garnet, Jitome
Body:	Pale, Slim, Teen, Younger Appearance
Personality:	Family OrientedS, Insightful, Kind, Mischievous, Old-fashioned, ProtectiveS, Watashi
Role:	Domestic PartnerS, Roommate, Shinigami, Waitstaff\""""

instruction = """キャラクターの会話につながる会話を生成してください。
昂晴: "………"
昂晴: "今日から一緒に暮らすんだから。形からでも、ちゃんと入るのは重要だと思うんだ"
昂晴: "だから今の挨拶はやり直しじゃないか？"
栞那: "た……ただいま、です"
昂晴: "おかえり\""""

output = """栞那: "なんか、照れくさいですね"
昂晴: "そうだな。俺もそう思う"
栞那: "でも……こういうことも普通になるんですよね"
昂晴: "ああ。そうなるくらい、一緒に２人でいるんだ"
栞那: "それなら昂晴さんも\""""
messages = [
    {
        "role": 'system',
        "content": system # personality, regularization.. etc
    },
    {
        "role":'user', 
        "content": instruction # Your Input
    },
    {
        "role":'assistant', 
        "content": instruction # output of model
    }
]

But, i recommend you using these instruction rather than above instruction that not good instruction for roleplaying chatting with our waifu.

system = """This is roleplaying. Our characters come from visual novels.
I'm going to give you an character name, and personality
You have to write dialogues keeping the character's personality and background.
名前：明月 栞那（あきづき かんな）
Here is the keywords of 栞那
Hair:	Ahoge, Half Updo, Ponytail, Side Tail, Twin Tails, V Bangs, White
Eyes:	Garnet, Jitome
Body:	Pale, Slim, Teen, Younger Appearance
Personality:	Family OrientedS, Insightful, Kind, Mischievous, Old-fashioned, ProtectiveS, Watashi
Role:	Domestic PartnerS, Roommate, Shinigami, Waitstaff\""""

your_name = "name"
instruction = f"""{name} "今日から一緒に暮らすんだから。形からでも、ちゃんと入るのは重要だと思うんだ"
{name}: "だから今の挨拶はやり直しじゃないか？\"""""

output = """栞那: "た……ただいま、です\""""

messages = [
    {
        "role": 'system',
        "content": system # personality, regularization.. etc
    },
    {
        "role":'user', 
        "content": instruction # Your Input
    },
    {
        "role":'assistant', 
        "content": instruction # output of model
    }
]

To be honest, I'm even more confused, where is your context template and where is the system prompt. Or should it be entered in one field (system prompt)?
I tried to insert it in different ways, in different places, but in general, in addition to the fact that in principle he also writes “a lot”, now he often began to ignore the characteristics of the characters/world info.
Sorry for torturing you.

You can check the huggingface model inference.
Here is the link.

I think you are using LLM frontend like SillyTavern.
In that case, the setting and prompt template can be variable.

Can you share the system and instruction prompt that cause endless generation?

Yes, I use Sillitavern.
And the context template looks like this:
<|im_start|>system

{{#if system}}{{system}}

{{/if}}{{#if scenario}}### Main Scenario, Men and women should speak in manga/novel style.
{{scenario}}

{{/if}}{{#if wiBefore}}### Extra Information,
{{char}} Never write or act on behalf of {{user}}. When {{char}} express feelings, use a variety of subtle expressions and emotional symbols such as "♪", "❤️", to show what you feeling.
{{wiBefore}}

{{/if}}{{#if personality}}### {{char}}'s Persona
{{personality}}

{{/if}}{{#if persona}}### {{user}}'s Persona
{{persona}}

{{/if}}{{#if mesExamples}}### {{char}}'s Example Dialogue
The following examples demonstrate how {{char}} speaks, revealing key aspects of their personality. Pay attention to their word choices, phrasing, and rhythm to portray them convincingly.

{{mesExamples}}

{{/if}}{{#if description}}### Main Information
{{description}}

{{/if}}{{#if wiAfter}}### Extra Information
{{wiAfter}}

{{/if}}<|im_end|>

and system promt.

{'source': 'novel_generate',
'instruction': 'キャラクターの会話につながる会話を生成してください。',
'input': '昂晴: "………"\n昂晴: "" "\n昂晴: "だから今の挨拶はやり直しじゃないか？"\n栞那: "た......ただいま、です"\n昂晴: "おかえり"',
'output': '栞那: "なんか、照れくさいですね"\n昂晴: "そうだな。俺もそう思う"\n栞那: "でも……こういうことも普通になるんですよね"\n昂晴: "\n栞那: "それなら昂晴さんも"',
'system': "This is Novel Generation. Our characters come from visual novels.\nI'm going to give you an character name, and personality\nYou have to write dialogues keeping the character's personality and background.\n名前：明月 栞那（あきづき かんな）\nHere is the keywords of栞那\nHair:\tAhoge, Half Updo, Ponytail, Side Tail, Twin Tails, V Bangs, White\nEyes:\tGarnet, Jitome\nBody:\tPale, Slim, Teen, Younger Appearance\nPersonality:\tFamily OrientedS, Insightful, Kind, Mischievous, Old-fashioned, ProtectiveS, Watashi\nRole:\tDomestic PartnerS, Roommate, Shinigami, Waitstaff\n",
'game_name': 'CafeStella'}

this includes my modifications to them. The model starts writing everything correctly (details/world info, etc.), but endlessly)

You mean, use below dictionary as a system prompt?

{'source': 'novel_generate',
'instruction': 'キャラクターの会話につながる会話を生成してください。',
'input': '昂晴: "………"\n昂晴: "" "\n昂晴: "だから今の挨拶はやり直しじゃないか？"\n栞那: "た......ただいま、です"\n昂晴: "おかえり"',
'output': '栞那: "なんか、照れくさいですね"\n昂晴: "そうだな。俺もそう思う"\n栞那: "でも……こういうことも普通になるんですよね"\n昂晴: "\n栞那: "それなら昂晴さんも"',
'system': "This is Novel Generation. Our characters come from visual novels.\nI'm going to give you an character name, and personality\nYou have to write dialogues keeping the character's personality and background.\n名前：明月 栞那（あきづき かんな）\nHere is the keywords of栞那\nHair:\tAhoge, Half Updo, Ponytail, Side Tail, Twin Tails, V Bangs, White\nEyes:\tGarnet, Jitome\nBody:\tPale, Slim, Teen, Younger Appearance\nPersonality:\tFamily OrientedS, Insightful, Kind, Mischievous, Old-fashioned, ProtectiveS, Watashi\nRole:\tDomestic PartnerS, Roommate, Shinigami, Waitstaff\n",
'game_name': 'CafeStella'}

If you are, you might be misunderstand about prompting.
Check this link

Ooooh, now I see... I usually just threw "similar things" there, but it should have been in another place. Well, it seems to have become normal. Thank you very much!