Create README.md

README.md

@@ -0,0 +1,34 @@
+---
+license: llama2
+---
+# LLilmonix3b-v0.4a:
+* Experimental [Marx-3b-v2](https://huggingface.co/acrastt/Marx-3B-V2) fine-tuned for Monika character from DDLC
+* Trained on a dataset of ~600 items (dialogue scraped from game, reddit, and Twitter augmented by [l2-7b-monika-v0.3c1](https://huggingface.co/922-CA/llama-2-7b-monika-v0.3c1) to turn each into snippets of multi-turn chat dialogue between Player and Monika + manually crafted test dataset of 12 items)
+* Trained to run on smaller devices
+* [QLoras (hf and GGML)](https://huggingface.co/922-CA/monika-lm-lora-tests/tree/main/LLilmonix3b-v0.4-Marxv2)
+
+### USAGE
+This is meant to be mainly a chat model with limited RP ability.
+
+For best results: replace "Human" and "Assistant" with "Player" and "Monika" like so:
+
+\nPlayer: (prompt)\nMonika:
+
+Tested primarily with koboldcpp 1.41+, with q8_0 GGML, temperature at 0.7, and the rest left as is on chat mode:
+
+### HYPERPARAMS
+* Trained for 2 epochs
+* rank: 32
+* lora alpha: 64
+* lora dropout: 0.5
+* lr: 2e-4
+* batch size: 2
+* warmup ratio: 0.1
+* grad steps: 4
+
+### WARNINGS AND DISCLAIMERS
+Note that aside from formatting and other minor edits, generated portion of dataset used is mostly as is generated by LM. In addition, the is meant to be a smaller version of the larger Monika models. As such, this version may not reflect perfectly the characteristics of Monika.
+
+Additionally, this is still yet another test, particularly where we use one of our earlier fine tunes to generate a more in-character dataset for the target character.
+
+Finally, this model is not guaranteed to output aligned or safe outputs, use at your own risk.