Create README.md

README.md

@@ -0,0 +1,80 @@
+---
+tags:
+- llama
+- alpaca
+- cot
+- vicuna
+- uncensored
+- merge
+- mix
+---
+
+## 13B-Thorns
+
+## Composition:
+
+13B-Thorns-l2 utilizes a new merge method called Spherical Linear Interpolation. By merging data as a spherical vector store concept, a combined pair of models have a smoother transition between feature spaces that are drastically unique to each model, potentially resulting in a more coherent fusion of both model's unique strengths.
+
+Thorns' design is based on the concept of purposed segmentation-
+
+Logic Segment:
+
+Fine-Tuned parent models were hand selected and reviewed for datasets, performance, least restrictive censorship, and community perception of coherence and utility. Ultimately we decided on four models to merge in pairs of two, then combine those offspring for a quad merged logic cluster.
+All four models were merged using the SLERP method. Yes the name is annoyingly funny. SLERP.
+
+We then decided the creativity and imagination segment could be as simple as one model, especially if its dataset design, tagging, training quality, and proven track record is above and beyond. KoboldAI's Holodeck model is the result of a dataset that is years of collected, organized, tagged, deduped, and cleaned data. Holodeck alone would be beyond sufficient for the segment we view as the 'subconscious' segment of the model ensemble, however we applied the LIMA RP PEFT to it for extended variety of a different kind.
+
+is also comprised of various roleplay themed LoRAs 
+
+-Model Merge Ensemble Key-
+{} = SLERP Merge | [] = PEFT Merge | () = Composite Model
+({({NousHermes+Chronos}[kimiko])+({Platupus+AiroborosM2.0}[janine])}{Holodeck[LIMARP]})
+
+
+
+
+[SuperCOT([gtp4xalpaca(manticorechatpygalpha+vicunaunlocked)]+[StoryV2(kaiokendev-SuperHOT-LoRA-prototype30b-8192)])]
+
+This model is the result of an experimental use of LoRAs on language models and model merges that are not the base HuggingFace-format LLaMA model they were intended for.
+The desired outcome is to additively apply desired features without paradoxically watering down a model's effective behavior.
+
+Potential limitations - LoRAs applied on top of each other may intercompete.
+
+Subjective results - very promising. Further experimental tests and objective tests are required.
+
+Instruct and Setup Suggestions:
+
+Alpaca instruct is primary, Vicuna instruct format may work.
+If using KoboldAI or Text-Generation-WebUI, recommend switching between Godlike and Storywriter presets and adjusting output length + instructions in memory.
+Other presets as well as custom settings can yield highly different results, especially Temperature.
+If poking it with a stick doesn't work try poking harder.
+
+## Language Models and LoRAs Used Credits:
+
+manticore-30b-chat-pyg-alpha [Epoch0.4] by openaccess-ai-collective
+
+https://huggingface.co/openaccess-ai-collective/manticore-30b-chat-pyg-alpha
+
+SuperCOT-LoRA [30B] by kaiokendev
+
+https://huggingface.co/kaiokendev/SuperCOT-LoRA
+
+Storytelling-LLaMa-LoRA [30B, Version 2] by GamerUnTouch
+
+https://huggingface.co/GamerUntouch/Storytelling-LLaMa-LoRAs
+
+SuperHOT Prototype [30b 8k ctx] by kaiokendev
+
+https://huggingface.co/kaiokendev/SuperHOT-LoRA-prototype
+
+ChanSung's GPT4-Alpaca-LoRA
+https://huggingface.co/chansung/gpt4-alpaca-lora-30b
+
+Neko-Institute-of-Science's Vicuna Unlocked LoRA (Checkpoint 46080)
+https://huggingface.co/Neko-Institute-of-Science/VicUnLocked-30b-LoRA
+
+Also thanks to Meta for LLaMA.
+
+Each model and LoRA was hand picked and considered for what it could contribute to this ensemble.
+Thanks to each and every one of you for your incredible work developing some of the best things
+to come out of this community.