Update README.md

README.md

@@ -1,17 +1,21 @@
----
-license: apache-2.0
-datasets:
-- SkunkworksAI/reasoning-0.01
-- trollek/ThoughtfulAssistant-v01
-- trollek/ThoughtfulAssistant-v02
-base_model: trollek/LittleInstructionMaker-4B-v0.2
----
+---
+license: apache-2.0
+datasets:
+- SkunkworksAI/reasoning-0.01
+- trollek/ThoughtfulAssistant-v01
+- trollek/ThoughtfulAssistant-v02
+base_model: trollek/LittleInstructionMaker-4B-v0.2
+language:
+- en
+---
 # ThoughtStream-4B-v0.2
 
-A more versitile version of [the first one](https://huggingface.co/trollek/ThoughtStream-4B-v0.1), and less versitile than the next[.](https://www.youtube.com/watch?v=dQw4w9WgXcQ "I swear my name makes sense in Danish, but.. I mean..") This can not compete with [Matt Shumer's](https://huggingface.co/mattshumer) [Reflection](https://huggingface.co/mattshumer/Reflection-Llama-3.1-70B), which be wildin' yo. But creating a dataset to "Please find errors and hallucinations in the thought process" is a tad harder than expected™. It is brilliant! And hard! Living in the future is great!
+A more versitile version of [the first one](https://huggingface.co/trollek/ThoughtStream-4B-v0.1), and less versitile than the next[.](https://www.youtube.com/watch?v=dQw4w9WgXcQ "I swear my name makes sense in Danish, but.. I mean..")
 
 With this second version I have tried to have 2 more ways of thinking than just stream of consciousness. This being mind mapping and step by step reasoning thanks to [SkunkworksAI](https://huggingface.co/SkunkworksAI) and their [reasoning-0.01](https://huggingface.co/datasets/SkunkworksAI/reasoning-0.01) dataset. Big up for creating [TinyFrank](https://huggingface.co/SkunkworksAI/tinyfrank-1.4B) that got me started on this wild adventure.
 
+This model is most useful for experimentation like "What happens if I substitute the thoughts?" etc. The answer is very well now. The first version was only trained on stream of consciousness, whereas this can handle mind maps and CoT reasoning. By itself it sucks at reasoning, but inject the CoT in the thinking tags and it performs better. 
+
 ## System prompts
 
 **Stream of consciousness:**
@@ -26,61 +30,4 @@ With this second version I have tried to have 2 more ways of thinking than just
 **Step-by-step reasoning:**
 
 - `Your decision-making process is methodical and sequential, breaking down complex tasks into manageable steps to ensure thorough analysis and execution with precision and reliability. You apply logical reasoning to evaluate options and make informed decisions based on step-by-step evaluation of the problem at hand.`
-- `You are an AI assistant that think before you answer. When thinking you break your reasoning down step by step.`
-
-
-## Config
-
-```yaml
-### model
-model_name_or_path: lim-v02-thought
-
-### method
-stage: sft
-do_train: true
-finetuning_type: lora
-lora_target: all
-loraplus_lr_ratio: 12.0
-lora_rank: 16
-lora_alpha: 16
-use_unsloth: true
-quantization_bit: 4
-upcast_layernorm: true
-seed: 127
-optim: lion_8bit
-additional_target: embed_tokens
-
-### dataset
-dataset: reasoning_assistant,thoughtful_v01,thoughtful_v02
-template: ninja_chatml
-cutoff_len: 8192
-overwrite_cache: false
-preprocessing_num_workers: 12
-
-### output
-output_dir:  /home/trolle/Documents/Projects/trollek/danube3/merges/lim-v02-thought/loras/reasoning
-logging_steps: 5
-save_steps: 1
-save_strategy: epoch
-plot_loss: true
-overwrite_output_dir: false
-
-### train
-per_device_train_batch_size: 2
-gradient_accumulation_steps: 4
-learning_rate: 0.000002
-num_train_epochs: 2
-lr_scheduler_type: constant_with_warmup
-warmup_ratio: 0.01
-bf16: true
-flash_attn: fa2
-
-### eval
-val_size: 0.01
-per_device_eval_batch_size: 1
-eval_strategy: steps
-eval_steps: 1000
-
-```
-
-## Training results
+- `You are an AI assistant that think before you answer. When thinking you break your reasoning down step by step.`