Update README.md
Browse files
README.md
CHANGED
@@ -7,12 +7,27 @@ tags: []
|
|
7 |
**Lastly, this model has not yet undergone extensive testing. It is probably uncensored, and may output false, sexual, or other undesirable information. Use at your own risk. I am not responsible for any harm caused by this model or its outputs.**
|
8 |
|
9 |
## Datasets used:
|
10 |
-
https://huggingface.co/datasets/G-reen/Duet-v0.5
|
11 |
-
https://huggingface.co/datasets/deepmind/code_contests
|
12 |
-
https://huggingface.co/datasets/nothingiisreal/Reddit-Dirty-And-WritingPrompts
|
13 |
-
https://huggingface.co/datasets/Gryphe/Opus-WritingPrompts
|
14 |
-
https://huggingface.co/datasets/Gryphe/Sonnet3.5-SlimOrcaDedupCleaned
|
15 |
-
https://huggingface.co/datasets/lighteval/MATH-Hard
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
16 |
|
17 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/65a5c0e82823ba72ed2cee7d/8PpaQ1krvuQAh7gKK_MgP.png)
|
18 |
*I'm currently experimenting with various training setups that utilize the free TPU on Kaggle, so the loss probably isn't as optimal as it could be (I ran out of compute and couldn't do any more experiments).
|
|
|
7 |
**Lastly, this model has not yet undergone extensive testing. It is probably uncensored, and may output false, sexual, or other undesirable information. Use at your own risk. I am not responsible for any harm caused by this model or its outputs.**
|
8 |
|
9 |
## Datasets used:
|
10 |
+
- https://huggingface.co/datasets/G-reen/Duet-v0.5
|
11 |
+
- https://huggingface.co/datasets/deepmind/code_contests
|
12 |
+
- https://huggingface.co/datasets/nothingiisreal/Reddit-Dirty-And-WritingPrompts
|
13 |
+
- https://huggingface.co/datasets/Gryphe/Opus-WritingPrompts
|
14 |
+
- https://huggingface.co/datasets/Gryphe/Sonnet3.5-SlimOrcaDedupCleaned
|
15 |
+
- https://huggingface.co/datasets/lighteval/MATH-Hard
|
16 |
+
|
17 |
+
## Prompt format: Llama3 Chat
|
18 |
+
```
|
19 |
+
<|begin_of_text|><|start_header_id|>system<|end_header_id|>
|
20 |
+
|
21 |
+
You are a helpful AI assistant for travel tips and recommendations<|eot_id|><|start_header_id|>user<|end_header_id|>
|
22 |
+
|
23 |
+
What can you help me with?<|eot_id|><|start_header_id|>assistant<|end_header_id|>
|
24 |
+
```
|
25 |
+
|
26 |
+
## System Prompts used during training:
|
27 |
+
- ```You are a creative writer who writes stories based on the given text.```
|
28 |
+
- ```You are an AI assistant that thinks step by step and solves problems.```
|
29 |
+
- ```Roleplay as the below character while also narrating and thinking step by step to solve problems:```
|
30 |
+
- (No system prompt)
|
31 |
|
32 |
![image/png](https://cdn-uploads.huggingface.co/production/uploads/65a5c0e82823ba72ed2cee7d/8PpaQ1krvuQAh7gKK_MgP.png)
|
33 |
*I'm currently experimenting with various training setups that utilize the free TPU on Kaggle, so the loss probably isn't as optimal as it could be (I ran out of compute and couldn't do any more experiments).
|