Added Models.
Browse files- .gitattributes +3 -0
- Pico-OpenLAiNN-500M_F16.gguf +3 -0
- Pico-OpenLAiNN-500M_Q4_0.gguf +3 -0
- Pico-OpenLAiNN-500M_Q8_0.gguf +3 -0
- README.md +47 -0
- UUFO.png +0 -0
.gitattributes
CHANGED
@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
|
|
33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
|
|
|
|
|
|
|
33 |
*.zip filter=lfs diff=lfs merge=lfs -text
|
34 |
*.zst filter=lfs diff=lfs merge=lfs -text
|
35 |
*tfevents* filter=lfs diff=lfs merge=lfs -text
|
36 |
+
Pico-OpenLAiNN-500M_F16.gguf filter=lfs diff=lfs merge=lfs -text
|
37 |
+
Pico-OpenLAiNN-500M_Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
|
38 |
+
Pico-OpenLAiNN-500M_Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
|
Pico-OpenLAiNN-500M_F16.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:1dc92ace75ff84e27422c68d7e26ec7ef6041ab2226407dd57ef2e328023490a
|
3 |
+
size 1090643808
|
Pico-OpenLAiNN-500M_Q4_0.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:8cde9b0cb158ddab7659f42b7a0706640fb2de3f5b29b21b0a1226675fd84449
|
3 |
+
size 340463520
|
Pico-OpenLAiNN-500M_Q8_0.gguf
ADDED
@@ -0,0 +1,3 @@
|
|
|
|
|
|
|
|
|
1 |
+
version https://git-lfs.github.com/spec/v1
|
2 |
+
oid sha256:8b252f4905886ad76bb9c0e529ac11fb58c297c7ba907c8e58f7266f1138fdbd
|
3 |
+
size 579737504
|
README.md
ADDED
@@ -0,0 +1,47 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
1 |
+
# Pico-OpenLAiNN-500M-GGUF 🤗
|
2 |
+
|
3 |
+
Hey there fellow researchers, developers, and AI enthusiasts! Today I'm releasing a new, biggest Pico-OpenLAiNN Model. This LLM was trained on the full 32B tokens that the entire Open-PicoLAiNN family is trained on.
|
4 |
+
|
5 |
+
These are the GGUF quants of the models. For the original models, you can find them [here](https://huggingface.co/UUFO-Aigis/Pico-OpenLAiNN-500M)
|
6 |
+
## Models Overview
|
7 |
+
|
8 |
+
- **Pico-OpenLAiNN-100**: The smallest of the bunch, this 100M parameter model is perfect for quick experiments and applications where computational resources are *extremely* limited.
|
9 |
+
- **Pico-OpenLAiNN-250**: This is the middle child of the PicoLAiNN family, it's still tiny at 250M parameters but is more capable than the 100M parameter model.
|
10 |
+
- **Pico-OpenLAiNN-500**: My current "Heavyweight" Model, this model has 500M parameters and is the most capable of the Pico-OpenLAiNN models.
|
11 |
+
## Pretraining Details
|
12 |
+
|
13 |
+
This specific version of Pico LAiNN was trained on just 32B tokens of the fineweb dataset.
|
14 |
+
|
15 |
+
## Other information:
|
16 |
+
|
17 |
+
- **Compatibility**: Built to be compatible with existing projects that use LLAMA 2's tokenizer and architecture.
|
18 |
+
- **Ease of Use**: No need to reinvent the wheel. These models are ready to be plugged into your applications.
|
19 |
+
- **Open Source**: Fully open source, so you can tweak, tune, and twist them to your heart's content.
|
20 |
+
|
21 |
+
# Benchy :3
|
22 |
+
|
23 |
+
| Tasks | Value | |Stderr|
|
24 |
+
|--------------|------:|---|-----:|
|
25 |
+
|arc_challenge | 0.1903|± | 0.115|
|
26 |
+
|arc_easy | 0.4617|± |0.0102|
|
27 |
+
|boolq | 0.6034|± |0.0086|
|
28 |
+
|hellaswag | 0.3400|± |0.0047|
|
29 |
+
|lambada_openai| 0.3670|± |0.0067|
|
30 |
+
|piqa | 0.6795|± |0.0109|
|
31 |
+
|winogrande | 0.4925|± |0.0141|
|
32 |
+
|
33 |
+
## Future Plans
|
34 |
+
|
35 |
+
- **More Models**: I'm currenetly training the bigger siblings of this models, including a 1B parameter version and beyond. 2-4 Billion parameter versions are planned. These will be Released as OpenLAiNN.
|
36 |
+
- **New architecture**: This is still up in the air and I'm still developing it, and will release if I deem it to be actually useful, so stay tuned, this will likely be named FLaRE-LAiNN.
|
37 |
+
- **Paper**: A detailed paper will be made available for those interested in the details.
|
38 |
+
|
39 |
+
## Credit Where Credit's Due
|
40 |
+
|
41 |
+
If you find these models useful and decide to use these models, a link to this repository would be highly appreciated. I am a one man show running this. Thanks 🤗
|
42 |
+
## Contact
|
43 |
+
If you have questions, Please reach out to me at [email protected]
|
44 |
+
|
45 |
+
<p align="center">
|
46 |
+
<img src="UUFO.png" alt="U.U.F.O Research Logo" width="250"/>
|
47 |
+
</p>
|
UUFO.png
ADDED
![]() |