UUFO-Aigis commited on
Commit
d631310
·
verified ·
1 Parent(s): e1e80be

Added Models.

Browse files
.gitattributes CHANGED
@@ -33,3 +33,6 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
 
 
 
 
33
  *.zip filter=lfs diff=lfs merge=lfs -text
34
  *.zst filter=lfs diff=lfs merge=lfs -text
35
  *tfevents* filter=lfs diff=lfs merge=lfs -text
36
+ Pico-OpenLAiNN-500M_F16.gguf filter=lfs diff=lfs merge=lfs -text
37
+ Pico-OpenLAiNN-500M_Q4_0.gguf filter=lfs diff=lfs merge=lfs -text
38
+ Pico-OpenLAiNN-500M_Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
Pico-OpenLAiNN-500M_F16.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:1dc92ace75ff84e27422c68d7e26ec7ef6041ab2226407dd57ef2e328023490a
3
+ size 1090643808
Pico-OpenLAiNN-500M_Q4_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8cde9b0cb158ddab7659f42b7a0706640fb2de3f5b29b21b0a1226675fd84449
3
+ size 340463520
Pico-OpenLAiNN-500M_Q8_0.gguf ADDED
@@ -0,0 +1,3 @@
 
 
 
 
1
+ version https://git-lfs.github.com/spec/v1
2
+ oid sha256:8b252f4905886ad76bb9c0e529ac11fb58c297c7ba907c8e58f7266f1138fdbd
3
+ size 579737504
README.md ADDED
@@ -0,0 +1,47 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # Pico-OpenLAiNN-500M-GGUF 🤗
2
+
3
+ Hey there fellow researchers, developers, and AI enthusiasts! Today I'm releasing a new, biggest Pico-OpenLAiNN Model. This LLM was trained on the full 32B tokens that the entire Open-PicoLAiNN family is trained on.
4
+
5
+ These are the GGUF quants of the models. For the original models, you can find them [here](https://huggingface.co/UUFO-Aigis/Pico-OpenLAiNN-500M)
6
+ ## Models Overview
7
+
8
+ - **Pico-OpenLAiNN-100**: The smallest of the bunch, this 100M parameter model is perfect for quick experiments and applications where computational resources are *extremely* limited.
9
+ - **Pico-OpenLAiNN-250**: This is the middle child of the PicoLAiNN family, it's still tiny at 250M parameters but is more capable than the 100M parameter model.
10
+ - **Pico-OpenLAiNN-500**: My current "Heavyweight" Model, this model has 500M parameters and is the most capable of the Pico-OpenLAiNN models.
11
+ ## Pretraining Details
12
+
13
+ This specific version of Pico LAiNN was trained on just 32B tokens of the fineweb dataset.
14
+
15
+ ## Other information:
16
+
17
+ - **Compatibility**: Built to be compatible with existing projects that use LLAMA 2's tokenizer and architecture.
18
+ - **Ease of Use**: No need to reinvent the wheel. These models are ready to be plugged into your applications.
19
+ - **Open Source**: Fully open source, so you can tweak, tune, and twist them to your heart's content.
20
+
21
+ # Benchy :3
22
+
23
+ | Tasks | Value | |Stderr|
24
+ |--------------|------:|---|-----:|
25
+ |arc_challenge | 0.1903|± | 0.115|
26
+ |arc_easy | 0.4617|± |0.0102|
27
+ |boolq | 0.6034|± |0.0086|
28
+ |hellaswag | 0.3400|± |0.0047|
29
+ |lambada_openai| 0.3670|± |0.0067|
30
+ |piqa | 0.6795|± |0.0109|
31
+ |winogrande | 0.4925|± |0.0141|
32
+
33
+ ## Future Plans
34
+
35
+ - **More Models**: I'm currenetly training the bigger siblings of this models, including a 1B parameter version and beyond. 2-4 Billion parameter versions are planned. These will be Released as OpenLAiNN.
36
+ - **New architecture**: This is still up in the air and I'm still developing it, and will release if I deem it to be actually useful, so stay tuned, this will likely be named FLaRE-LAiNN.
37
+ - **Paper**: A detailed paper will be made available for those interested in the details.
38
+
39
+ ## Credit Where Credit's Due
40
+
41
+ If you find these models useful and decide to use these models, a link to this repository would be highly appreciated. I am a one man show running this. Thanks 🤗
42
+ ## Contact
43
+ If you have questions, Please reach out to me at [email protected]
44
+
45
+ <p align="center">
46
+ <img src="UUFO.png" alt="U.U.F.O Research Logo" width="250"/>
47
+ </p>
UUFO.png ADDED