BSC-LT
/

salamandra-7b

Text Generation

text-generation-inference

Inference Endpoints

Model card Files Files and versions Community

mapama247 commited on Sep 30, 2024

Commit

d3157b9

·

1 Parent(s): f884579

add num query groups to table

Files changed (1) hide show

README.md +2 -1

README.md CHANGED Viewed

@@ -83,6 +83,7 @@ The full list of hyperparameters for each model can be found [here](https://gith
 | Layer normalization     | RMS Norm      |
 | Flash attention         | ✅            |
 | Grouped Query Attention | ✅            |
 ---
@@ -629,4 +630,4 @@ The Barcelona Supercomputing Center, as the owner and creator of the model, shal
 |:---:|:---:|:---:|
 |2B| WiP | WiP |
 |7B| [Link](https://huggingface.co/projecte-aina/salamandra-7b) | [Link](https://huggingface.co/projecte-aina/salamandra-7b-instruct) |
-|40B| WiP | WiP |

 | Layer normalization     | RMS Norm      |
 | Flash attention         | ✅            |
 | Grouped Query Attention | ✅            |
+| Num. query groups       | 8             |
 ---
 |:---:|:---:|:---:|
 |2B| WiP | WiP |
 |7B| [Link](https://huggingface.co/projecte-aina/salamandra-7b) | [Link](https://huggingface.co/projecte-aina/salamandra-7b-instruct) |
+|40B| WiP | WiP |