chore: add info on dtypes
Browse files
README.md
CHANGED
|
@@ -32,3 +32,4 @@ StripedHyena is a hybrid architecture composed of multi-head, grouped-query atte
|
|
| 32 |
|
| 33 |
To use StripedHyena outside of the playground, you will need to install custom kernels. Please follow the instructions from the [standalone repository](https://github.com/togethercomputer/stripedhyena).
|
| 34 |
|
|
|
|
|
|
| 32 |
|
| 33 |
To use StripedHyena outside of the playground, you will need to install custom kernels. Please follow the instructions from the [standalone repository](https://github.com/togethercomputer/stripedhyena).
|
| 34 |
|
| 35 |
+
StripedHyena is a mixed precision model. Make sure to keep your `poles` and `residues` in `float32` precision, especially for longer prompts or training.
|