|
--- |
|
tags: |
|
- 'mergekit ' |
|
- sliced |
|
license: apache-2.0 |
|
--- |
|
|
|
## Model Details |
|
|
|
This model represents a sliced variant derived from the mistralai/Mistral-7B-v0.1 base model using the mergekit. |
|
|
|
### Model Description |
|
|
|
The model was created by slicing the mistralai/Mistral-7B-v0.1 base model with mergekit, resulting in a model with 916,484,096 parameters. |
|
|
|
Please note, this model is intended for testing purposes and is not designed to generate coherent text. |
|
|
|
## 🧩 Configuration |
|
|
|
```yaml |
|
slices: |
|
- sources: |
|
- model: mistralai/Mistral-7B-v0.1 |
|
layer_range: [0, 3] |
|
|
|
merge_method: passthrough |
|
dtype: bfloat16 |
|
``` |
