Update README.md
Browse files
README.md
CHANGED
@@ -10,7 +10,7 @@ tags:
|
|
10 |
|
11 |
## Model Details
|
12 |
|
13 |
-
We have developed and released the family
|
14 |
|
15 |
We continue to expand [Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) with sound understanding capabilities by leveraging 700M tokens [Instruction Speech v1](https://huggingface.co/datasets/Vi-VLM/Vista) dataset.
|
16 |
|
@@ -37,7 +37,7 @@ We continue to expand [Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-lla
|
|
37 |
## Training process
|
38 |
**Training Metrics Image**: Below is a snapshot of the training loss curve visualized.
|
39 |
|
40 |
-
. This family is natively understanding audio and text input.
|
14 |
|
15 |
We continue to expand [Meta-Llama-3-8B-Instruct](https://huggingface.co/meta-llama/Meta-Llama-3-8B-Instruct) with sound understanding capabilities by leveraging 700M tokens [Instruction Speech v1](https://huggingface.co/datasets/Vi-VLM/Vista) dataset.
|
16 |
|
|
|
37 |
## Training process
|
38 |
**Training Metrics Image**: Below is a snapshot of the training loss curve visualized.
|
39 |
|
40 |
+

|
41 |
|
42 |
### Hardware
|
43 |
|
|
|
61 |
| **epsilon** | 1e-6 |
|
62 |
| **Gradient Cliping** | 1.0 |
|
63 |
|
64 |
+
###
|
65 |
+
Accelerate FSDP Config
|
66 |
|
67 |
```
|
68 |
compute_environment: LOCAL_MACHINE
|
|
|
166 |
```
|
167 |
@article{Llama-3-Sound: Sound Instruction LLM 2024,
|
168 |
title={Llama-3-Sound},
|
169 |
+
author={Homebrew Research},
|
170 |
year=2024,
|
171 |
month=July},
|
172 |
+
url={https://huggingface.co/jan-hq/Jan-Llama3-0708}
|
173 |
```
|
174 |
|
175 |
## Acknowledgement
|