Update README.md
Browse files
README.md
CHANGED
@@ -59,7 +59,7 @@ On an AWS `g5.48xlarge` instance, upgrade vLLM to the latest version as per [doc
|
|
59 |
```shell
|
60 |
python3 -m vllm.entrypoints.openai.api_server --model amazon/MegaBeam-Mistral-7B-300k --tensor-parallel-size 8
|
61 |
```
|
62 |
-
Note
|
63 |
|
64 |
On an instance with larger GPU RAM (e.g. `p4d.24xlarge`), feel free to increase the value of the `max_position_embeddings`(e.g. to 350K), which the model should be able to process.
|
65 |
|
@@ -143,3 +143,6 @@ predictor = model.deploy(
|
|
143 |
|
144 |
## Limitations ##
|
145 |
Before using the MegaBeam-Mistral-7B-300k model, it is important to perform your own independent assessment, and take measures to ensure that your use would comply with your own specific quality control practices and standards, and that your use would comply with the local rules, laws, regulations, licenses and terms that apply to you, and your content.
|
|
|
|
|
|
|
|
59 |
```shell
|
60 |
python3 -m vllm.entrypoints.openai.api_server --model amazon/MegaBeam-Mistral-7B-300k --tensor-parallel-size 8
|
61 |
```
|
62 |
+
**Important Note** - We have set the `max_position_embeddings` in the [`config.json`](config.json) to 288,800 in order to fit model's KV-cache on a single `g5.48xlarge` instance, which has 8 x A10 GPUs (24GB RAM per GPU).
|
63 |
|
64 |
On an instance with larger GPU RAM (e.g. `p4d.24xlarge`), feel free to increase the value of the `max_position_embeddings`(e.g. to 350K), which the model should be able to process.
|
65 |
|
|
|
143 |
|
144 |
## Limitations ##
|
145 |
Before using the MegaBeam-Mistral-7B-300k model, it is important to perform your own independent assessment, and take measures to ensure that your use would comply with your own specific quality control practices and standards, and that your use would comply with the local rules, laws, regulations, licenses and terms that apply to you, and your content.
|
146 |
+
|
147 |
+
## The AWS Contributors ##
|
148 |
+
Chen Wu, Yin Song, Verdi March, Eden Duthie
|