chenwuml commited on
Commit
cf92c17
·
verified ·
1 Parent(s): 1f99523

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +4 -1
README.md CHANGED
@@ -59,7 +59,7 @@ On an AWS `g5.48xlarge` instance, upgrade vLLM to the latest version as per [doc
59
  ```shell
60
  python3 -m vllm.entrypoints.openai.api_server --model amazon/MegaBeam-Mistral-7B-300k --tensor-parallel-size 8
61
  ```
62
- Note that we have set the `max_position_embeddings` in the [`config.json`](config.json) to 288,800 in order to fit model's KV-cache on a single `g5.48xlarge` instance, which has 8 x A10 GPUs (24GB RAM per GPU).
63
 
64
  On an instance with larger GPU RAM (e.g. `p4d.24xlarge`), feel free to increase the value of the `max_position_embeddings`(e.g. to 350K), which the model should be able to process.
65
 
@@ -143,3 +143,6 @@ predictor = model.deploy(
143
 
144
  ## Limitations ##
145
  Before using the MegaBeam-Mistral-7B-300k model, it is important to perform your own independent assessment, and take measures to ensure that your use would comply with your own specific quality control practices and standards, and that your use would comply with the local rules, laws, regulations, licenses and terms that apply to you, and your content.
 
 
 
 
59
  ```shell
60
  python3 -m vllm.entrypoints.openai.api_server --model amazon/MegaBeam-Mistral-7B-300k --tensor-parallel-size 8
61
  ```
62
+ **Important Note** - We have set the `max_position_embeddings` in the [`config.json`](config.json) to 288,800 in order to fit model's KV-cache on a single `g5.48xlarge` instance, which has 8 x A10 GPUs (24GB RAM per GPU).
63
 
64
  On an instance with larger GPU RAM (e.g. `p4d.24xlarge`), feel free to increase the value of the `max_position_embeddings`(e.g. to 350K), which the model should be able to process.
65
 
 
143
 
144
  ## Limitations ##
145
  Before using the MegaBeam-Mistral-7B-300k model, it is important to perform your own independent assessment, and take measures to ensure that your use would comply with your own specific quality control practices and standards, and that your use would comply with the local rules, laws, regulations, licenses and terms that apply to you, and your content.
146
+
147
+ ## The AWS Contributors ##
148
+ Chen Wu, Yin Song, Verdi March, Eden Duthie