Update README.md
Browse files
README.md
CHANGED
@@ -580,26 +580,20 @@ Experiment, and find out what works and doesn't.
|
|
580 |
2) After you created your account update your billing and navigate to the deploy page.
|
581 |
3) Select the following
|
582 |
- GPU Type: A6000
|
583 |
-
- GPU Quantity:
|
584 |
- Category: Creator
|
585 |
- Image: Jon Durbin
|
586 |
- Coupon Code: JonDurbin
|
587 |
4) Deploy the VM!
|
588 |
5) Navigate to 'Running Instances' to retrieve instructions to login to the VM
|
589 |
6) Once inside the VM, open the terminal and run `volume=$PWD/data`
|
590 |
-
7) Run `model=jondurbin/airoboros-
|
591 |
8) `sudo docker run --gpus all --shm-size 1g -p 8080:80 -v $volume:/data ghcr.io/huggingface/text-generation-inference:1.3 --model-id $model`
|
592 |
9) The model will take some time to load...
|
593 |
10) Once loaded the model will be available on port 8080
|
594 |
|
595 |
For assistance with the VM join the [Massed Compute Discord Server](https://discord.gg/Mj4YMQY3DA)
|
596 |
|
597 |
-
### Latitude.sh
|
598 |
-
|
599 |
-
[Latitude](https://www.latitude.sh/r/4BBD657C) has h100 instances available (as of today, 2024-02-08) for $3/hr!
|
600 |
-
|
601 |
-
They have a few blueprints available for testing LLMs, but a single h100 should be plenty to run this model with 8k ctx.
|
602 |
-
|
603 |
## Support me
|
604 |
|
605 |
- https://bmc.link/jondurbin
|
|
|
580 |
2) After you created your account update your billing and navigate to the deploy page.
|
581 |
3) Select the following
|
582 |
- GPU Type: A6000
|
583 |
+
- GPU Quantity: 4
|
584 |
- Category: Creator
|
585 |
- Image: Jon Durbin
|
586 |
- Coupon Code: JonDurbin
|
587 |
4) Deploy the VM!
|
588 |
5) Navigate to 'Running Instances' to retrieve instructions to login to the VM
|
589 |
6) Once inside the VM, open the terminal and run `volume=$PWD/data`
|
590 |
+
7) Run `model=jondurbin/airoboros-110b-3.3`
|
591 |
8) `sudo docker run --gpus all --shm-size 1g -p 8080:80 -v $volume:/data ghcr.io/huggingface/text-generation-inference:1.3 --model-id $model`
|
592 |
9) The model will take some time to load...
|
593 |
10) Once loaded the model will be available on port 8080
|
594 |
|
595 |
For assistance with the VM join the [Massed Compute Discord Server](https://discord.gg/Mj4YMQY3DA)
|
596 |
|
|
|
|
|
|
|
|
|
|
|
|
|
597 |
## Support me
|
598 |
|
599 |
- https://bmc.link/jondurbin
|