add BNB
Browse files
README.md
CHANGED
|
@@ -15,9 +15,9 @@ This is a version of `hivemind/gpt-j-6B-8bit` for low-RAM loading, i.e., free Co
|
|
| 15 |
|
| 16 |
> **NOTE:** PRIOR to loading the model, you need to "patch" it to be compatible with loading 8bit weights etc. See the original model card above for details on how to do this.
|
| 17 |
|
| 18 |
-
install `transformers` and `
|
| 19 |
```sh
|
| 20 |
-
pip install transformers accelerate
|
| 21 |
```
|
| 22 |
Patch the model, load using `device_map="auto"`:
|
| 23 |
|
|
|
|
| 15 |
|
| 16 |
> **NOTE:** PRIOR to loading the model, you need to "patch" it to be compatible with loading 8bit weights etc. See the original model card above for details on how to do this.
|
| 17 |
|
| 18 |
+
install `transformers`, `accelerate`, and `bitsandbytes` if needed:
|
| 19 |
```sh
|
| 20 |
+
pip install transformers accelerate bitsandbytes
|
| 21 |
```
|
| 22 |
Patch the model, load using `device_map="auto"`:
|
| 23 |
|