Juraj Bednar
commited on
More hints on what to do with CUDA Out of memory errors (#925)
Browse files
README.md
CHANGED
|
@@ -1022,6 +1022,10 @@ Please reduce any below
|
|
| 1022 |
- `gradient_accumulation_steps`
|
| 1023 |
- `sequence_len`
|
| 1024 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1025 |
> `failed (exitcode: -9)`
|
| 1026 |
|
| 1027 |
Usually means your system has run out of system memory.
|
|
|
|
| 1022 |
- `gradient_accumulation_steps`
|
| 1023 |
- `sequence_len`
|
| 1024 |
|
| 1025 |
+
If it does not help, try running without deepspeed and without accelerate (replace "accelerate launch" with "python") in the command.
|
| 1026 |
+
|
| 1027 |
+
Using adamw_bnb_8bit might also save you some memory.
|
| 1028 |
+
|
| 1029 |
> `failed (exitcode: -9)`
|
| 1030 |
|
| 1031 |
Usually means your system has run out of system memory.
|