Upload RECONSTRUCTION.md with huggingface_hub
Browse files- RECONSTRUCTION.md +19 -0
RECONSTRUCTION.md
ADDED
|
@@ -0,0 +1,19 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
|
| 2 |
+
# Model Checkpoint Parts
|
| 3 |
+
This checkpoint is split into 5 parts.
|
| 4 |
+
To reconstruct the model file, run:
|
| 5 |
+
|
| 6 |
+
```python
|
| 7 |
+
import os
|
| 8 |
+
|
| 9 |
+
def join_files(base_path, num_parts, output_path):
|
| 10 |
+
with open(output_path, 'wb') as outfile:
|
| 11 |
+
for i in range(num_parts):
|
| 12 |
+
part_path = f"{base_path}.part{i}"
|
| 13 |
+
with open(part_path, 'rb') as infile:
|
| 14 |
+
outfile.write(infile.read())
|
| 15 |
+
|
| 16 |
+
# Usage:
|
| 17 |
+
join_files("mp_rank_00_model_states.pt", 5, "savanna_evo2_40b.pt")
|
| 18 |
+
```
|
| 19 |
+
|