Jacob Bayless
commited on
Commit
·
644965b
1
Parent(s):
ebfd4a6
updated README
Browse files
README.md
CHANGED
|
@@ -1,3 +1,24 @@
|
|
| 1 |
---
|
| 2 |
license: mit
|
|
|
|
|
|
|
| 3 |
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
---
|
| 2 |
license: mit
|
| 3 |
+
language:
|
| 4 |
+
- en
|
| 5 |
---
|
| 6 |
+
|
| 7 |
+
|
| 8 |
+
This is a very basic pyTorch transformer model that sorts lists of numbers. It was trained with nanoGPT.
|
| 9 |
+
|
| 10 |
+
The context window is 256 tokens, so the input list can be up to 127 tokens long. Numbers can be 0 to 99, separated by comma tokens.
|
| 11 |
+
|
| 12 |
+
It was trained for about one day on a laptop with a single NVIDIA RTX 2070 eGPU, so don't expect anything amazing.
|
| 13 |
+
In practice it sorts these lists correctly about 90% of the time, which is good enough to satisfy my curiosity.
|
| 14 |
+
|
| 15 |
+
To run, I recommend cloning nanoGPT (https://github.com/karpathy/nanoGPT) and installing its prerequisites.
|
| 16 |
+
Create a new branch and copy these files into the nanoGPT folder, overwriting the included sample.py and train.py.
|
| 17 |
+
|
| 18 |
+
To run:
|
| 19 |
+
|
| 20 |
+
> python sample.py --out_dir=out-sort-lists --start="(5,4,3,2,1): [" --num_samples=1 --temperature=0.0001 --max_new_tokens=127
|
| 21 |
+
|
| 22 |
+
To train:
|
| 23 |
+
|
| 24 |
+
> python train.py config/train_sort.py
|