Update README.md
Browse files
README.md
CHANGED
@@ -18,9 +18,12 @@ MCTS is a decisive factor contributing to the world champion level performance.
|
|
18 |
With the recent advancement of large language model in transformer[5] based decoder with a next token prediction objective[6], and it's application in Chess[7][8], how does a language model (the GoFormer here) perform in a Go game?
|
19 |
[9] finetunes 124M, 355M, and 744M GPT-2[10] on 56,638 Go game in SGF format. To the best of the knowledge, this is the first time a language model is trained from scratch with 1.36M Go games, with a specially designed tokenizer.
|
20 |
|
21 |
-
Can GoFormer perform reasonably well just by next move (token) prediction, without MCTS[3][4]? Let's find out.
|
22 |
-
|
23 |
-
- if
|
|
|
|
|
|
|
24 |
|
25 |
## Data Preprocessing
|
26 |
We take the leftmost variation of the game tree in SGF format and translate it into PGN.
|
|
|
18 |
With the recent advancement of large language model in transformer[5] based decoder with a next token prediction objective[6], and it's application in Chess[7][8], how does a language model (the GoFormer here) perform in a Go game?
|
19 |
[9] finetunes 124M, 355M, and 744M GPT-2[10] on 56,638 Go game in SGF format. To the best of the knowledge, this is the first time a language model is trained from scratch with 1.36M Go games, with a specially designed tokenizer.
|
20 |
|
21 |
+
Can GoFormer perform reasonably well just by next move (token) prediction, without MCTS[3][4]? Let's find out.
|
22 |
+
My research goals are that:
|
23 |
+
- if language model can reason and plan, it can play Go very well.
|
24 |
+
- if GoFormer can perform reasonably well, it can be used as a baseline for future research in Go game, without the use of tree search.
|
25 |
+
|
26 |
+
P.S: it is an intial release of model, and it is expected not to perform very well. But as we have more data, we will see if it can stand a battle with MCTS based engine like [Leela Zero](https://github.com/leela-zero/leela-zero).
|
27 |
|
28 |
## Data Preprocessing
|
29 |
We take the leftmost variation of the game tree in SGF format and translate it into PGN.
|