Text Generation
Transformers
Safetensors
llama
go
text-generation-inference
Inference Endpoints
kenhktsui commited on
Commit
3590575
·
verified ·
1 Parent(s): 296d216

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +6 -3
README.md CHANGED
@@ -18,9 +18,12 @@ MCTS is a decisive factor contributing to the world champion level performance.
18
  With the recent advancement of large language model in transformer[5] based decoder with a next token prediction objective[6], and it's application in Chess[7][8], how does a language model (the GoFormer here) perform in a Go game?
19
  [9] finetunes 124M, 355M, and 744M GPT-2[10] on 56,638 Go game in SGF format. To the best of the knowledge, this is the first time a language model is trained from scratch with 1.36M Go games, with a specially designed tokenizer.
20
 
21
- Can GoFormer perform reasonably well just by next move (token) prediction, without MCTS[3][4]? Let's find out. The hope is that:
22
- - if language model can reason and plan, it can play Go very well. If it cannot, there is something worth investigating.
23
- - if GoFormer can perform reasonably well, it can be used as a baseline for future research in Go game, and even a baseline for heuristic search, without the use of tree search.
 
 
 
24
 
25
  ## Data Preprocessing
26
  We take the leftmost variation of the game tree in SGF format and translate it into PGN.
 
18
  With the recent advancement of large language model in transformer[5] based decoder with a next token prediction objective[6], and it's application in Chess[7][8], how does a language model (the GoFormer here) perform in a Go game?
19
  [9] finetunes 124M, 355M, and 744M GPT-2[10] on 56,638 Go game in SGF format. To the best of the knowledge, this is the first time a language model is trained from scratch with 1.36M Go games, with a specially designed tokenizer.
20
 
21
+ Can GoFormer perform reasonably well just by next move (token) prediction, without MCTS[3][4]? Let's find out.
22
+ My research goals are that:
23
+ - if language model can reason and plan, it can play Go very well.
24
+ - if GoFormer can perform reasonably well, it can be used as a baseline for future research in Go game, without the use of tree search.
25
+
26
+ P.S: it is an intial release of model, and it is expected not to perform very well. But as we have more data, we will see if it can stand a battle with MCTS based engine like [Leela Zero](https://github.com/leela-zero/leela-zero).
27
 
28
  ## Data Preprocessing
29
  We take the leftmost variation of the game tree in SGF format and translate it into PGN.