Update README.md

README.md

@@ -26,11 +26,13 @@ Can GoFormer perform reasonably well just by next move (token) prediction, witho
 We take the leftmost variation of the game tree in SGF format and translate it into PGN. 
 
 ## Tokenizer Design
+A tokenizer is designed particularly for Go game.
 Since it is a 19 x 19 game. We use uppercase alphabet to encode x position and lowercase alphabet to encode y position.
 We use alphabet instead of numbers to make a clear that 1 token, but not 2 tokens, represents 1 position, to avoid unnecessary learning to map 2 tokens into 1 position.
 We also use a special token '>' to denote the move by the winner's of the game.  
 While [7][8] does not indicate who is the winner until the result appended at the end, we argue that without indicating the winner, language model cannot know the winner's move during decoding in inference due to the autoregressive nature.
 '>' is the symbol to prompt GoFormer for a move during decoding.
+'X' represents pass.
 
 ## Model Input and Output
 
@@ -51,7 +53,7 @@ To exclude illegal move, we ask GoFormer to suggests K moves, ranked by probabil
 This model achieves an eval_loss of 0.419 at step 7,600 (approximately 10.90 epoch).
 
 ## Future Work
-- Collate more Go data
+- Collate more Go data, particularly self play data. It is quite clear that the size of the existing data is quite trivial compared to modern language model.
 
 # Reference
 [1] Silver, D., Huang, A., Maddison, C. et al. Mastering the game of Go with deep neural networks and tree search. Nature 529, 484–489 (2016).