{"action":"created","issue":{"url":"https://api.github.com/repos/gcp/leela-zero/issues/1313","repository_url":"https://api.github.com/repos/gcp/leela-zero","labels_url":"https://api.github.com/repos/gcp/leela-zero/issues/1313/labels{/name}","comments_url":"https://api.github.com/repos/gcp/leela-zero/issues/1313/comments","events_url":"https://api.github.com/repos/gcp/leela-zero/issues/1313/events","html_url":"https://github.com/gcp/leela-zero/issues/1313","id":320088748,"number":1313,"title":"Handicap training","user":{"login":"Dorus","id":4461459,"avatar_url":"https://avatars2.githubusercontent.com/u/4461459?v=4","gravatar_id":"","url":"https://api.github.com/users/Dorus","html_url":"https://github.com/Dorus","followers_url":"https://api.github.com/users/Dorus/followers","following_url":"https://api.github.com/users/Dorus/following{/other_user}","gists_url":"https://api.github.com/users/Dorus/gists{/gist_id}","starred_url":"https://api.github.com/users/Dorus/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Dorus/subscriptions","organizations_url":"https://api.github.com/users/Dorus/orgs","repos_url":"https://api.github.com/users/Dorus/repos","events_url":"https://api.github.com/users/Dorus/events{/privacy}","received_events_url":"https://api.github.com/users/Dorus/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2018-05-03T21:36:10Z","updated_at":"2018-05-04T08:16:53Z","closed_at":null,"author_association":"NONE","body":"There was some talk about training a handicap NN in the other issue, however it's already quite long and handi games are offtopic there too. So want to continue that part of the discussion elsewhere. \r\n\r\nhttps://github.com/gcp/leela-zero/issues/1311#issuecomment-386282213\r\n> herazul commented 9 hours ago\r\n>\r\n> I think the most interesting thing by far would be to train  a network that can handle several stones handicaps and weird komi, but i  have no clue how. Seems like Golaxy kinda managed to do it, but i have  no idea how.\r\n\r\nhttps://github.com/gcp/leela-zero/issues/1311#issuecomment-386283469\r\n>  gcp commented 9 hours ago \r\n>\r\n> Yes, this is something that would work. If it is the best way to do it, I do not know. Training with (high) handicap is trickier, as you somehow need to  ensure the games are not too lopsided. So you need imbalanced playouts,  or different temperature, or something.\r\n\r\nhttps://github.com/gcp/leela-zero/issues/1311#issuecomment-386287315\r\n>  Eddh commented 9 hours ago •\r\n>\r\n> For a next run I agree that the most interesting things to try would be variable komi and handicap. Following @bittsitt idea, it should be possible to have 1 input plane for komi 7.5, 1 input plane for komi 6.5, 1 input plane for komi 0.5 etc ?\r\n> \r\n> For handicap games, would it be possible to have more random conditions for the game, in addition to the random komi ? Placement of random stones on the board, somewhat randomized playout count, etc. Randomized training condition have already been used in neural network training https://www.youtube.com/watch?v=9EN_HoEk3KY&t=23m36s so that the neural net can better adapt to new conditions.\r\n> \r\n> Also, so that the neural net can take into account the end score rather than just win/loss, is it possible to score a games as 1.15 if it is won by resign, but also 1.15 if it is won with a score that is more than 15. That way the system would want to resign if it is losing by more than 15 points difference, but carry on otherwise, which would be similar to human play.\r\n> \r\n> I am aware that all the pretty ideas will need work and expertise to be implemented at some point though.\r\n\r\nIMHO training with handicap should be easy if you are also willing to train with different komi values. Just start off training with 9 handicap stones but -150 komi to compensate, or 5 handicap stones and -75 komi etc. It might take some fiddling to find the right komi for each handicap value, but eventually you would want to train the value net for any komi. I'm not even convinced komi needs to be a network input, it could indeed be added at the policy and value block only. When playing against handicap stone, the bot could start at -15 komi for each handi stone, and slowly switch up to the required komi each time the win rate for the next komi got into acceptable range. MCTS is said to work best in the 40-60% range, so it could start at -75 komi and switch to -60 komi once that goes into the 40% range (or we might require smaller steps, but the idea stays the same).\r\n\r\nMy basic idea is that against handicap, the perfect game should result in something like -150 points with 9 handi games. The NN only needs to play this perfect game, only when the opponent makes a mistake it should pick up points. you can still tune how aggressively you push towards the played komi of 0.5, the more aggressively you push towards this, the more likely the bot will overplay."},"comment":{"url":"https://api.github.com/repos/gcp/leela-zero/issues/comments/386533939","html_url":"https://github.com/gcp/leela-zero/issues/1313#issuecomment-386533939","issue_url":"https://api.github.com/repos/gcp/leela-zero/issues/1313","id":386533939,"user":{"login":"marcocalignano","id":33530997,"avatar_url":"https://avatars3.githubusercontent.com/u/33530997?v=4","gravatar_id":"","url":"https://api.github.com/users/marcocalignano","html_url":"https://github.com/marcocalignano","followers_url":"https://api.github.com/users/marcocalignano/followers","following_url":"https://api.github.com/users/marcocalignano/following{/other_user}","gists_url":"https://api.github.com/users/marcocalignano/gists{/gist_id}","starred_url":"https://api.github.com/users/marcocalignano/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/marcocalignano/subscriptions","organizations_url":"https://api.github.com/users/marcocalignano/orgs","repos_url":"https://api.github.com/users/marcocalignano/repos","events_url":"https://api.github.com/users/marcocalignano/events{/privacy}","received_events_url":"https://api.github.com/users/marcocalignano/received_events","type":"User","site_admin":false},"created_at":"2018-05-04T08:16:53Z","updated_at":"2018-05-04T08:16:53Z","author_association":"CONTRIBUTOR","body":"I my opinion, since on a Handicap game you play different than usually, you know your opponent is less strong, so you make move that are not the best to confuse your opponent and the get your advantage, one solution would be to train a different network with handicap games. Then we could use the \"normal\" one in an even game, and the \"handicap\" one for handicap game. A bit like a human does.  "}}
{"actor":{"display_login":"marcocalignano"}}