{"action":"created","issue":{"url":"https://api.github.com/repos/gcp/leela-zero/issues/1435","repository_url":"https://api.github.com/repos/gcp/leela-zero","labels_url":"https://api.github.com/repos/gcp/leela-zero/issues/1435/labels{/name}","comments_url":"https://api.github.com/repos/gcp/leela-zero/issues/1435/comments","events_url":"https://api.github.com/repos/gcp/leela-zero/issues/1435/events","html_url":"https://github.com/gcp/leela-zero/pull/1435","id":323553965,"number":1435,"title":"[Experimental] enhance tree search in low winrate situations","user":{"login":"alreadydone","id":3064145,"avatar_url":"https://avatars0.githubusercontent.com/u/3064145?v=4","gravatar_id":"","url":"https://api.github.com/users/alreadydone","html_url":"https://github.com/alreadydone","followers_url":"https://api.github.com/users/alreadydone/followers","following_url":"https://api.github.com/users/alreadydone/following{/other_user}","gists_url":"https://api.github.com/users/alreadydone/gists{/gist_id}","starred_url":"https://api.github.com/users/alreadydone/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alreadydone/subscriptions","organizations_url":"https://api.github.com/users/alreadydone/orgs","repos_url":"https://api.github.com/users/alreadydone/repos","events_url":"https://api.github.com/users/alreadydone/events{/privacy}","received_events_url":"https://api.github.com/users/alreadydone/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":12,"created_at":"2018-05-16T10:06:14Z","updated_at":"2018-05-17T12:46:01Z","closed_at":null,"author_association":"NONE","pull_request":{"url":"https://api.github.com/repos/gcp/leela-zero/pulls/1435","html_url":"https://github.com/gcp/leela-zero/pull/1435","diff_url":"https://github.com/gcp/leela-zero/pull/1435.diff","patch_url":"https://github.com/gcp/leela-zero/pull/1435.patch"},"body":"Idea: use the existing NN (without additional training) more effectively in low winrate situations by modifying the tree search. \r\n\r\n#### 1. make c_puct proportional to winrate\r\nWhen winrate is low, Makes tree search more sensitive to slight change in winrate and not so sensitive to policy prior. Potentially benefits handicapped play; getting from 0.1% to 1% would be a big deal. (Note that the opponent has high (close to 100%) winrate so c_puct on his/her turn is twice as usual (~50% winrate).) fpu reduction is also made proportional to winrate though I'm not sure whether this is the right thing to do.\r\n\r\nSuggested way of testing: \r\n+ match /next with higher playouts against /next with lower playouts but with handicap stones (using the same network)\r\n+ match my patch with higher playouts against /next with lower playouts but with handicap stones\r\n+ compare to see if my patch fares better\r\n+ **added**: when matched against /next with equal playouts in even games, the performance should not be worse; it's designed to work exactly the same way when winrate is 50%\r\n\r\nI'll test myself when I manage to compile. I tried to build in VS2015 through university's cloud service but keep getting \r\n` fatal error C1033: cannot open program database '\\\\Client\\G$\\leela-zero-next\\msvc\\VS2015\\x64\\Debug\\vc140.idb' `\r\n\r\n#### 2. simulate a weaker opponent in tree search by effectively reducing playouts (work in progress)\r\n\r\nWhen backing up the winrate of a newly expanded node to its ancestors, ignore the opponent's moves with a certain probability (depending on current winrate). More precisely, the Q-value of the opponent's last move will be updated with say 90% probability, the second-to-last move with 81% probability, the third-to-last 72.9%, and so on. (Namely the backup process has a 10% chance of terminating at each of the opponent's moves.) The probability should depend on the winrate (of that node or of the root node) or the strength difference with the actual opponent, but I haven't come up with a formula. I think I need to add an argument (whether it's the opponent's turn) and an output (whether backup has terminated) to `play_simulation` to achieve what I want; I'll do it tomorrow."},"comment":{"url":"https://api.github.com/repos/gcp/leela-zero/issues/comments/389853770","html_url":"https://github.com/gcp/leela-zero/pull/1435#issuecomment-389853770","issue_url":"https://api.github.com/repos/gcp/leela-zero/issues/1435","id":389853770,"user":{"login":"larspensjo","id":2412342,"avatar_url":"https://avatars1.githubusercontent.com/u/2412342?v=4","gravatar_id":"","url":"https://api.github.com/users/larspensjo","html_url":"https://github.com/larspensjo","followers_url":"https://api.github.com/users/larspensjo/followers","following_url":"https://api.github.com/users/larspensjo/following{/other_user}","gists_url":"https://api.github.com/users/larspensjo/gists{/gist_id}","starred_url":"https://api.github.com/users/larspensjo/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/larspensjo/subscriptions","organizations_url":"https://api.github.com/users/larspensjo/orgs","repos_url":"https://api.github.com/users/larspensjo/repos","events_url":"https://api.github.com/users/larspensjo/events{/privacy}","received_events_url":"https://api.github.com/users/larspensjo/received_events","type":"User","site_admin":false},"created_at":"2018-05-17T12:46:01Z","updated_at":"2018-05-17T12:46:01Z","author_association":"NONE","body":"@gcp So what you are saying is something like this?\r\n\r\nA naive interpretation when comparing a1% win rate to a 2% win rate would be that the second one has twice as high chance to win. But given the average error on the win rate, it may be impossible to compare a 1% win rate to a 2% win rate in a meaningful way."}}
{"actor":{"display_login":"larspensjo"}}