{"action":"created","issue":{"url":"https://api.github.com/repos/gcp/leela-zero/issues/1030","repository_url":"https://api.github.com/repos/gcp/leela-zero","labels_url":"https://api.github.com/repos/gcp/leela-zero/issues/1030/labels{/name}","comments_url":"https://api.github.com/repos/gcp/leela-zero/issues/1030/comments","events_url":"https://api.github.com/repos/gcp/leela-zero/issues/1030/events","html_url":"https://github.com/gcp/leela-zero/issues/1030","id":304758264,"number":1030,"title":"Bigger, stronger, not faster (4dc12a8e etc)","user":{"login":"gcp","id":1104290,"avatar_url":"https://avatars2.githubusercontent.com/u/1104290?v=4","gravatar_id":"","url":"https://api.github.com/users/gcp","html_url":"https://github.com/gcp","followers_url":"https://api.github.com/users/gcp/followers","following_url":"https://api.github.com/users/gcp/following{/other_user}","gists_url":"https://api.github.com/users/gcp/gists{/gist_id}","starred_url":"https://api.github.com/users/gcp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gcp/subscriptions","organizations_url":"https://api.github.com/users/gcp/orgs","repos_url":"https://api.github.com/users/gcp/repos","events_url":"https://api.github.com/users/gcp/events{/privacy}","received_events_url":"https://api.github.com/users/gcp/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":103,"created_at":"2018-03-13T13:10:53Z","updated_at":"2018-03-18T19:07:13Z","closed_at":null,"author_association":"OWNER","body":"There are a number of tests with bigger networks running, notably 128x20 and 192x10 configurations. Probably due to issue #1025, the 128x20 networks failed rather badly, but 192x10 has been doing OK (it was not affected by the bug due to fortunate circumstances).\r\n\r\nIf the new networks are noticeably stronger than the best 128x10, it makes sense to switch. It looks like we may already be very close to this point. \r\n\r\nWhat the boundary *exactly* is is not so clear: if 128x10 has stalled (it has not, so far), then it's clear. If the new net is several hundreds of Elo stronger, then it's clear. In the middle there's some trade-off between having 1/2 the games played at a higher level, versus being able to iterate new networks faster at a smaller size. I am not worrying too much about this zone, as inevitably we will hit either of the \"clear\" cases sooner rather than later.\r\n\r\nI would also prefer to be able to retrain the 128x20 so it can be tested against 192x10, but I don't consider it a blocker for moving up (if 128x20 ends up surpassing 192x10 significantly, we'll just let it promote...).\r\n\r\n(I originally planned for 128x10 to be the \"final size\" for this \"first run\", but the progress made with bootstrap/net2net, improved client speed and the current strength are certainly making me want to keep going at it for a bit longer...)"},"comment":{"url":"https://api.github.com/repos/gcp/leela-zero/issues/comments/374034437","html_url":"https://github.com/gcp/leela-zero/issues/1030#issuecomment-374034437","issue_url":"https://api.github.com/repos/gcp/leela-zero/issues/1030","id":374034437,"user":{"login":"gcp","id":1104290,"avatar_url":"https://avatars2.githubusercontent.com/u/1104290?v=4","gravatar_id":"","url":"https://api.github.com/users/gcp","html_url":"https://github.com/gcp","followers_url":"https://api.github.com/users/gcp/followers","following_url":"https://api.github.com/users/gcp/following{/other_user}","gists_url":"https://api.github.com/users/gcp/gists{/gist_id}","starred_url":"https://api.github.com/users/gcp/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gcp/subscriptions","organizations_url":"https://api.github.com/users/gcp/orgs","repos_url":"https://api.github.com/users/gcp/repos","events_url":"https://api.github.com/users/gcp/events{/privacy}","received_events_url":"https://api.github.com/users/gcp/received_events","type":"User","site_admin":false},"created_at":"2018-03-18T19:07:13Z","updated_at":"2018-03-18T19:07:13Z","author_association":"OWNER","body":"I had a realization this weekend about the AlphaGo Zero vs Alpha Zero setup, and our choice of 1600 playouts vs 3200 visits etc.\r\n\r\nThe MCTS part of the self-play games is the strength gainer. That is, it takes an existing network, produces stronger games with the search, and then tries to use the output to improve the network again. The \"gain factor\" of a set of games is dependent on the amount of search done.\r\n\r\nIn AlphaGo Zero, they used p=1600, and gated improved on scoring 55%. With Alpha Zero, they lowered it to p=800, which still worked...but they removed the gating. This is important! It's possible that with p=800 the amount of \"gain\" per set of games isn't necessarily good enough to pass the 55% gating.\r\n\r\nThis is relevant to us because when we switched to v=3200, this was the number originally calculated to be equivalent to p=1600 for weaker networks. It was already known that it was on the low end for stronger networks, and that it should be closer to 5000 (IIRC). I didn't pay much attention to this because we knew that Alpha Zero worked with less playouts anyway...but I did not realize the important of the lack of gating in this context.\r\n\r\nIt's quite possible for example the larger networks aren't doing so well because there's simply not much extra info in the games we're generating. A lot of quantity, but less quality. More playouts also improves the possibility of a low policy move (potentially found with noise) entering a positive feedback loop (the Haylee logs show very clearly that the search is able to pick those up, with more playouts).\r\n\r\nSo, I think there's a good chance our procedure will actually work better if we increase the amount of visits. We'll get less games, but more possibility for a new network to beat the 55% gating. Maybe 5000 or 6400? Any thoughts about the above?"}}
{"actor":{"display_login":"gcp"}}