{"action":"created","issue":{"url":"https://api.github.com/repos/gcp/leela-zero/issues/1905","repository_url":"https://api.github.com/repos/gcp/leela-zero","labels_url":"https://api.github.com/repos/gcp/leela-zero/issues/1905/labels{/name}","comments_url":"https://api.github.com/repos/gcp/leela-zero/issues/1905/comments","events_url":"https://api.github.com/repos/gcp/leela-zero/issues/1905/events","html_url":"https://github.com/gcp/leela-zero/issues/1905","id":366048983,"node_id":"MDU6SXNzdWUzNjYwNDg5ODM=","number":1905,"title":"(ENTIRELY AUTOMATED) Google Cloud Tesla V100 Free Trial : video+text tutorial with managed instance group","user":{"login":"wonderingabout","id":38690718,"node_id":"MDQ6VXNlcjM4NjkwNzE4","avatar_url":"https://avatars3.githubusercontent.com/u/38690718?v=4","gravatar_id":"","url":"https://api.github.com/users/wonderingabout","html_url":"https://github.com/wonderingabout","followers_url":"https://api.github.com/users/wonderingabout/followers","following_url":"https://api.github.com/users/wonderingabout/following{/other_user}","gists_url":"https://api.github.com/users/wonderingabout/gists{/gist_id}","starred_url":"https://api.github.com/users/wonderingabout/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wonderingabout/subscriptions","organizations_url":"https://api.github.com/users/wonderingabout/orgs","repos_url":"https://api.github.com/users/wonderingabout/repos","events_url":"https://api.github.com/users/wonderingabout/events{/privacy}","received_events_url":"https://api.github.com/users/wonderingabout/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":77,"created_at":"2018-10-02T19:37:49Z","updated_at":"2018-11-06T22:12:41Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"hi,\r\n\r\nas i said recently on discord, google colab (tesla K80) doesnt work so well, because after 1-2 hours of computing you are disconnected (no gpu backend) and have to wait another time to get a gpu available... to use it only 1-2 hours before being blocked again, see their faq : https://research.google.com/colaboratory/faq.html#gpu-availability\r\n\r\nhowever, google cloud is different, they give you totally free of charge 300 dollars/257 euros credit that can be used with a Tesla V100 (0,75$/hour of free credit consumed, with preemptible costs with 300 dollars you are good for a very long time for free!),\r\nat the condition that your id is verified with a vald credit card, to avoid abuse (but you'll never be charged anything, even when your free trial credit ends)\r\n\r\n**IMPORTANT UPDATE : \r\nnow, you dont need to manually start autogtp anymore ! see final instructions google doc : \r\nhttps://docs.google.com/document/d/1P_c-RbeLKjv1umc4rMEgvIVrUUZSeY0WAtYHjaxjD64/edit**\r\n\r\n\r\nabout speed of the tesla v100 : \r\ni experimented with it, and its very very powerful !\r\nit was slightly slower during first run during the tutorial, but you can expect to produce games at arround this speed : \r\n- arround 11 +/-1 games 5%resign in 60 minutes with master branch\r\n- and 13 +/-1 games 5%resign in 60 minutes with next branch, \r\n- and 16 +/- 1 games 5%resign in 60 minutes with next branch `-g 2` (`./autogtp -g 2` is faster but requires 4vcpu/5,75gb ram)\r\n-if you have a 0% resign game with 700 moves it will last arround 1 hour with master branch and arround 40 minutes with next branch at the time i'm writing this tutorial\r\n\r\nhere is what i get with next branch (includes 3 no resign games of 700 moves and 2200 seconds each) : \r\n\r\n![google cloud finally 16multi](https://user-images.githubusercontent.com/38690718/46749135-6c204080-ccb5-11e8-9c8b-22f2c95b295f.PNG)\r\n\r\n\r\n\r\nand came with a text and video tutorial (a minimalist one with no explanations, and a detailed one)\r\n\r\nSHORT VERSION : \r\nminimalist text instructions : \r\nhttps://pastebin.com/SL1iBAey\r\n\r\nminimalist and quick tutorial (no explanations) : \r\n(will be remade soon to add all improvements) https://www.youtube.com/watch?v=LBeh4cfVcPg\r\n\r\nLONG VERSION : \r\n\r\nlong and detailed text instructions : \r\nhttps://pastebin.com/552UN25c\r\n\r\ndetailed and long tutorial : \r\n(will be remade soon to add all improvements) https://www.youtube.com/watch?v=64SDJuibv78&\r\n\r\ni myself dont have a very powerful gpu, so i'm glad i can help with 40b which is ressource expansive,\r\n\r\nfrom my calculations, one free trial user can produce arround 250 games per day at a h24 7/7 usage =  games per free trial,\r\nso with 100 people doing this free trial, we'd produce 25 000 extra games per 24 hours, and 175 000 games/week.\r\n\r\nit doesnt consume your ressouce as the V100 runs in the cloud, so just start your instance and forget it, **no need to let the SSH window or your computer open !**\r\n ̷p̷l̷e̷a̷s̷e̷ ̷d̷o̷n̷t̷ ̷f̷o̷r̷g̷e̷t̷ ̷t̷o̷ ̷s̷t̷o̷p̷ ̷t̷h̷e̷ ̷i̷n̷s̷t̷a̷n̷c̷e̷ ̷a̷f̷t̷e̷r̷ ̷y̷o̷u̷ ̷e̷x̷i̷t̷e̷d̷ ̷t̷h̷e̷ ̷s̷s̷h̷ ̷w̷i̷n̷d̷o̷w̷,̷ ̷o̷r̷ ̷t̷h̷e̷ ̷c̷r̷e̷d̷i̷t̷ ̷i̷s̷ ̷w̷a̷s̷t̷e̷d̷ ̷f̷o̷r̷ ̷n̷o̷ ̷r̷e̷a̷s̷o̷n̷,̷ ̷a̷s̷ ̷e̷x̷p̷l̷a̷i̷n̷e̷d̷ ̷i̷n̷ ̷t̷h̷e̷ ̷t̷u̷t̷o̷r̷i̷a̷l̷ not needed to stop your isntance anymore, as autogtp produces games automatically\r\n\r\nlast thing, these ressources can also be used for other projects, like training further the 192x15, as suggested in #1889 \r\n\r\nthe more people use it, the faster development will be,\r\nplease share and thank you for reading this\r\n\r\nedit : \r\nafter a quick search, i found that microsoft azure and oracle cloud offer a free trial credit for GPU cloud computing : \r\n\r\n- Azure (200 dollars) : \r\nhttps://azure.microsoft.com/\r\n\r\n- Oracle (300 dollars) : \r\nhttps://cloud.oracle.com/tryit\r\n\r\nedit 2 : simplified commands in pastebin tutorial, and added part 3b instructions to use next branch (25-30% faster to produce games), and instructions to install latest nvidia short-lived(fresh) drivers in part 1b\r\n\r\nedit 3 : in part 4, `./autogtp -g 2` produces games significantly faster, even with 5% resign only games only (16 vs 13 games in 60 minutes), but also partly because when a 0% resign  game is generated, the extra gpu power of the v100 can be used to produce another game simultaneously\r\nrun instead (requires 4vcpu and 5,75 gb ram)\r\n`./autogtp -g 2`\r\n(265 games/24 hours VS 208 games/24hours)\r\n\r\nedit 4 : \r\n- greatly shrinked commands, much faster to create a new instance now ! \r\n- added startup-script in metadata\r\n\r\n(remove -g 2 if you chose the weak hardware option, slower)\r\nnow, as soon as your instance has started (green circle) you dont need to open an SSH window to start producing games anymore (and you should not, or autogtp will run twice, causing hardware to overload and to be slower and unstable) , and no need to have your computer powered on anymore !\r\n- added : journal command to check the progress of the startup-script \r\n`sudo journalctl -u google-startup-scripts.service -b -e -f` \r\n- added : details of the instance modification after its creation, to add the startup-script metadata after finishing the install of all pre leela-zero packages (nvidia,all packages,etc)\r\n- will do : 2 new video tutorials (minimalist + detailed) as soon as the preemptibility issue is solved (auto restart after preemptible stop)\r\n\r\nedit 5 : (final version) : \r\n- ENTIRELY AUTOMATED added instance creation, delete, and recreation, and auto restart, thanks to the use of an instance template and a managed instance group as @alreadydone suggested\r\n- final version instructions in this google doc : https://docs.google.com/document/d/1P_c-RbeLKjv1umc4rMEgvIVrUUZSeY0WAtYHjaxjD64/edit\r\n- next version pull request README.md can be previewed here : https://github.com/wonderingabout/leela-zero/tree/next\r\n- i plan to make a new video tutorial another time\r\n"},"comment":{"url":"https://api.github.com/repos/gcp/leela-zero/issues/comments/436427887","html_url":"https://github.com/gcp/leela-zero/issues/1905#issuecomment-436427887","issue_url":"https://api.github.com/repos/gcp/leela-zero/issues/1905","id":436427887,"node_id":"MDEyOklzc3VlQ29tbWVudDQzNjQyNzg4Nw==","user":{"login":"alreadydone","id":3064145,"node_id":"MDQ6VXNlcjMwNjQxNDU=","avatar_url":"https://avatars0.githubusercontent.com/u/3064145?v=4","gravatar_id":"","url":"https://api.github.com/users/alreadydone","html_url":"https://github.com/alreadydone","followers_url":"https://api.github.com/users/alreadydone/followers","following_url":"https://api.github.com/users/alreadydone/following{/other_user}","gists_url":"https://api.github.com/users/alreadydone/gists{/gist_id}","starred_url":"https://api.github.com/users/alreadydone/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/alreadydone/subscriptions","organizations_url":"https://api.github.com/users/alreadydone/orgs","repos_url":"https://api.github.com/users/alreadydone/repos","events_url":"https://api.github.com/users/alreadydone/events{/privacy}","received_events_url":"https://api.github.com/users/alreadydone/received_events","type":"User","site_admin":false},"created_at":"2018-11-06T22:12:41Z","updated_at":"2018-11-06T22:12:41Z","author_association":"CONTRIBUTOR","body":"I forgot mentioning one thing: have you ever run `leelaz --tune-only --full-tuner -w ...`? That should speed up things a bit more. It takes a long time and isn't worth doing every time you create a new instance, but you can do it once on each type of GPU and record the content of the leelaz_opencl_tuning  file and maybe add to your script another command to write that content into leelaz_opencl_tuning on the instance."}}
{"actor":{"display_login":"alreadydone"}}