{"action":"created","issue":{"url":"https://api.github.com/repos/Tencent/PhoenixGo/issues/75","repository_url":"https://api.github.com/repos/Tencent/PhoenixGo","labels_url":"https://api.github.com/repos/Tencent/PhoenixGo/issues/75/labels{/name}","comments_url":"https://api.github.com/repos/Tencent/PhoenixGo/issues/75/comments","events_url":"https://api.github.com/repos/Tencent/PhoenixGo/issues/75/events","html_url":"https://github.com/Tencent/PhoenixGo/issues/75","id":391445303,"node_id":"MDU6SXNzdWUzOTE0NDUzMDM=","number":75,"title":"Why tensorRT does not support batch size higher than 4 ?","user":{"login":"wonderingabout","id":38690718,"node_id":"MDQ6VXNlcjM4NjkwNzE4","avatar_url":"https://avatars3.githubusercontent.com/u/38690718?v=4","gravatar_id":"","url":"https://api.github.com/users/wonderingabout","html_url":"https://github.com/wonderingabout","followers_url":"https://api.github.com/users/wonderingabout/followers","following_url":"https://api.github.com/users/wonderingabout/following{/other_user}","gists_url":"https://api.github.com/users/wonderingabout/gists{/gist_id}","starred_url":"https://api.github.com/users/wonderingabout/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wonderingabout/subscriptions","organizations_url":"https://api.github.com/users/wonderingabout/orgs","repos_url":"https://api.github.com/users/wonderingabout/repos","events_url":"https://api.github.com/users/wonderingabout/events{/privacy}","received_events_url":"https://api.github.com/users/wonderingabout/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":6,"created_at":"2018-12-16T08:16:32Z","updated_at":"2018-12-20T20:35:03Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"@wodesuck \r\nTensorRT 3.0.4 with cuda 9.0 cudnn 7.1.4 works no problem with batch size 4 on these 2 machines : \r\n\r\nmachine 1 : \r\n- ubuntu 16.04\r\n- ryzen r7 1700, 16gb ram, GTX 1060 6GB\r\n\r\nmachine 2 : \r\n- ubuntu 16.04\r\n- Xeon Phi 2 cores/4 threads (cloud), 16gb ram, Tesla P100\r\n\r\nBut when i increase batch size for example to 8 (or any number which is 5 or more), the engine has a lot of errors and it does not work, reproduced on both of these machines\r\n\r\nExample of errors : \r\nfirst : \r\n![tensor1v2](https://user-images.githubusercontent.com/38690718/50051443-76761500-0112-11e9-8954-60eb6904927f.png)\r\n\r\nthen after a few seconds : \r\n![tensor2v2](https://user-images.githubusercontent.com/38690718/50051457-e4224100-0112-11e9-8a85-306f17f381d0.png)\r\n\r\n\r\nHowever, with batch size 4, tensorRT works good : \r\n![tensor3v3](https://user-images.githubusercontent.com/38690718/50051467-1d5ab100-0113-11e9-9ef7-ccc8a042e95d.png)\r\n\r\nAlso, when enable tensorRT is set to 0 (OFF),\r\n\r\nbatch sizes of 5 and more are supported again : \r\nFor example here i set batch size to 48 (thinking time went down from 20s with batch size 4 tensorRT ON to 6.5s with batch size 48 tensorRT OFF for 4000 simulations per move) on tesla P100 : \r\n\r\n![tensor4](https://user-images.githubusercontent.com/38690718/50051552-fbfac480-0114-11e9-8805-913d75e2cd30.png)\r\n![tensor5](https://user-images.githubusercontent.com/38690718/50051562-2cdaf980-0115-11e9-90ef-a7ac0ab83d23.png)\r\n\r\n\r\nAnd here with batch size 64 tensorRT OFF on tesla P100 : \r\n\r\n![tensor8](https://user-images.githubusercontent.com/38690718/50051575-52680300-0115-11e9-9cec-5aac553a8001.png)\r\n![tensor9](https://user-images.githubusercontent.com/38690718/50051582-6b70b400-0115-11e9-92ed-24ba43491609.png)\r\n\r\n\r\nNote :  without tensorRT, batch size are supported even up to 64 and more (has been tested on Windows 10 with a Tesla V100 successfully)\r\nIncreasing batch size to 8 has the same effect on speed improvement than tensorRT enabled at batch size 4\r\nOf course, batch size 16,32,64 with tensorRT OFF are much faster to compute games than batch size 4 with tensorRT ON\r\n\r\n\r\nSo, any plans to improve the software ?\r\nthanks\r\n\r\n\r\n\r\n "},"comment":{"url":"https://api.github.com/repos/Tencent/PhoenixGo/issues/comments/449127338","html_url":"https://github.com/Tencent/PhoenixGo/issues/75#issuecomment-449127338","issue_url":"https://api.github.com/repos/Tencent/PhoenixGo/issues/75","id":449127338,"node_id":"MDEyOklzc3VlQ29tbWVudDQ0OTEyNzMzOA==","user":{"login":"wonderingabout","id":38690718,"node_id":"MDQ6VXNlcjM4NjkwNzE4","avatar_url":"https://avatars3.githubusercontent.com/u/38690718?v=4","gravatar_id":"","url":"https://api.github.com/users/wonderingabout","html_url":"https://github.com/wonderingabout","followers_url":"https://api.github.com/users/wonderingabout/followers","following_url":"https://api.github.com/users/wonderingabout/following{/other_user}","gists_url":"https://api.github.com/users/wonderingabout/gists{/gist_id}","starred_url":"https://api.github.com/users/wonderingabout/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/wonderingabout/subscriptions","organizations_url":"https://api.github.com/users/wonderingabout/orgs","repos_url":"https://api.github.com/users/wonderingabout/repos","events_url":"https://api.github.com/users/wonderingabout/events{/privacy}","received_events_url":"https://api.github.com/users/wonderingabout/received_events","type":"User","site_admin":false},"created_at":"2018-12-20T20:35:03Z","updated_at":"2018-12-20T20:35:03Z","author_association":"CONTRIBUTOR","body":"update : \r\n\r\nthe contribution that i was doing for OGS website has been finished : \r\nit is featuring PhoenixGo : \r\n\r\nhttps://docs.google.com/document/d/1BsZq980zOX02g34ElwOo3Ec1BiD7tGZYZQfKF73GGUU/edit#\r\n\r\nit will also be available on a separate github\r\n\r\nI hope that tomorrow i will have time to try these steps now"}}
{"actor":{"display_login":"wonderingabout"}}