{"action":"created","issue":{"url":"https://api.github.com/repos/uber/sbnet/issues/24","repository_url":"https://api.github.com/repos/uber/sbnet","labels_url":"https://api.github.com/repos/uber/sbnet/issues/24/labels{/name}","comments_url":"https://api.github.com/repos/uber/sbnet/issues/24/comments","events_url":"https://api.github.com/repos/uber/sbnet/issues/24/events","html_url":"https://github.com/uber/sbnet/issues/24","id":355753607,"node_id":"MDU6SXNzdWUzNTU3NTM2MDc=","number":24,"title":"Huge loss in accuracy using SBNet","user":{"login":"dhingratul","id":4759327,"node_id":"MDQ6VXNlcjQ3NTkzMjc=","avatar_url":"https://avatars2.githubusercontent.com/u/4759327?v=4","gravatar_id":"","url":"https://api.github.com/users/dhingratul","html_url":"https://github.com/dhingratul","followers_url":"https://api.github.com/users/dhingratul/followers","following_url":"https://api.github.com/users/dhingratul/following{/other_user}","gists_url":"https://api.github.com/users/dhingratul/gists{/gist_id}","starred_url":"https://api.github.com/users/dhingratul/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/dhingratul/subscriptions","organizations_url":"https://api.github.com/users/dhingratul/orgs","repos_url":"https://api.github.com/users/dhingratul/repos","events_url":"https://api.github.com/users/dhingratul/events{/privacy}","received_events_url":"https://api.github.com/users/dhingratul/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-08-30T21:04:18Z","updated_at":"2018-09-06T18:48:16Z","closed_at":null,"author_association":"NONE","body":"I was able to see the claimed speedup in terms of inference times, but there is a huge loss in terms of accuracy. Please see a simple MNIST example where I artificially created the images and masks. I am seeing a precision of about 11% which is as good as random guessing with a 10 class problem. I also tried to train the algorithm longer, but the precision saturates at about 11%. Without using SbNet you can get to a precision of ~80% after a few epochs. \r\n\r\n\r\n[MNIST_sbnet_repr.py.tar.gz](https://github.com/uber/sbnet/files/2337932/MNIST_sbnet_repr.py.tar.gz)\r\n"},"comment":{"url":"https://api.github.com/repos/uber/sbnet/issues/comments/419201942","html_url":"https://github.com/uber/sbnet/issues/24#issuecomment-419201942","issue_url":"https://api.github.com/repos/uber/sbnet/issues/24","id":419201942,"node_id":"MDEyOklzc3VlQ29tbWVudDQxOTIwMTk0Mg==","user":{"login":"andrei-pokrovsky","id":11221446,"node_id":"MDQ6VXNlcjExMjIxNDQ2","avatar_url":"https://avatars3.githubusercontent.com/u/11221446?v=4","gravatar_id":"","url":"https://api.github.com/users/andrei-pokrovsky","html_url":"https://github.com/andrei-pokrovsky","followers_url":"https://api.github.com/users/andrei-pokrovsky/followers","following_url":"https://api.github.com/users/andrei-pokrovsky/following{/other_user}","gists_url":"https://api.github.com/users/andrei-pokrovsky/gists{/gist_id}","starred_url":"https://api.github.com/users/andrei-pokrovsky/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/andrei-pokrovsky/subscriptions","organizations_url":"https://api.github.com/users/andrei-pokrovsky/orgs","repos_url":"https://api.github.com/users/andrei-pokrovsky/repos","events_url":"https://api.github.com/users/andrei-pokrovsky/events{/privacy}","received_events_url":"https://api.github.com/users/andrei-pokrovsky/received_events","type":"User","site_admin":false},"created_at":"2018-09-06T18:48:16Z","updated_at":"2018-09-06T18:48:16Z","author_association":"COLLABORATOR","body":"I recommend first trying to reproduce your network results with 0 sparsity (fully dense network) implemented using sbnet. With that approach the results should be identical but if it doesn't work it would indicate there's a bug in your implementation. This way you can bisect the debugging."}}
{"actor":{"display_login":"andrei-pokrovsky"}}