{"action":"created","issue":{"url":"https://api.github.com/repos/rhiever/sklearn-benchmarks/issues/9","repository_url":"https://api.github.com/repos/rhiever/sklearn-benchmarks","labels_url":"https://api.github.com/repos/rhiever/sklearn-benchmarks/issues/9/labels{/name}","comments_url":"https://api.github.com/repos/rhiever/sklearn-benchmarks/issues/9/comments","events_url":"https://api.github.com/repos/rhiever/sklearn-benchmarks/issues/9/events","html_url":"https://github.com/rhiever/sklearn-benchmarks/issues/9","id":137616582,"number":9,"title":"Useful metrics to gather for the benchmark","user":{"login":"rhiever","id":1719223,"avatar_url":"https://avatars.githubusercontent.com/u/1719223?v=3","gravatar_id":"","url":"https://api.github.com/users/rhiever","html_url":"https://github.com/rhiever","followers_url":"https://api.github.com/users/rhiever/followers","following_url":"https://api.github.com/users/rhiever/following{/other_user}","gists_url":"https://api.github.com/users/rhiever/gists{/gist_id}","starred_url":"https://api.github.com/users/rhiever/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rhiever/subscriptions","organizations_url":"https://api.github.com/users/rhiever/orgs","repos_url":"https://api.github.com/users/rhiever/repos","events_url":"https://api.github.com/users/rhiever/events{/privacy}","received_events_url":"https://api.github.com/users/rhiever/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/rhiever/sklearn-benchmarks/labels/question","name":"question","color":"cc317c"}],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2016-03-01T16:12:29Z","updated_at":"2016-03-01T18:15:18Z","closed_at":null,"body":"Currently, we're only calculating balanced accuracy for the supervised classification tasks. What would be useful metrics to gather for the benchmark?"},"comment":{"url":"https://api.github.com/repos/rhiever/sklearn-benchmarks/issues/comments/190838974","html_url":"https://github.com/rhiever/sklearn-benchmarks/issues/9#issuecomment-190838974","issue_url":"https://api.github.com/repos/rhiever/sklearn-benchmarks/issues/9","id":190838974,"user":{"login":"AkshayVarik","id":10430826,"avatar_url":"https://avatars.githubusercontent.com/u/10430826?v=3","gravatar_id":"","url":"https://api.github.com/users/AkshayVarik","html_url":"https://github.com/AkshayVarik","followers_url":"https://api.github.com/users/AkshayVarik/followers","following_url":"https://api.github.com/users/AkshayVarik/following{/other_user}","gists_url":"https://api.github.com/users/AkshayVarik/gists{/gist_id}","starred_url":"https://api.github.com/users/AkshayVarik/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/AkshayVarik/subscriptions","organizations_url":"https://api.github.com/users/AkshayVarik/orgs","repos_url":"https://api.github.com/users/AkshayVarik/repos","events_url":"https://api.github.com/users/AkshayVarik/events{/privacy}","received_events_url":"https://api.github.com/users/AkshayVarik/received_events","type":"User","site_admin":false},"created_at":"2016-03-01T18:15:18Z","updated_at":"2016-03-01T18:15:18Z","body":"Ya once we plot the ROC ,AUC will naturally reflect from that I believe..\nAlso a chi squared test could be done, but that would mean we fit models\nwith different predictors to understand which of them don't add to the\nresidual deviance much..basically will help in dimensionality reduction.. I\nbelieve we haven't done that at all..I was wondering if the variables that\nfall within a model could influence our choice of parameters.. Just a\nthought that I have..\nOn Mar 1, 2016 13:03, \"Randy Olson\" <notifications@github.com> wrote:\n\n> I've been thinking about other metrics, such as AUC (area under the ROC\n> curve), and perhaps those other metrics you mention may be useful also.\n>\n> —\n> Reply to this email directly or view it on GitHub\n> <https://github.com/rhiever/sklearn-benchmarks/issues/9#issuecomment-190835409>\n> .\n>\n"}}