{"action":"created","issue":{"url":"https://api.github.com/repos/neptune-ml/open-solution-mapping-challenge/issues/173","repository_url":"https://api.github.com/repos/neptune-ml/open-solution-mapping-challenge","labels_url":"https://api.github.com/repos/neptune-ml/open-solution-mapping-challenge/issues/173/labels{/name}","comments_url":"https://api.github.com/repos/neptune-ml/open-solution-mapping-challenge/issues/173/comments","events_url":"https://api.github.com/repos/neptune-ml/open-solution-mapping-challenge/issues/173/events","html_url":"https://github.com/neptune-ml/open-solution-mapping-challenge/issues/173","id":350088816,"node_id":"MDU6SXNzdWUzNTAwODg4MTY=","number":173,"title":"How prediction confidence score is calculated?","user":{"login":"maksimovkonstantin","id":10925437,"node_id":"MDQ6VXNlcjEwOTI1NDM3","avatar_url":"https://avatars2.githubusercontent.com/u/10925437?v=4","gravatar_id":"","url":"https://api.github.com/users/maksimovkonstantin","html_url":"https://github.com/maksimovkonstantin","followers_url":"https://api.github.com/users/maksimovkonstantin/followers","following_url":"https://api.github.com/users/maksimovkonstantin/following{/other_user}","gists_url":"https://api.github.com/users/maksimovkonstantin/gists{/gist_id}","starred_url":"https://api.github.com/users/maksimovkonstantin/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/maksimovkonstantin/subscriptions","organizations_url":"https://api.github.com/users/maksimovkonstantin/orgs","repos_url":"https://api.github.com/users/maksimovkonstantin/repos","events_url":"https://api.github.com/users/maksimovkonstantin/events{/privacy}","received_events_url":"https://api.github.com/users/maksimovkonstantin/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-08-13T15:52:39Z","updated_at":"2018-08-14T08:50:04Z","closed_at":null,"author_association":"NONE","body":"Hi!\r\n\r\nHow is the prediction confidence score calculated?\r\n\r\nCan you pls point this part in your code, my coding experience is not enough to go through all the code myself and find(( "},"comment":{"url":"https://api.github.com/repos/neptune-ml/open-solution-mapping-challenge/issues/comments/412801207","html_url":"https://github.com/neptune-ml/open-solution-mapping-challenge/issues/173#issuecomment-412801207","issue_url":"https://api.github.com/repos/neptune-ml/open-solution-mapping-challenge/issues/173","id":412801207,"node_id":"MDEyOklzc3VlQ29tbWVudDQxMjgwMTIwNw==","user":{"login":"jakubczakon","id":20753508,"node_id":"MDQ6VXNlcjIwNzUzNTA4","avatar_url":"https://avatars0.githubusercontent.com/u/20753508?v=4","gravatar_id":"","url":"https://api.github.com/users/jakubczakon","html_url":"https://github.com/jakubczakon","followers_url":"https://api.github.com/users/jakubczakon/followers","following_url":"https://api.github.com/users/jakubczakon/following{/other_user}","gists_url":"https://api.github.com/users/jakubczakon/gists{/gist_id}","starred_url":"https://api.github.com/users/jakubczakon/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jakubczakon/subscriptions","organizations_url":"https://api.github.com/users/jakubczakon/orgs","repos_url":"https://api.github.com/users/jakubczakon/repos","events_url":"https://api.github.com/users/jakubczakon/events{/privacy}","received_events_url":"https://api.github.com/users/jakubczakon/received_events","type":"User","site_admin":false},"created_at":"2018-08-14T08:50:04Z","updated_at":"2018-08-14T08:50:04Z","author_association":"COLLABORATOR","body":"Sure @maksimovkonstantin no problem.\r\n\r\nSo in the \"vanilla\" unet version we simply weigh object predictions by area which is done here:\r\n\r\nhttps://github.com/neptune-ml/open-solution-mapping-challenge/blob/master/src/postprocessing.py#L228-L236\r\n\r\nThat gets you to around 0.927 ish .\r\n\r\nHowever, if you want to use second level model and get to that 0.943 you need to train your second level model. Basically what it does is it predicts iou per building and use those predictions as scores.\r\n\r\nFirst features are extracted with this:\r\nhttps://github.com/neptune-ml/open-solution-mapping-challenge/blob/master/src/postprocessing.py#L18-L25\r\n\r\nthen model is trained with this:\r\n\r\nhttps://github.com/neptune-ml/open-solution-mapping-challenge/blob/master/src/models.py#L212-L249\r\n\r\n\r\n\r\n"}}
{"actor":{"display_login":"jakubczakon"}}