{"action":"created","issue":{"url":"https://api.github.com/repos/amdegroot/ssd.pytorch/issues/62","repository_url":"https://api.github.com/repos/amdegroot/ssd.pytorch","labels_url":"https://api.github.com/repos/amdegroot/ssd.pytorch/issues/62/labels{/name}","comments_url":"https://api.github.com/repos/amdegroot/ssd.pytorch/issues/62/comments","events_url":"https://api.github.com/repos/amdegroot/ssd.pytorch/issues/62/events","html_url":"https://github.com/amdegroot/ssd.pytorch/issues/62","id":256981915,"node_id":"MDU6SXNzdWUyNTY5ODE5MTU=","number":62,"title":"Run eval.py and result is 71.x, compared with yours 77.x","user":{"login":"hli2020","id":9053054,"node_id":"MDQ6VXNlcjkwNTMwNTQ=","avatar_url":"https://avatars1.githubusercontent.com/u/9053054?v=4","gravatar_id":"","url":"https://api.github.com/users/hli2020","html_url":"https://github.com/hli2020","followers_url":"https://api.github.com/users/hli2020/followers","following_url":"https://api.github.com/users/hli2020/following{/other_user}","gists_url":"https://api.github.com/users/hli2020/gists{/gist_id}","starred_url":"https://api.github.com/users/hli2020/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hli2020/subscriptions","organizations_url":"https://api.github.com/users/hli2020/orgs","repos_url":"https://api.github.com/users/hli2020/repos","events_url":"https://api.github.com/users/hli2020/events{/privacy}","received_events_url":"https://api.github.com/users/hli2020/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":6,"created_at":"2017-09-12T09:44:47Z","updated_at":"2018-09-05T07:04:32Z","closed_at":null,"author_association":"NONE","body":"Hi,\r\n\r\nI just finished the training process and run the test process by `eval.py`; however, I got a much lower result (compared with the results reported in `readme`, see [here](https://github.com/amdegroot/ssd.pytorch#map).). After a further digging, there are some concerns:\r\n+ the parameters, confidence_threshold and top_k, are not used at all. See [here](https://github.com/amdegroot/ssd.pytorch/blob/master/eval.py#L41).\r\n+ there is no NMS process? See [here](https://github.com/amdegroot/ssd.pytorch/blob/master/eval.py#L301).\r\n\r\nOne more thing, you report that **without pre-training and using data augmentation alone, you have a 77.43% performance**. What setting do you use? I just Xavier init all layers without a pretrained model, keeping all other parameters unchanged as in your repo; the training completely fails (at first, the loss is ~15; then 20k iter it goes down to 7.x and kept the same for the rest iter (max_iter=120k); the test mAP is 0.4, cf with pre-train 71.x that I got).\r\n\r\nThanks so much for your help!\r\nHongyang, Francis"},"comment":{"url":"https://api.github.com/repos/amdegroot/ssd.pytorch/issues/comments/418622029","html_url":"https://github.com/amdegroot/ssd.pytorch/issues/62#issuecomment-418622029","issue_url":"https://api.github.com/repos/amdegroot/ssd.pytorch/issues/62","id":418622029,"node_id":"MDEyOklzc3VlQ29tbWVudDQxODYyMjAyOQ==","user":{"login":"kimna4","id":30453641,"node_id":"MDQ6VXNlcjMwNDUzNjQx","avatar_url":"https://avatars2.githubusercontent.com/u/30453641?v=4","gravatar_id":"","url":"https://api.github.com/users/kimna4","html_url":"https://github.com/kimna4","followers_url":"https://api.github.com/users/kimna4/followers","following_url":"https://api.github.com/users/kimna4/following{/other_user}","gists_url":"https://api.github.com/users/kimna4/gists{/gist_id}","starred_url":"https://api.github.com/users/kimna4/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/kimna4/subscriptions","organizations_url":"https://api.github.com/users/kimna4/orgs","repos_url":"https://api.github.com/users/kimna4/repos","events_url":"https://api.github.com/users/kimna4/events{/privacy}","received_events_url":"https://api.github.com/users/kimna4/received_events","type":"User","site_admin":false},"created_at":"2018-09-05T07:04:32Z","updated_at":"2018-09-05T07:04:32Z","author_association":"NONE","body":"Really thank you for your reply.\n\nBut, I didn't modify source codes of the project.\n\nI just modified my comment. If you have time, could you read my comment\nagain?\n\nThank you in advance\n\nInhan Kim\n\n\n\n2018년 9월 5일 (수) 오후 3:31, Hongyang Li <notifications@github.com>님이 작성:\n\n> There must be some data format/ ground truth annotation error in your code\n> to have a low mAP. (eg, x, y, w, h or x1 y1 x2 y2)\n> On Wed, Sep 5, 2018 at 08:04 kimna4 <notifications@github.com> wrote:\n>\n> > @hli2020 <https://github.com/hli2020> @kalyo-zjl\n> > <https://github.com/kalyo-zjl> @OxInsky <https://github.com/OxInsky> I\n> > tried many times to get 77.x mAP. But I can't. The maximum mAP is 0.4x on\n> > VOC 2007 test set. Have you solved this? please tell me the parameters.\n> > Thank you\n> >\n> > —\n> > You are receiving this because you were mentioned.\n> > Reply to this email directly, view it on GitHub\n> > <\n> https://github.com/amdegroot/ssd.pytorch/issues/62#issuecomment-418609584\n> >,\n> > or mute the thread\n> > <\n> https://github.com/notifications/unsubscribe-auth/AIojfkG9xtyei3puU5CvU18MkZihjnQNks5uX2lcgaJpZM4PUYVU\n> >\n> > .\n> >\n> --\n> Hongyang\n>\n> —\n> You are receiving this because you commented.\n> Reply to this email directly, view it on GitHub\n> <https://github.com/amdegroot/ssd.pytorch/issues/62#issuecomment-418614682>,\n> or mute the thread\n> <https://github.com/notifications/unsubscribe-auth/AdCviVf0R3otOw0uf8XzouorEndHvgqmks5uX2-rgaJpZM4PUYVU>\n> .\n>\n"}}
{"actor":{"display_login":"kimna4"}}