{"action":"created","issue":{"url":"https://api.github.com/repos/rbgirshick/py-faster-rcnn/issues/256","repository_url":"https://api.github.com/repos/rbgirshick/py-faster-rcnn","labels_url":"https://api.github.com/repos/rbgirshick/py-faster-rcnn/issues/256/labels{/name}","comments_url":"https://api.github.com/repos/rbgirshick/py-faster-rcnn/issues/256/comments","events_url":"https://api.github.com/repos/rbgirshick/py-faster-rcnn/issues/256/events","html_url":"https://github.com/rbgirshick/py-faster-rcnn/issues/256","id":165486408,"number":256,"title":"low evaluation results (mAP) on VOC 2007 using python eval on ZF net???","user":{"login":"hengck23","id":5112846,"avatar_url":"https://avatars.githubusercontent.com/u/5112846?v=3","gravatar_id":"","url":"https://api.github.com/users/hengck23","html_url":"https://github.com/hengck23","followers_url":"https://api.github.com/users/hengck23/followers","following_url":"https://api.github.com/users/hengck23/following{/other_user}","gists_url":"https://api.github.com/users/hengck23/gists{/gist_id}","starred_url":"https://api.github.com/users/hengck23/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hengck23/subscriptions","organizations_url":"https://api.github.com/users/hengck23/orgs","repos_url":"https://api.github.com/users/hengck23/repos","events_url":"https://api.github.com/users/hengck23/events{/privacy}","received_events_url":"https://api.github.com/users/hengck23/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2016-07-14T06:33:59Z","updated_at":"2016-08-18T09:19:11Z","closed_at":"2016-07-14T09:03:21Z","body":"Hi, I am getting low results of Mean AP = 0.444 using the python eval code provided. Please let me know if my results are correct?\r\n\r\nsetting:\r\nZF_faster_rcnn_final.caffemodel  (in dowloaded \"faster_rcnn_models.tgz\") \r\ntest.prototxt\r\nfaster_rcnn_end2end.yml\r\nTEST IMDB=\"voc_2007_test\"\r\n\r\nmy results:\r\n~~~~~~~~\r\nResults:\r\n0.493\r\n0.508\r\n0.386\r\n0.284\r\n0.130\r\n0.565\r\n0.504\r\n0.650\r\n0.215\r\n0.347\r\n0.604\r\n0.584\r\n0.674\r\n0.540\r\n0.420\r\n0.177\r\n0.305\r\n0.511\r\n0.660\r\n0.316\r\n0.444 (mAP)\r\n~~~~~~~~\r\n\r\nResults computed with the unofficial Python eval code.\r\nResults should be very close to the official MATLAB eval code.\r\nRecompute with `./tools/reval.py --matlab ...` for your paper.\r\n\r\nPress any key to continue  "},"comment":{"url":"https://api.github.com/repos/rbgirshick/py-faster-rcnn/issues/comments/240669110","html_url":"https://github.com/rbgirshick/py-faster-rcnn/issues/256#issuecomment-240669110","issue_url":"https://api.github.com/repos/rbgirshick/py-faster-rcnn/issues/256","id":240669110,"user":{"login":"thomaspreece","id":2067021,"avatar_url":"https://avatars.githubusercontent.com/u/2067021?v=3","gravatar_id":"","url":"https://api.github.com/users/thomaspreece","html_url":"https://github.com/thomaspreece","followers_url":"https://api.github.com/users/thomaspreece/followers","following_url":"https://api.github.com/users/thomaspreece/following{/other_user}","gists_url":"https://api.github.com/users/thomaspreece/gists{/gist_id}","starred_url":"https://api.github.com/users/thomaspreece/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/thomaspreece/subscriptions","organizations_url":"https://api.github.com/users/thomaspreece/orgs","repos_url":"https://api.github.com/users/thomaspreece/repos","events_url":"https://api.github.com/users/thomaspreece/events{/privacy}","received_events_url":"https://api.github.com/users/thomaspreece/received_events","type":"User","site_admin":false},"created_at":"2016-08-18T09:19:11Z","updated_at":"2016-08-18T09:19:11Z","body":"The downloaded model does not have lower performance. You were running the evaluation with the incorrect prototxt file. The downloaded models are trained using the faster_rcnn_alt_opt method, so you should of used the faster_rcnn_alt_opt/faster_rcnn_test.pt.\r\nFor all intents and purposes the two model files are the same except for some reason a couple of layers are renamed on the end2end version and hence when you run it in caffe it doesn't use the pretrained weights for that layer as it cannot match them to the name, this results in random weights being assigned to that layer and is the reason why running the evaluation multiple times results in slightly different accuracies.\r\n"}}
{"actor":{"display_login":"thomaspreece"}}