{"action":"created","issue":{"url":"https://api.github.com/repos/VisionLearningGroup/R-C3D/issues/13","repository_url":"https://api.github.com/repos/VisionLearningGroup/R-C3D","labels_url":"https://api.github.com/repos/VisionLearningGroup/R-C3D/issues/13/labels{/name}","comments_url":"https://api.github.com/repos/VisionLearningGroup/R-C3D/issues/13/comments","events_url":"https://api.github.com/repos/VisionLearningGroup/R-C3D/issues/13/events","html_url":"https://github.com/VisionLearningGroup/R-C3D/issues/13","id":305006872,"number":13,"title":"about the test performance on thumos14 dataset","user":{"login":"jxiangli","id":34464136,"avatar_url":"https://avatars2.githubusercontent.com/u/34464136?v=4","gravatar_id":"","url":"https://api.github.com/users/jxiangli","html_url":"https://github.com/jxiangli","followers_url":"https://api.github.com/users/jxiangli/followers","following_url":"https://api.github.com/users/jxiangli/following{/other_user}","gists_url":"https://api.github.com/users/jxiangli/gists{/gist_id}","starred_url":"https://api.github.com/users/jxiangli/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jxiangli/subscriptions","organizations_url":"https://api.github.com/users/jxiangli/orgs","repos_url":"https://api.github.com/users/jxiangli/repos","events_url":"https://api.github.com/users/jxiangli/events{/privacy}","received_events_url":"https://api.github.com/users/jxiangli/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":4,"created_at":"2018-03-14T03:02:36Z","updated_at":"2018-03-19T15:12:06Z","closed_at":null,"author_association":"NONE","body":"Thanks a lot for sharing the code.\r\nWhen i test your released  R-C3D model on THUMOS'14 dataset, the mAP is lower than yours.  The results are as follows:\r\n         Fixed threshold for tiou score: [ 0.1  0.2  0.3  0.4  0.5]\r\n         mean AP at different thresholds: [ 0.44844552  0.42947421  0.40517887  0.36456814  0.29133984]\r\nThe mAP@0.5 is simillar, but the others are bad. I don't know why. could you give me some suggestions?  I'm looking forward to your reply. Thank you very much!"},"comment":{"url":"https://api.github.com/repos/VisionLearningGroup/R-C3D/issues/comments/374247079","html_url":"https://github.com/VisionLearningGroup/R-C3D/issues/13#issuecomment-374247079","issue_url":"https://api.github.com/repos/VisionLearningGroup/R-C3D/issues/13","id":374247079,"user":{"login":"huijuan88","id":5479167,"avatar_url":"https://avatars0.githubusercontent.com/u/5479167?v=4","gravatar_id":"","url":"https://api.github.com/users/huijuan88","html_url":"https://github.com/huijuan88","followers_url":"https://api.github.com/users/huijuan88/followers","following_url":"https://api.github.com/users/huijuan88/following{/other_user}","gists_url":"https://api.github.com/users/huijuan88/gists{/gist_id}","starred_url":"https://api.github.com/users/huijuan88/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/huijuan88/subscriptions","organizations_url":"https://api.github.com/users/huijuan88/orgs","repos_url":"https://api.github.com/users/huijuan88/repos","events_url":"https://api.github.com/users/huijuan88/events{/privacy}","received_events_url":"https://api.github.com/users/huijuan88/received_events","type":"User","site_admin":false},"created_at":"2018-03-19T15:12:06Z","updated_at":"2018-03-19T15:12:06Z","author_association":"COLLABORATOR","body":"The provided model is trained with the provided setting.\r\n\r\nTry to download the new td_cnn_end2end.yml<https://github.com/VisionLearningGroup/R-C3D/blob/master/experiments/thumos14/td_cnn_end2end.yml> file, and re-test to see the results. Don’t need to re-train, just re-test with the new file.\r\n\r\n\r\n\r\n\r\n\r\nOn Mar 19, 2018, at 05:51, jxiangli <notifications@github.com<mailto:notifications@github.com>> wrote:\r\n\r\n\r\nI changed nms threshold to be lower than mAP evaluation threshold(evaluation thresholds - 0.1).\r\n\r\nThe detection results:\r\nIoU thresh : [ 0.1 0.2 0.3 0.4 0.5]\r\nmean AP : [ 0.53106971, 0.50220919, 0.45672874, 0.38260949, 0.2914133 ]\r\n\r\nThe mAP@0.1 and mAP@0.2 are still lower than the results in your paper.\r\n\r\nBesides, I trained the model for 60K iterations without changing the hyperparameters . The buffer is 512 frames.\r\nThe detection results:\r\nIoU thresh : [ 0.1 0.2 0.3 0.4 0.5]\r\nmean AP : [ 0.45547029, 0.44375547, 0.40285578, 0.33335985, 0.26478926 ]\r\n\r\nThen I changed the buffer to 768 frames, and the model was trained for 56K iterations . The detection results are as follows.\r\nThe detection results:\r\nIoU thresh : [ 0.1 0.2 0.3 0.4 0.5]\r\nmean AP : [0.48046211, 0.46369506, 0.42404733, 0.35621547, 0.26572283 ]\r\n\r\nAs you can see, the results are both bad. I am not sure whether I missed something. Could you help me?\r\nLooking forward to your reply!\r\n\r\n—\r\nYou are receiving this because you commented.\r\nReply to this email directly, view it on GitHub<https://github.com/VisionLearningGroup/R-C3D/issues/13#issuecomment-374156297>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AFOa_4x7orR-Yiu_WtC-K85J4wrl536Jks5tf3-xgaJpZM4SpxCb>.\r\n\r\n"}}
{"actor":{"display_login":"huijuan88"}}