{"action":"created","issue":{"url":"https://api.github.com/repos/tidsp/caffe-jacinto/issues/24","repository_url":"https://api.github.com/repos/tidsp/caffe-jacinto","labels_url":"https://api.github.com/repos/tidsp/caffe-jacinto/issues/24/labels{/name}","comments_url":"https://api.github.com/repos/tidsp/caffe-jacinto/issues/24/comments","events_url":"https://api.github.com/repos/tidsp/caffe-jacinto/issues/24/events","html_url":"https://github.com/tidsp/caffe-jacinto/issues/24","id":361647691,"node_id":"MDU6SXNzdWUzNjE2NDc2OTE=","number":24,"title":"Quantization failed when testing SSD","user":{"login":"gasburner","id":32696651,"node_id":"MDQ6VXNlcjMyNjk2NjUx","avatar_url":"https://avatars3.githubusercontent.com/u/32696651?v=4","gravatar_id":"","url":"https://api.github.com/users/gasburner","html_url":"https://github.com/gasburner","followers_url":"https://api.github.com/users/gasburner/followers","following_url":"https://api.github.com/users/gasburner/following{/other_user}","gists_url":"https://api.github.com/users/gasburner/gists{/gist_id}","starred_url":"https://api.github.com/users/gasburner/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gasburner/subscriptions","organizations_url":"https://api.github.com/users/gasburner/orgs","repos_url":"https://api.github.com/users/gasburner/repos","events_url":"https://api.github.com/users/gasburner/events{/privacy}","received_events_url":"https://api.github.com/users/gasburner/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2018-09-19T09:01:31Z","updated_at":"2018-09-22T05:51:41Z","closed_at":null,"author_association":"NONE","body":"First,I tried the quantization function in mnist, and it works well.\r\nThen,I tried the quantization function on the SSD provided in caffe-jacinto-models and the original SSD.\r\nBoth of them failed with error == cudaSuccess (7 vs. 0) too many resources requested for lanuch.\r\nAlso I noticed that I could get the right result only if I set the iterations=1. (no matter what batch size i set or how many gpus i used.)\r\nAnd in the log provided in caffe-jacinto-models, I can't find the test_quantize part,which makes me quiet confused.\r\nSo please tell me how to use the quantization function in SSD correctly.\r\nThank you very much.\r\n\r\n\r\nHere is my log.\r\n\r\n\r\nI0910 14:36:28.498530 13312 common.cpp:475] GPU 0 'TITAN Xp' has compute capability 6.1\r\nI0910 14:36:29.036025 13312 caffe.cpp:902] This is NVCaffe 0.17.0 started at Mon Sep 10 14:36:28 2018\r\nI0910 14:36:29.036056 13312 caffe.cpp:904] CuDNN version: 7104\r\nI0910 14:36:29.036072 13312 caffe.cpp:905] CuBLAS version: 9000\r\nI0910 14:36:29.036077 13312 caffe.cpp:906] CUDA version: 9000\r\nI0910 14:36:29.036082 13312 caffe.cpp:907] CUDA driver version: 9010\r\nI0910 14:36:29.036089 13312 caffe.cpp:908] Arguments:\r\n[0]: /home/junxiang/caffe-jacinto/build/tools/caffe.bin\r\n[1]: test_detection\r\n[2]: --model=training/voc0712/JDetNet/20180828_14-57_ds_PSP_dsFac_32_hdDS8_1/test_quantize/test.prototxt\r\n[3]: --iterations=496\r\n[4]: --weights=training/voc0712/JDetNet/20180828_14-57_ds_PSP_dsFac_32_hdDS8_1/sparse/voc0712_ssdJacintoNetV2_iter_120000.caffemodel\r\n…………………………………………\r\n\r\nI0910 14:36:48.912307 13312 net.cpp:2195] Enabling quantization at output of: Concat mbox_loc\r\nI0910 14:36:48.912477 13312 net.cpp:2195] Enabling quantization at output of: Concat mbox_conf\r\nI0910 14:36:48.912649 13312 net.cpp:2195] Enabling quantization at output of: Concat mbox_priorbox\r\nI0910 14:36:48.917215 13350 common.cpp:192] New stream 0x7fa3ac006960, device 0, thread 13350\r\nF0910 14:36:48.941680 13312 permute_layer.cu:70] Check failed: error == cudaSuccess (7 vs. 0) too many resources requested for launch\r\n*** Check failure stack trace: ***\r\n@ 0x7fa4660295cd google::LogMessage::Fail()\r\n@ 0x7fa46602b433 google::LogMessage::SendToLog()\r\n@ 0x7fa46602915b google::LogMessage::Flush()\r\n@ 0x7fa46602be1e google::LogMessageFatal::~LogMessageFatal()\r\n@ 0x7fa467a7ce48 caffe::PermuteLayer<>::Forward_gpu()\r\n@ 0x7fa4673e8a7f caffe::Layer<>::Forward()\r\n@ 0x7fa4672561fe caffe::Net::ForwardFromTo()\r\n@ 0x7fa46725633d caffe::Net::Forward()\r\n@ 0x44cc4b test_detection()\r\n@ 0x4521f2 main\r\n@ 0x7fa4647ab830 __libc_start_main\r\n@ 0x449699 _start\r\n@ (nil) (unknown)\r\n\r\nThanks a lot!"},"comment":{"url":"https://api.github.com/repos/tidsp/caffe-jacinto/issues/comments/423719030","html_url":"https://github.com/tidsp/caffe-jacinto/issues/24#issuecomment-423719030","issue_url":"https://api.github.com/repos/tidsp/caffe-jacinto/issues/24","id":423719030,"node_id":"MDEyOklzc3VlQ29tbWVudDQyMzcxOTAzMA==","user":{"login":"mathmanu","id":2990503,"node_id":"MDQ6VXNlcjI5OTA1MDM=","avatar_url":"https://avatars0.githubusercontent.com/u/2990503?v=4","gravatar_id":"","url":"https://api.github.com/users/mathmanu","html_url":"https://github.com/mathmanu","followers_url":"https://api.github.com/users/mathmanu/followers","following_url":"https://api.github.com/users/mathmanu/following{/other_user}","gists_url":"https://api.github.com/users/mathmanu/gists{/gist_id}","starred_url":"https://api.github.com/users/mathmanu/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mathmanu/subscriptions","organizations_url":"https://api.github.com/users/mathmanu/orgs","repos_url":"https://api.github.com/users/mathmanu/repos","events_url":"https://api.github.com/users/mathmanu/events{/privacy}","received_events_url":"https://api.github.com/users/mathmanu/received_events","type":"User","site_admin":false},"created_at":"2018-09-22T05:51:41Z","updated_at":"2018-09-22T05:51:41Z","author_association":"NONE","body":"Can you run train_image_object_detection.sh without any change (use the network that uses by default) and let me know if you get a crash? You can reduce max_iter values int the script to perform a quick training."}}
{"actor":{"display_login":"mathmanu"}}