{"action":"created","issue":{"url":"https://api.github.com/repos/BVLC/caffe/issues/4312","repository_url":"https://api.github.com/repos/BVLC/caffe","labels_url":"https://api.github.com/repos/BVLC/caffe/issues/4312/labels{/name}","comments_url":"https://api.github.com/repos/BVLC/caffe/issues/4312/comments","events_url":"https://api.github.com/repos/BVLC/caffe/issues/4312/events","html_url":"https://github.com/BVLC/caffe/issues/4312","id":160170809,"number":4312,"title":"trainning MNIST error with rCDUA","user":{"login":"gaoxian001","id":19929604,"avatar_url":"https://avatars.githubusercontent.com/u/19929604?v=3","gravatar_id":"","url":"https://api.github.com/users/gaoxian001","html_url":"https://github.com/gaoxian001","followers_url":"https://api.github.com/users/gaoxian001/followers","following_url":"https://api.github.com/users/gaoxian001/following{/other_user}","gists_url":"https://api.github.com/users/gaoxian001/gists{/gist_id}","starred_url":"https://api.github.com/users/gaoxian001/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gaoxian001/subscriptions","organizations_url":"https://api.github.com/users/gaoxian001/orgs","repos_url":"https://api.github.com/users/gaoxian001/repos","events_url":"https://api.github.com/users/gaoxian001/events{/privacy}","received_events_url":"https://api.github.com/users/gaoxian001/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2016-06-14T12:38:18Z","updated_at":"2016-06-15T13:21:23Z","closed_at":"2016-06-15T13:21:23Z","body":"Environment：\r\nOS:    CentOS 6.5\r\nGPU:  Tesla K40m\r\n\r\nInstall Caffe in one Client Node that without GPU, when trainning MNIST there are some fatal errors happened, such as:\r\n\r\n[root@Client caffe-master]# ./build/tools/caffe train --solver=examples/mnist/lenet_solver.prototxt -gpu 0\r\nI0614 20:31:55.515300 20796 caffe.cpp:185] Using GPUs 0\r\nI0614 20:31:55.516058 20796 caffe.cpp:190] GPU 0: Tesla K40m\r\nI0614 20:31:55.622133 20796 solver.cpp:48] Initializing solver from parameters: \r\n...\r\nI0614 20:31:56.035938 20796 net.cpp:425] loss <- ip2\r\nI0614 20:31:56.035956 20796 net.cpp:425] loss <- label\r\nI0614 20:31:56.035976 20796 net.cpp:399] loss -> loss\r\nI0614 20:31:56.036015 20796 layer_factory.hpp:77] Creating layer loss\r\n[rCUDA-1684] InfiniBand error: Verb ibv_reg_mr returned -1 (errno 22).\r\n[rCUDA-874] rCUDA error: Socket peer closed connection.\r\n*** Aborted at 1465907516 (unix time) try \"date -d @1465907516\" if you are using GNU date ***\r\nPC: @       0x3396c32925 (unknown)\r\n*** SIGABRT (@0x513c) received by PID 20796 (TID 0x7fa15de4c700) from PID 20796; stack trace: ***\r\n    @       0x339740f710 (unknown)\r\n    @       0x3396c32925 (unknown)\r\n    @       0x3396c34105 (unknown)\r\n    @     0x7fa1629aa571 rCUDAcommIB::waitCompletion<>()\r\n    @     0x7fa1629aa733 rCUDAcommCliIB::getRemoteRes()\r\n    @     0x7fa16537f565 ReceiveFunctionsResult()\r\n    @       0x33974079d1 (unknown)\r\n    @       0x3396ce8b6d (unknown)\r\nAborted (core dumped)"},"comment":{"url":"https://api.github.com/repos/BVLC/caffe/issues/comments/226184815","html_url":"https://github.com/BVLC/caffe/issues/4312#issuecomment-226184815","issue_url":"https://api.github.com/repos/BVLC/caffe/issues/4312","id":226184815,"user":{"login":"seanbell","id":1844189,"avatar_url":"https://avatars.githubusercontent.com/u/1844189?v=3","gravatar_id":"","url":"https://api.github.com/users/seanbell","html_url":"https://github.com/seanbell","followers_url":"https://api.github.com/users/seanbell/followers","following_url":"https://api.github.com/users/seanbell/following{/other_user}","gists_url":"https://api.github.com/users/seanbell/gists{/gist_id}","starred_url":"https://api.github.com/users/seanbell/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/seanbell/subscriptions","organizations_url":"https://api.github.com/users/seanbell/orgs","repos_url":"https://api.github.com/users/seanbell/repos","events_url":"https://api.github.com/users/seanbell/events{/privacy}","received_events_url":"https://api.github.com/users/seanbell/received_events","type":"User","site_admin":false},"created_at":"2016-06-15T13:21:23Z","updated_at":"2016-06-15T13:21:23Z","body":"Yes, closing since I don't think you are using the master branch of BVLC/caffe.  The string `rCUDA` appears nowhere in BVLC/caffe.\r\n\r\nYou should ask whoever is the maintainer of the your codebase for help."}}
{"actor":{"display_login":"seanbell"}}