{"action":"created","issue":{"url":"https://api.github.com/repos/fchollet/keras/issues/4830","repository_url":"https://api.github.com/repos/fchollet/keras","labels_url":"https://api.github.com/repos/fchollet/keras/issues/4830/labels{/name}","comments_url":"https://api.github.com/repos/fchollet/keras/issues/4830/comments","events_url":"https://api.github.com/repos/fchollet/keras/issues/4830/events","html_url":"https://github.com/fchollet/keras/issues/4830","id":197487724,"number":4830,"title":"Issue when using TensorFlow backend with GPU.","user":{"login":"sixsamuraisoldier","id":13212675,"avatar_url":"https://avatars.githubusercontent.com/u/13212675?v=3","gravatar_id":"","url":"https://api.github.com/users/sixsamuraisoldier","html_url":"https://github.com/sixsamuraisoldier","followers_url":"https://api.github.com/users/sixsamuraisoldier/followers","following_url":"https://api.github.com/users/sixsamuraisoldier/following{/other_user}","gists_url":"https://api.github.com/users/sixsamuraisoldier/gists{/gist_id}","starred_url":"https://api.github.com/users/sixsamuraisoldier/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sixsamuraisoldier/subscriptions","organizations_url":"https://api.github.com/users/sixsamuraisoldier/orgs","repos_url":"https://api.github.com/users/sixsamuraisoldier/repos","events_url":"https://api.github.com/users/sixsamuraisoldier/events{/privacy}","received_events_url":"https://api.github.com/users/sixsamuraisoldier/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":5,"created_at":"2016-12-25T05:08:55Z","updated_at":"2016-12-25T05:46:45Z","closed_at":null,"body":"Hey there, I am having a problem when using the TensorFlow backend with my GPU.\r\nIt starts to train, but my GPU utilization from nvidia-smi shows that there is almost no utilization yet nvcc-version still gives the correct response.\r\nAnd I've also installed nvidia-modprobe\r\n\r\nOutput of nvidia-smi:\r\nSat Dec 24 21:08:00 2016       \r\n+-----------------------------------------------------------------------------+\r\n| NVIDIA-SMI 375.20                 Driver Version: 375.20                    |\r\n|-------------------------------+----------------------+----------------------+\r\n| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |\r\n| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |\r\n|===============================+======================+======================|\r\n|   0  TITAN X (Pascal)    On   | 0000:03:00.0     Off |                  N/A |\r\n| 23%   33C    P8    10W / 250W |    160MiB / 12221MiB |      0%      Default |\r\n+-------------------------------+----------------------+----------------------+\r\n|   1  TITAN X (Pascal)    On   | 0000:04:00.0      On |                  N/A |\r\n| 23%   41C    P8    11W / 250W |    968MiB / 12218MiB |      3%      Default |\r\n+-------------------------------+----------------------+----------------------+\r\n                                                                               \r\n+-----------------------------------------------------------------------------+\r\n| Processes:                                                       GPU Memory |\r\n|  GPU       PID  Type  Process name                               Usage      |\r\n|=============================================================================|\r\n|    0       991    C   /usr/bin/python                                149MiB |\r\n|    1      1421    G   /usr/bin/X                                     580MiB |\r\n|    1      2336    G   compiz                                         152MiB |\r\n|    1      2746    G   /opt/g                                         189MiB |\r\n|    1     25069    G   ...s-passed-by-fd --v8-snapshot-passed-by-fd    34MiB |\r\n+-----------------------------------------------------------------------------\r\n\r\nOutput of nvcc --version:\r\nnvcc: NVIDIA (R) Cuda compiler driver\r\nCopyright (c) 2005-2016 NVIDIA Corporation\r\nBuilt on Wed_May__4_21:01:56_CDT_2016\r\nCuda compilation tools, release 8.0, V8.0.26\r\n\r\n\r\nWhen I run the keras model, I get this as an output:\r\nI tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcublas.so locally\r\nI tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcudnn.so locally\r\nI tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcufft.so locally\r\nI tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcuda.so.1 locally\r\nI tensorflow/stream_executor/dso_loader.cc:128] successfully opened CUDA library libcurand.so locally\r\nCouldn't import dot_parser, loading of dot files will not be possible.\r\nX_train shape: (50000, 32, 32, 3)\r\n50000 train samples\r\n10000 test samples\r\nUsing real-time data augmentation.\r\nEpoch 1/200\r\nE tensorflow/stream_executor/cuda/cuda_driver.cc:509] failed call to cuInit: CUDA_ERROR_UNKNOWN\r\nI tensorflow/stream_executor/cuda/cuda_diagnostics.cc:158] retrieving CUDA diagnostic information for host: Tesseract\r\nI tensorflow/stream_executor/cuda/cuda_diagnostics.cc:165] hostname: Tesseract\r\nI tensorflow/stream_executor/cuda/cuda_diagnostics.cc:189] libcuda reported version is: 375.20.0\r\nI tensorflow/stream_executor/cuda/cuda_diagnostics.cc:363] driver version file contents: \"\"\"NVRM version: NVIDIA UNIX x86_64 Kernel Module  375.20  Tue Nov 15 16:49:10 PST 2016\r\nGCC version:  gcc version 4.8.5 (Ubuntu 4.8.5-2ubuntu1~14.04.1) \r\n\"\"\"\r\nI tensorflow/stream_executor/cuda/cuda_diagnostics.cc:193] kernel reported version is: 375.20.0\r\nI tensorflow/stream_executor/cuda/cuda_diagnostics.cc:300] kernel version seems to match DSO: 375.20.0\r\n\r\nThanks in advance, any help would be much appreciated!\r\n"},"comment":{"url":"https://api.github.com/repos/fchollet/keras/issues/comments/269111165","html_url":"https://github.com/fchollet/keras/issues/4830#issuecomment-269111165","issue_url":"https://api.github.com/repos/fchollet/keras/issues/4830","id":269111165,"user":{"login":"patyork","id":1304633,"avatar_url":"https://avatars.githubusercontent.com/u/1304633?v=3","gravatar_id":"","url":"https://api.github.com/users/patyork","html_url":"https://github.com/patyork","followers_url":"https://api.github.com/users/patyork/followers","following_url":"https://api.github.com/users/patyork/following{/other_user}","gists_url":"https://api.github.com/users/patyork/gists{/gist_id}","starred_url":"https://api.github.com/users/patyork/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/patyork/subscriptions","organizations_url":"https://api.github.com/users/patyork/orgs","repos_url":"https://api.github.com/users/patyork/repos","events_url":"https://api.github.com/users/patyork/events{/privacy}","received_events_url":"https://api.github.com/users/patyork/received_events","type":"User","site_admin":false},"created_at":"2016-12-25T05:46:45Z","updated_at":"2016-12-25T05:46:45Z","body":"In the [TensorFlow thread I referenced](https://github.com/tensorflow/tensorflow/issues/394#issuecomment-165034780), someone had the same experience of it running fine, and then suddenly getting this error. I'd recommend starting there and working your way down the thread applying the things they say. It seems to have resolved it for everyone."}}
{"actor":{"display_login":"patyork"}}