{"action":"created","issue":{"url":"https://api.github.com/repos/tensorflow/tensorflow/issues/4685","repository_url":"https://api.github.com/repos/tensorflow/tensorflow","labels_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/4685/labels{/name}","comments_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/4685/comments","events_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/4685/events","html_url":"https://github.com/tensorflow/tensorflow/issues/4685","id":180324821,"number":4685,"title":"ResourceExhaustedError when formating ImageNet data to TFRecord","user":{"login":"hfutxrg","id":3160803,"avatar_url":"https://avatars.githubusercontent.com/u/3160803?v=3","gravatar_id":"","url":"https://api.github.com/users/hfutxrg","html_url":"https://github.com/hfutxrg","followers_url":"https://api.github.com/users/hfutxrg/followers","following_url":"https://api.github.com/users/hfutxrg/following{/other_user}","gists_url":"https://api.github.com/users/hfutxrg/gists{/gist_id}","starred_url":"https://api.github.com/users/hfutxrg/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/hfutxrg/subscriptions","organizations_url":"https://api.github.com/users/hfutxrg/orgs","repos_url":"https://api.github.com/users/hfutxrg/repos","events_url":"https://api.github.com/users/hfutxrg/events{/privacy}","received_events_url":"https://api.github.com/users/hfutxrg/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":{"login":"cshallue","id":20667461,"avatar_url":"https://avatars.githubusercontent.com/u/20667461?v=3","gravatar_id":"","url":"https://api.github.com/users/cshallue","html_url":"https://github.com/cshallue","followers_url":"https://api.github.com/users/cshallue/followers","following_url":"https://api.github.com/users/cshallue/following{/other_user}","gists_url":"https://api.github.com/users/cshallue/gists{/gist_id}","starred_url":"https://api.github.com/users/cshallue/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cshallue/subscriptions","organizations_url":"https://api.github.com/users/cshallue/orgs","repos_url":"https://api.github.com/users/cshallue/repos","events_url":"https://api.github.com/users/cshallue/events{/privacy}","received_events_url":"https://api.github.com/users/cshallue/received_events","type":"User","site_admin":false},"assignees":[{"login":"cshallue","id":20667461,"avatar_url":"https://avatars.githubusercontent.com/u/20667461?v=3","gravatar_id":"","url":"https://api.github.com/users/cshallue","html_url":"https://github.com/cshallue","followers_url":"https://api.github.com/users/cshallue/followers","following_url":"https://api.github.com/users/cshallue/following{/other_user}","gists_url":"https://api.github.com/users/cshallue/gists{/gist_id}","starred_url":"https://api.github.com/users/cshallue/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cshallue/subscriptions","organizations_url":"https://api.github.com/users/cshallue/orgs","repos_url":"https://api.github.com/users/cshallue/repos","events_url":"https://api.github.com/users/cshallue/events{/privacy}","received_events_url":"https://api.github.com/users/cshallue/received_events","type":"User","site_admin":false}],"milestone":null,"comments":15,"created_at":"2016-09-30T14:56:41Z","updated_at":"2016-09-30T23:02:20Z","closed_at":null,"body":"Hi All Tensorflow users,\r\n\r\nI followed the steps in https://github.com/tensorflow/models/tree/master/inception to build the imagenet to TFRecord format. But when I execute the command \r\n\"bazel-bin/inception/download_and_preprocess_imagenet \"${DATA_DIR}\"\", \r\nI have the following errors:\r\n\r\nException in thread Thread-6:\r\nTraceback (most recent call last):\r\n  File \"/usr/lib64/python2.7/threading.py\", line 811, in __bootstrap_inner\r\n  File \"/usr/lib64/python2.7/threading.py\", line 764, in run\r\n  File \"/home/rengan/.cache/bazel/_bazel_rengan/2c9ddf610cf302ceda3f2b852669382c/execroot/inception/bazel-out/local-fastbuild/bin/inception/build_imagenet_data.runfiles/inception/inception/data/build_imagenet_data.py\", line 388, in _process_image_files_batch\r\n    image_buffer, height, width = _process_image(filename, coder)\r\n  File \"/home/rengan/.cache/bazel/_bazel_rengan/2c9ddf610cf302ceda3f2b852669382c/execroot/inception/bazel-out/local-fastbuild/bin/inception/build_imagenet_data.runfiles/inception/inception/data/build_imagenet_data.py\", line 316, in _process_image\r\n    image_data = tf.gfile.FastGFile(filename, 'r').read()\r\n  File \"/usr/lib/python2.7/site-packages/tensorflow/python/lib/io/file_io.py\", line 102, in read\r\n  File \"/usr/lib/python2.7/site-packages/tensorflow/python/lib/io/file_io.py\", line 72, in _preread_check\r\n  File \"/usr/lib64/python2.7/contextlib.py\", line 24, in __exit__\r\n  File \"/usr/lib/python2.7/site-packages/tensorflow/python/framework/errors.py\", line 463, in raise_exception_on_not_ok_status\r\nResourceExhaustedError: /cm/shared/scratch/rengan/Data/imagenet-data/raw-data/train//n03124170/n03124170_675.JPEG\r\n\r\nThere are similar errors in other threads. It successfully generated 128 validation data but less than 100 training data (should be 1024).\r\n\r\n### What related GitHub issues or StackOverflow threads have you found by searching the web for your problem?: no related issues were found.\r\n\r\n### Environment info\r\nOperating System: Redhat 7.2\r\n\r\nInstalled version of CUDA and cuDNN: \r\n(please attach the output of `ls -l /path/to/cuda/lib/libcud*`):\r\n-rw-r--r-- 1 root root   560184 May  8 02:00 /usr/local/cuda-8.0/lib64/libcudadevrt.a\r\nlrwxrwxrwx 1 root root       16 Sep 12 15:13 /usr/local/cuda-8.0/lib64/libcudart.so -> libcudart.so.8.0\r\nlrwxrwxrwx 1 root root       19 Sep 12 15:13 /usr/local/cuda-8.0/lib64/libcudart.so.8.0 -> libcudart.so.8.0.27\r\n-rwxr-xr-x 1 root root   394472 May  8 02:00 /usr/local/cuda-8.0/lib64/libcudart.so.8.0.27\r\n-rw-r--r-- 1 root root   737516 May  8 02:00 /usr/local/cuda-8.0/lib64/libcudart_static.a\r\n-rwxr-xr-x 1 root root 60696704 Sep 24 09:12 /usr/local/cuda-8.0/lib64/libcudnn.so\r\n-rwxr-xr-x 1 root root 60696704 Sep 24 09:12 /usr/local/cuda-8.0/lib64/libcudnn.so.5\r\n-rwxr-xr-x 1 root root 60696704 Sep 24 09:12 /usr/local/cuda-8.0/lib64/libcudnn.so.5.1.3\r\n-rw-r--r-- 1 root root 59715990 Sep 24 09:12 /usr/local/cuda-8.0/lib64/libcudnn_static.a\r\n\r\nIf installed from source, provide \r\n1. The commit hash (`git rev-parse HEAD`): f98c5ded31d7da0c2d127c28b2c16f0307a368f0\r\n2. The output of `bazel version`: \r\nWARNING: Output base '/home/rengan/.cache/bazel/_bazel_rengan/2c9ddf610cf302ceda3f2b852669382c' is on NFS. This may lead to surprising failures and undetermined behavior.\r\n.\r\nBuild label: 0.3.1- (@non-git)\r\nBuild target: bazel-out/local-fastbuild/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar\r\nBuild time: Thu Sep 29 22:19:27 2016 (1475187567)\r\nBuild timestamp: 1475187567\r\nBuild timestamp as int: 1475187567\r\n\r\nThanks in advance."},"comment":{"url":"https://api.github.com/repos/tensorflow/tensorflow/issues/comments/250871652","html_url":"https://github.com/tensorflow/tensorflow/issues/4685#issuecomment-250871652","issue_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/4685","id":250871652,"user":{"login":"cshallue","id":20667461,"avatar_url":"https://avatars.githubusercontent.com/u/20667461?v=3","gravatar_id":"","url":"https://api.github.com/users/cshallue","html_url":"https://github.com/cshallue","followers_url":"https://api.github.com/users/cshallue/followers","following_url":"https://api.github.com/users/cshallue/following{/other_user}","gists_url":"https://api.github.com/users/cshallue/gists{/gist_id}","starred_url":"https://api.github.com/users/cshallue/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cshallue/subscriptions","organizations_url":"https://api.github.com/users/cshallue/orgs","repos_url":"https://api.github.com/users/cshallue/repos","events_url":"https://api.github.com/users/cshallue/events{/privacy}","received_events_url":"https://api.github.com/users/cshallue/received_events","type":"User","site_admin":false},"created_at":"2016-09-30T23:02:20Z","updated_at":"2016-09-30T23:02:20Z","body":"This might indeed be a bug in tf.gfile.FastGFile().\r\nPlease let me know if it finishes successfully for you now (the preprocessing usually takes 1+ hours)."}}
{"actor":{"display_login":"cshallue"}}