{"action":"created","issue":{"url":"https://api.github.com/repos/tensorflow/tensorflow/issues/323","repository_url":"https://api.github.com/repos/tensorflow/tensorflow","labels_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/323/labels{/name}","comments_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/323/comments","events_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/323/events","html_url":"https://github.com/tensorflow/tensorflow/issues/323","id":118251717,"number":323,"title":"Tensorflow crashed when using AdamOptimizer","user":{"login":"chuanwen","id":2024931,"avatar_url":"https://avatars.githubusercontent.com/u/2024931?v=3","gravatar_id":"","url":"https://api.github.com/users/chuanwen","html_url":"https://github.com/chuanwen","followers_url":"https://api.github.com/users/chuanwen/followers","following_url":"https://api.github.com/users/chuanwen/following{/other_user}","gists_url":"https://api.github.com/users/chuanwen/gists{/gist_id}","starred_url":"https://api.github.com/users/chuanwen/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/chuanwen/subscriptions","organizations_url":"https://api.github.com/users/chuanwen/orgs","repos_url":"https://api.github.com/users/chuanwen/repos","events_url":"https://api.github.com/users/chuanwen/events{/privacy}","received_events_url":"https://api.github.com/users/chuanwen/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":20,"created_at":"2015-11-22T08:56:05Z","updated_at":"2016-06-03T14:04:17Z","closed_at":"2015-12-03T03:47:13Z","body":"I tried the cnn in the tutorial for MNIST data, but initialize the parameters with stddev=1 (instead of stddev=0.1).\r\n\r\nError message:\r\nI tensorflow/core/common_runtime/local_device.cc:25] Local device intra op parallelism threads: 4\r\nI tensorflow/core/common_runtime/local_session.cc:45] Local session inter op parallelism threads: 4\r\n0 0.098\r\nW tensorflow/core/common_runtime/executor.cc:1027] 0x4472710 Compute status: Invalid argument: ReluGrad input is not finite. : Tensor had NaN values\r\n\t [[Node: gradients/Relu_grad/Relu/CheckNumerics = CheckNumerics[T=DT_FLOAT, message=\"ReluGrad input is not finite.\", _device=\"/job:localhost/replica:0/task:0/cpu:0\"](add)]]\r\nW tensorflow/core/common_runtime/executor.cc:1027] 0x4472710 Compute status: Invalid argument: ReluGrad input is not finite. : Tensor had NaN values\r\n\t [[Node: gradients/Relu_1_grad/Relu_1/CheckNumerics = CheckNumerics[T=DT_FLOAT, message=\"ReluGrad input is not finite.\", _device=\"/job:localhost/replica:0/task:0/cpu:0\"](add_1)]]\r\nW tensorflow/core/common_runtime/executor.cc:1027] 0x4472710 Compute status: Invalid argument: ReluGrad input is not finite. : Tensor had NaN values\r\n\t [[Node: gradients/Relu_2_grad/Relu_2/CheckNumerics = CheckNumerics[T=DT_FLOAT, message=\"ReluGrad input is not finite.\", _device=\"/job:localhost/replica:0/task:0/cpu:0\"](add_2)]]\r\nTraceback (most recent call last):\r\n  File \"2_cnn.py\", line 67, in <module>\r\n    session.run(train_step, feed_dict={X: x, Y: y, keep_prob: 0.5})\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py\", line 345, in run\r\n    results = self._do_run(target_list, unique_fetch_targets, feed_dict_string)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/client/session.py\", line 419, in _do_run\r\n    e.code)\r\ntensorflow.python.framework.errors.InvalidArgumentError: ReluGrad input is not finite. : Tensor had NaN values\r\n\t [[Node: gradients/Relu_grad/Relu/CheckNumerics = CheckNumerics[T=DT_FLOAT, message=\"ReluGrad input is not finite.\", _device=\"/job:localhost/replica:0/task:0/cpu:0\"](add)]]\r\nCaused by op u'gradients/Relu_grad/Relu/CheckNumerics', defined at:\r\n  File \"2_cnn.py\", line 55, in <module>\r\n    train_step = tf.train.AdamOptimizer().minimize(loss)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/optimizer.py\", line 165, in minimize\r\n    gate_gradients=gate_gradients)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/training/optimizer.py\", line 205, in compute_gradients\r\n    loss, var_list, gate_gradients=(gate_gradients == Optimizer.GATE_OP))\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gradients.py\", line 414, in gradients\r\n    in_grads = _AsList(grad_fn(op_wrapper, *out_grads))\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/nn_grad.py\", line 107, in _ReluGrad\r\n    t = _VerifyTensor(op.inputs[0], op.name, \"ReluGrad input is not finite.\")\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/nn_grad.py\", line 100, in _VerifyTensor\r\n    verify_input = array_ops.check_numerics(t, message=msg)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_array_ops.py\", line 48, in check_numerics\r\n    name=name)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/op_def_library.py\", line 633, in apply_op\r\n    op_def=op_def)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py\", line 1710, in create_op\r\n    original_op=self._default_original_op, op_def=op_def)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py\", line 988, in __init__\r\n    self._traceback = _extract_stack()\r\n\r\n...which was originally created as op u'Relu', defined at:\r\n  File \"2_cnn.py\", line 32, in <module>\r\n    h_conv1 = tf.nn.relu(conv2d(X1, w_conv1) + b_conv1)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/gen_nn_ops.py\", line 506, in relu\r\n    return _op_def_lib.apply_op(\"Relu\", features=features, name=name)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/op_def_library.py\", line 633, in apply_op\r\n    op_def=op_def)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py\", line 1710, in create_op\r\n    original_op=self._default_original_op, op_def=op_def)\r\n  File \"/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/ops.py\", line 988, in __init__\r\n    self._traceback = _extract_stack()\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/tensorflow/tensorflow/issues/comments/223587404","html_url":"https://github.com/tensorflow/tensorflow/issues/323#issuecomment-223587404","issue_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/323","id":223587404,"user":{"login":"LeavesBreathe","id":14891677,"avatar_url":"https://avatars.githubusercontent.com/u/14891677?v=3","gravatar_id":"","url":"https://api.github.com/users/LeavesBreathe","html_url":"https://github.com/LeavesBreathe","followers_url":"https://api.github.com/users/LeavesBreathe/followers","following_url":"https://api.github.com/users/LeavesBreathe/following{/other_user}","gists_url":"https://api.github.com/users/LeavesBreathe/gists{/gist_id}","starred_url":"https://api.github.com/users/LeavesBreathe/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/LeavesBreathe/subscriptions","organizations_url":"https://api.github.com/users/LeavesBreathe/orgs","repos_url":"https://api.github.com/users/LeavesBreathe/repos","events_url":"https://api.github.com/users/LeavesBreathe/events{/privacy}","received_events_url":"https://api.github.com/users/LeavesBreathe/received_events","type":"User","site_admin":false},"created_at":"2016-06-03T14:04:17Z","updated_at":"2016-06-03T14:04:17Z","body":"@lightscalar big thanks"}}
{"actor":{"display_login":"LeavesBreathe"}}