{"action":"created","issue":{"url":"https://api.github.com/repos/tensorflow/tensorflow/issues/15564","repository_url":"https://api.github.com/repos/tensorflow/tensorflow","labels_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/15564/labels{/name}","comments_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/15564/comments","events_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/15564/events","html_url":"https://github.com/tensorflow/tensorflow/issues/15564","id":284010225,"node_id":"MDU6SXNzdWUyODQwMTAyMjU=","number":15564,"title":"sparsemax implementation is not infinite-safe","user":{"login":"gcampax","id":876889,"node_id":"MDQ6VXNlcjg3Njg4OQ==","avatar_url":"https://avatars2.githubusercontent.com/u/876889?v=4","gravatar_id":"","url":"https://api.github.com/users/gcampax","html_url":"https://github.com/gcampax","followers_url":"https://api.github.com/users/gcampax/followers","following_url":"https://api.github.com/users/gcampax/following{/other_user}","gists_url":"https://api.github.com/users/gcampax/gists{/gist_id}","starred_url":"https://api.github.com/users/gcampax/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/gcampax/subscriptions","organizations_url":"https://api.github.com/users/gcampax/orgs","repos_url":"https://api.github.com/users/gcampax/repos","events_url":"https://api.github.com/users/gcampax/events{/privacy}","received_events_url":"https://api.github.com/users/gcampax/received_events","type":"User","site_admin":false},"labels":[{"id":299643928,"node_id":"MDU6TGFiZWwyOTk2NDM5Mjg=","url":"https://api.github.com/repos/tensorflow/tensorflow/labels/stat:contributions%20welcome","name":"stat:contributions welcome","color":"009800","default":false}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":15,"created_at":"2017-12-21T21:31:31Z","updated_at":"2018-06-20T12:51:20Z","closed_at":null,"author_association":"NONE","body":"### System information\r\n- **Have I written custom code (as opposed to using a stock example script provided in TensorFlow)**:\r\nYes, but I have a simple reproducer below\r\n\r\n- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**: Fedora 27\r\n- **TensorFlow installed from (source or binary)**: pip3 install --user tensorflow\r\n- **TensorFlow version (use command below)**:  1.4.0\r\n- **Python version**: 3.6.3\r\n- **Bazel version (if compiling from source)**: no\r\n- **GCC/Compiler version (if compiling from source)**: no\r\n- **CUDA/cuDNN version**: no\r\n- **GPU model and memory**: no (x86_64 CPU)\r\n- **Exact command to reproduce**:\r\n\r\n### Describe the problem\r\ntf.contrib.sparsemax.sparsemax does not produce the correctly expected results when infinities are present in the input.\r\n\r\nMathematically, a negative infinity should be treated the same as a very large negative number, and produce a 0 in the output.\r\n\r\nThis is a real-world problem when sparsemax is combined with contrib.seq2seq.LuongAttention (as suggested in its documentation, and also as suggested in the sparsemax paper), because entries past the input length in the batch will have a score of -inf by default.\r\n\r\n### Source code / logs\r\n\r\nReproducer (assume tensorflow as tf)\r\n```\r\n>>> tf.contrib.sparsemax.sparsemax([[1., -1e+5]]).eval(session=sess)\r\narray([[ 1.,  0.]], dtype=float32)\r\n>>> tf.contrib.sparsemax.sparsemax([[1., float('-inf')]]).eval(session=sess)\r\n2017-12-21 22:13:55.697302: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: flat indices[0, :] = [0, -1] does not index into param (shape: [1,2]).\r\n2017-12-21 22:13:55.697997: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: flat indices[0, :] = [0, -1] does not index into param (shape: [1,2]).\r\n2017-12-21 22:13:55.699634: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: flat indices[0, :] = [0, -1] does not index into param (shape: [1,2]).\r\n2017-12-21 22:13:55.699980: W tensorflow/core/framework/op_kernel.cc:1192] Invalid argument: flat indices[0, :] = [0, -1] does not index into param (shape: [1,2]).\r\nTraceback (most recent call last):\r\n  File \"/home/redacted/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py\", line 1323, in _do_call\r\n    return fn(*args)\r\n  File \"/home/redacted/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py\", line 1302, in _run_fn\r\n    status, run_metadata)\r\n  File \"/home/redacted/.local/lib/python3.6/site-packages/tensorflow/python/framework/errors_impl.py\", line 473, in __exit__\r\n    c_api.TF_GetCode(self.status.status))\r\ntensorflow.python.framework.errors_impl.InvalidArgumentError: flat indices[0, :] = [0, -1] does not index into param (shape: [1,2]).\r\n\t [[Node: sparsemax_7/GatherNd = GatherNd[Tindices=DT_INT32, Tparams=DT_FLOAT, _device=\"/job:localhost/replica:0/task:0/device:CPU:0\"](sparsemax_7/sub, sparsemax_7/stack)]]\r\n\r\nDuring handling of the above exception, another exception occurred:\r\n\r\nTraceback (most recent call last):\r\n  File \"<stdin>\", line 1, in <module>\r\n  File \"/home/redacted/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py\", line 570, in eval\r\n    return _eval_using_default_session(self, feed_dict, self.graph, session)\r\n  File \"/home/redacted/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py\", line 4455, in _eval_using_default_session\r\n    return session.run(tensors, feed_dict)\r\n  File \"/home/redacted/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py\", line 889, in run\r\n    run_metadata_ptr)\r\n  File \"/home/redacted/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py\", line 1120, in _run\r\n    feed_dict_tensor, options, run_metadata)\r\n  File \"/home/redacted/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py\", line 1317, in _do_run\r\n    options, run_metadata)\r\n  File \"/home/redacted/.local/lib/python3.6/site-packages/tensorflow/python/client/session.py\", line 1336, in _do_call\r\n    raise type(e)(node_def, op, message)\r\ntensorflow.python.framework.errors_impl.InvalidArgumentError: flat indices[0, :] = [0, -1] does not index into param (shape: [1,2]).\r\n\t [[Node: sparsemax_7/GatherNd = GatherNd[Tindices=DT_INT32, Tparams=DT_FLOAT, _device=\"/job:localhost/replica:0/task:0/device:CPU:0\"](sparsemax_7/sub, sparsemax_7/stack)]]\r\n\r\nCaused by op 'sparsemax_7/GatherNd', defined at:\r\n  File \"<stdin>\", line 1, in <module>\r\n  File \"/home/redacted/.local/lib/python3.6/site-packages/tensorflow/contrib/sparsemax/python/ops/sparsemax.py\", line 69, in sparsemax\r\n    tau_sum = array_ops.gather_nd(z_cumsum, indices)\r\n  File \"/home/redacted/.local/lib/python3.6/site-packages/tensorflow/python/ops/gen_array_ops.py\", line 1971, in gather_nd\r\n    \"GatherNd\", params=params, indices=indices, name=name)\r\n  File \"/home/redacted/.local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py\", line 787, in _apply_op_helper\r\n    op_def=op_def)\r\n  File \"/home/redacted/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py\", line 2956, in create_op\r\n    op_def=op_def)\r\n  File \"/home/redacted/.local/lib/python3.6/site-packages/tensorflow/python/framework/ops.py\", line 1470, in __init__\r\n    self._traceback = self._graph._extract_stack()  # pylint: disable=protected-access\r\n\r\nInvalidArgumentError (see above for traceback): flat indices[0, :] = [0, -1] does not index into param (shape: [1,2]).\r\n\t [[Node: sparsemax_7/GatherNd = GatherNd[Tindices=DT_INT32, Tparams=DT_FLOAT, _device=\"/job:localhost/replica:0/task:0/device:CPU:0\"](sparsemax_7/sub, sparsemax_7/stack)]]\r\n```\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/tensorflow/tensorflow/issues/comments/398737310","html_url":"https://github.com/tensorflow/tensorflow/issues/15564#issuecomment-398737310","issue_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/15564","id":398737310,"node_id":"MDEyOklzc3VlQ29tbWVudDM5ODczNzMxMA==","user":{"login":"clarken92","id":1445226,"node_id":"MDQ6VXNlcjE0NDUyMjY=","avatar_url":"https://avatars3.githubusercontent.com/u/1445226?v=4","gravatar_id":"","url":"https://api.github.com/users/clarken92","html_url":"https://github.com/clarken92","followers_url":"https://api.github.com/users/clarken92/followers","following_url":"https://api.github.com/users/clarken92/following{/other_user}","gists_url":"https://api.github.com/users/clarken92/gists{/gist_id}","starred_url":"https://api.github.com/users/clarken92/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/clarken92/subscriptions","organizations_url":"https://api.github.com/users/clarken92/orgs","repos_url":"https://api.github.com/users/clarken92/repos","events_url":"https://api.github.com/users/clarken92/events{/privacy}","received_events_url":"https://api.github.com/users/clarken92/received_events","type":"User","site_admin":false},"created_at":"2018-06-20T12:51:20Z","updated_at":"2018-06-20T12:51:20Z","author_association":"NONE","body":"One more thing I forget to say, I also add the option for computing sparsemax in case of 2 classes where only one logit value is available. This behavior is similar to sigmoid function. In this case, I assume that the logit for class 0 is 0 and the logit for class 1 is the logit itself."}}
{"actor":{"display_login":"clarken92"}}