{"action":"created","issue":{"url":"https://api.github.com/repos/tensorflow/tensorflow/issues/5972","repository_url":"https://api.github.com/repos/tensorflow/tensorflow","labels_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/5972/labels{/name}","comments_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/5972/comments","events_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/5972/events","html_url":"https://github.com/tensorflow/tensorflow/issues/5972","id":192496073,"number":5972,"title":"optimize variable with dynamic shape not supported","user":{"login":"albertz","id":59132,"avatar_url":"https://avatars.githubusercontent.com/u/59132?v=3","gravatar_id":"","url":"https://api.github.com/users/albertz","html_url":"https://github.com/albertz","followers_url":"https://api.github.com/users/albertz/followers","following_url":"https://api.github.com/users/albertz/following{/other_user}","gists_url":"https://api.github.com/users/albertz/gists{/gist_id}","starred_url":"https://api.github.com/users/albertz/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/albertz/subscriptions","organizations_url":"https://api.github.com/users/albertz/orgs","repos_url":"https://api.github.com/users/albertz/repos","events_url":"https://api.github.com/users/albertz/events{/privacy}","received_events_url":"https://api.github.com/users/albertz/received_events","type":"User","site_admin":false},"labels":[{"id":404586594,"url":"https://api.github.com/repos/tensorflow/tensorflow/labels/awaiting%20googler","name":"awaiting googler","color":"b60205","default":false}],"state":"closed","locked":false,"assignee":{"login":"keveman","id":229914,"avatar_url":"https://avatars.githubusercontent.com/u/229914?v=3","gravatar_id":"","url":"https://api.github.com/users/keveman","html_url":"https://github.com/keveman","followers_url":"https://api.github.com/users/keveman/followers","following_url":"https://api.github.com/users/keveman/following{/other_user}","gists_url":"https://api.github.com/users/keveman/gists{/gist_id}","starred_url":"https://api.github.com/users/keveman/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/keveman/subscriptions","organizations_url":"https://api.github.com/users/keveman/orgs","repos_url":"https://api.github.com/users/keveman/repos","events_url":"https://api.github.com/users/keveman/events{/privacy}","received_events_url":"https://api.github.com/users/keveman/received_events","type":"User","site_admin":false},"assignees":[{"login":"keveman","id":229914,"avatar_url":"https://avatars.githubusercontent.com/u/229914?v=3","gravatar_id":"","url":"https://api.github.com/users/keveman","html_url":"https://github.com/keveman","followers_url":"https://api.github.com/users/keveman/followers","following_url":"https://api.github.com/users/keveman/following{/other_user}","gists_url":"https://api.github.com/users/keveman/gists{/gist_id}","starred_url":"https://api.github.com/users/keveman/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/keveman/subscriptions","organizations_url":"https://api.github.com/users/keveman/orgs","repos_url":"https://api.github.com/users/keveman/repos","events_url":"https://api.github.com/users/keveman/events{/privacy}","received_events_url":"https://api.github.com/users/keveman/received_events","type":"User","site_admin":false}],"milestone":null,"comments":1,"created_at":"2016-11-30T07:39:55Z","updated_at":"2016-12-08T19:12:07Z","closed_at":"2016-12-08T19:12:07Z","body":"TensorFlow 0.11.0.\r\n\r\nI want to use a variable where the shape is unknown in advance and it will change from time to time (although ndim is known and fixed).\r\n\r\nI declare it like:\r\n\r\n```\r\ninitializer = tf.random_uniform_initializer()\r\nshape = (s0, s1, s2)  # these are symbolic vars\r\nfoo_var = tf.Variable(initializer(shape=shape), name=\"foo\", validate_shape=False)\r\n```\r\nThis seems to work when I create the computation graph up to the point where I want to optimize w.r.t. this variable, i.e.:\r\n```\r\noptimizer = tf.train.AdamOptimizer(learning_rate=0.1, epsilon=1e-4)\r\noptim = optimizer.minimize(loss, var_list=[foo_var])\r\n```\r\n\r\nThat fails in the optimizer in some function `create_zeros_slot` where it seems to depend on the static shape information (it uses `primary.get_shape().as_list()`).\r\n\r\nSo, using the optimizer works only with variables with static shape? Is that a bug?\r\n\r\nMy current solution is some hacky monkey patching:\r\n```\r\ndef _tf_create_slot_var(primary, val, scope):\r\n  \"\"\"Helper function for creating a slot variable.\"\"\"\r\n\r\n  from tensorflow.python.ops import variables\r\n  slot = variables.Variable(val, name=scope, trainable=False, validate_shape=primary.get_shape().is_fully_defined())\r\n  # pylint: disable=protected-access\r\n  if isinstance(primary, variables.Variable) and primary._save_slice_info:\r\n    # Primary is a partitioned variable, so we need to also indicate that\r\n    # the slot is a partitioned variable.  Slots have the same partitioning\r\n    # as their primaries.\r\n    real_slot_name = scope[len(primary.op.name + \"/\"):-1]\r\n    slice_info = primary._save_slice_info\r\n    slot._set_save_slice_info(variables.Variable.SaveSliceInfo(\r\n        slice_info.full_name + \"/\" + real_slot_name,\r\n        slice_info.full_shape[:],\r\n        slice_info.var_offset[:],\r\n        slice_info.var_shape[:]))\r\n  # pylint: enable=protected-access\r\n  return slot\r\n\r\n\r\ndef _tf_create_zeros_slot(primary, name, dtype=None, colocate_with_primary=True):\r\n  \"\"\"Create a slot initialized to 0 with same shape as the primary object.\r\n\r\n  Args:\r\n    primary: The primary `Variable` or `Tensor`.\r\n    name: Name to use for the slot variable.\r\n    dtype: Type of the slot variable.  Defaults to the type of `primary`.\r\n    colocate_with_primary: Boolean.  If True the slot is located\r\n      on the same device as `primary`.\r\n\r\n  Returns:\r\n    A `Variable` object.\r\n  \"\"\"\r\n  if dtype is None:\r\n    dtype = primary.dtype\r\n  from tensorflow.python.ops import array_ops\r\n  val = array_ops.zeros(\r\n      primary.get_shape().as_list() if primary.get_shape().is_fully_defined() else tf.shape(primary),\r\n      dtype=dtype)\r\n  from tensorflow.python.training import slot_creator\r\n  return slot_creator.create_slot(primary, val, name, colocate_with_primary=colocate_with_primary)\r\n\r\n\r\ndef monkey_patch_tf_slot_creator():\r\n    \"\"\"\r\n    The TensorFlow optimizers cannot handle variables with unknown shape.\r\n    We hack this.\r\n    \"\"\"\r\n    from tensorflow.python.training import slot_creator\r\n    slot_creator._create_slot_var = _tf_create_slot_var\r\n    slot_creator.create_zeros_slot = _tf_create_zeros_slot\r\n```\r\n\r\n(That was also asked [on StackOverflow](http://stackoverflow.com/questions/40863082/optimize-variable-with-dynamic-shape/).)"},"comment":{"url":"https://api.github.com/repos/tensorflow/tensorflow/issues/comments/265827057","html_url":"https://github.com/tensorflow/tensorflow/issues/5972#issuecomment-265827057","issue_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/5972","id":265827057,"user":{"login":"keveman","id":229914,"avatar_url":"https://avatars.githubusercontent.com/u/229914?v=3","gravatar_id":"","url":"https://api.github.com/users/keveman","html_url":"https://github.com/keveman","followers_url":"https://api.github.com/users/keveman/followers","following_url":"https://api.github.com/users/keveman/following{/other_user}","gists_url":"https://api.github.com/users/keveman/gists{/gist_id}","starred_url":"https://api.github.com/users/keveman/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/keveman/subscriptions","organizations_url":"https://api.github.com/users/keveman/orgs","repos_url":"https://api.github.com/users/keveman/repos","events_url":"https://api.github.com/users/keveman/events{/privacy}","received_events_url":"https://api.github.com/users/keveman/received_events","type":"User","site_admin":false},"created_at":"2016-12-08T19:12:07Z","updated_at":"2016-12-08T19:12:07Z","body":"@albertz Your comment on StackOverflow \r\n\r\n> During optimization, it wont change. At some point, I reset it explicitly to a new value which will likely have another shape. Then I want to do optimization again. I don't want to recreate the computation graph every time\r\n\r\nThis is something that TensorFlow does not support. In a TensorFlow graph, shape of variables have to be fixed at graph construction time. When you want to \"reset\" to a new value with a different shape, you have to essentially reconstruct the rest of the graph. This is by design and not a bug. Closing this one."}}
{"actor":{"display_login":"keveman"}}