{"action":"created","issue":{"url":"https://api.github.com/repos/fchollet/keras/issues/2976","repository_url":"https://api.github.com/repos/fchollet/keras","labels_url":"https://api.github.com/repos/fchollet/keras/issues/2976/labels{/name}","comments_url":"https://api.github.com/repos/fchollet/keras/issues/2976/comments","events_url":"https://api.github.com/repos/fchollet/keras/issues/2976/events","html_url":"https://github.com/fchollet/keras/issues/2976","id":160151419,"number":2976,"title":"unrolling recurrent networks causes theano optimization error","user":{"login":"ReallyCoolName","id":19924204,"avatar_url":"https://avatars.githubusercontent.com/u/19924204?v=3","gravatar_id":"","url":"https://api.github.com/users/ReallyCoolName","html_url":"https://github.com/ReallyCoolName","followers_url":"https://api.github.com/users/ReallyCoolName/followers","following_url":"https://api.github.com/users/ReallyCoolName/following{/other_user}","gists_url":"https://api.github.com/users/ReallyCoolName/gists{/gist_id}","starred_url":"https://api.github.com/users/ReallyCoolName/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ReallyCoolName/subscriptions","organizations_url":"https://api.github.com/users/ReallyCoolName/orgs","repos_url":"https://api.github.com/users/ReallyCoolName/repos","events_url":"https://api.github.com/users/ReallyCoolName/events{/privacy}","received_events_url":"https://api.github.com/users/ReallyCoolName/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":3,"created_at":"2016-06-14T10:48:49Z","updated_at":"2016-06-15T13:43:52Z","closed_at":"2016-06-15T13:43:52Z","body":"Just reinstalled theano+keras to make sure. I've tried a few things and it seems the problem only happens when the RNN being unrolled is before a `RepeatVector` layer (unrolling a network after the `RepeatVector` doesn't cause any problem).\r\n\r\nMy code:\r\n```\r\nembedding_models =[]\r\nfor _ in xrange(n_simultaneous):\r\n    embedding_models += [Sequential()]\r\n    embedding_models[-1].add(Embedding(28,2,batch_input_shape=(b_sz,50 )))\r\n\r\nmodel.add(Merge(embedding_models,mode='concat'))\r\nmodel.add(GRU(10,stateful=True,unroll=True,return_sequences=True))\r\nmodel.add(Flatten())\r\nmodel.add(RepeatVector(10))\r\nmodel.add(GRU(10,stateful=True,unroll=False,return_sequences=True))\r\nmodel.add(TimeDistributedDense(180,activation='softmax'))'''\r\nmodel.compile(loss=categorial_ce, optimizer='rmsprop',metrics=['accuracy'])\r\nX_train,Y_train = get_batch()\r\nmodel.fit(X_train, Y_train, batch_size=b_sz, nb_epoch=4, verbose=1,validation_split=v_splt)```\r\n\r\n\r\nThe error: (50 times `TensorConstant{10}` if anyone's wondering. probably number of timesteps. And no, `config.mode` is not `FAST_COMPILE`, is `Mode`, which is equivalent to `FAST_RUN`):\r\n\r\n\r\n```ERROR (theano.gof.opt): Optimization failure due to: constant_folding\r\nERROR (theano.gof.opt): node: Elemwise{add,no_inplace}(TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10}, TensorConstant{10})\r\nERROR (theano.gof.opt): TRACEBACK:\r\nERROR (theano.gof.opt): Traceback (most recent call last):\r\n  File \"/home/azureuser/anaconda2/lib/python2.7/site-packages/theano/gof/opt.py\", line 1772, in process_node\r\n    replacements = lopt.transform(node)\r\n  File \"/home/azureuser/anaconda2/lib/python2.7/site-packages/theano/tensor/opt.py\", line 5827, in constant_folding\r\n    required = thunk()\r\n  File \"/home/azureuser/anaconda2/lib/python2.7/site-packages/theano/gof/op.py\", line 912, in rval\r\n    r = p(n, [x[0] for x in i], o)\r\n  File \"/home/azureuser/anaconda2/lib/python2.7/site-packages/theano/tensor/elemwise.py\", line 839, in perform\r\n    super(Elemwise, self).perform(node, inputs, output_storage)\r\n  File \"/home/azureuser/anaconda2/lib/python2.7/site-packages/theano/gof/op.py\", line 769, in perform\r\n    \"Did you used Theano flags mode=FAST_COMPILE?\"\r\nMethodNotDefined: ('perform', <class 'theano.tensor.elemwise.Elemwise'>, 'Elemwise', 'Did you used Theano flags mode=FAST_COMPILE? You can use optimizer=fast_compile instead.')\r\n\r\nERROR (theano.gof.opt): Optimization failure due to: local_mul_zero\r\nERROR (theano.gof.opt): node: Elemwise{mul,no_inplace}(TensorConstant{50}, Elemwise{switch,no_inplace}.0, Elemwise{switch,no_inplace}.0)\r\nERROR (theano.gof.opt): TRACEBACK:\r\nERROR (theano.gof.opt): Traceback (most recent call last):\r\n  File \"/home/azureuser/anaconda2/lib/python2.7/site-packages/theano/gof/opt.py\", line 1772, in process_node\r\n    replacements = lopt.transform(node)\r\n  File \"/home/azureuser/anaconda2/lib/python2.7/site-packages/theano/tensor/opt.py\", line 5193, in local_mul_zero\r\n    value = get_scalar_constant_value(i)\r\n  File \"/home/azureuser/anaconda2/lib/python2.7/site-packages/theano/tensor/basic.py\", line 660, in get_scalar_constant_value\r\n    for i in v.owner.inputs]\r\n  File \"/home/azureuser/anaconda2/lib/python2.7/site-packages/theano/tensor/basic.py\", line 662, in get_scalar_constant_value\r\n    v.owner.op.perform(v.owner, const, ret)\r\n  File \"/home/azureuser/anaconda2/lib/python2.7/site-packages/theano/tensor/elemwise.py\", line 839, in perform\r\n    super(Elemwise, self).perform(node, inputs, output_storage)\r\n  File \"/home/azureuser/anaconda2/lib/python2.7/site-packages/theano/gof/op.py\", line 769, in perform\r\n    \"Did you used Theano flags mode=FAST_COMPILE?\"\r\nMethodNotDefined: ('perform', <class 'theano.tensor.elemwise.Elemwise'>, 'Elemwise', 'Did you used Theano flags mode=FAST_COMPILE? You can use optimizer=fast_compile instead.')```\r\n\r\nI'd try to help, but I spent all my sanity during the weekend trying to debug my implementation of a neural tensor network."},"comment":{"url":"https://api.github.com/repos/fchollet/keras/issues/comments/226190988","html_url":"https://github.com/fchollet/keras/issues/2976#issuecomment-226190988","issue_url":"https://api.github.com/repos/fchollet/keras/issues/2976","id":226190988,"user":{"login":"ReallyCoolName","id":19924204,"avatar_url":"https://avatars.githubusercontent.com/u/19924204?v=3","gravatar_id":"","url":"https://api.github.com/users/ReallyCoolName","html_url":"https://github.com/ReallyCoolName","followers_url":"https://api.github.com/users/ReallyCoolName/followers","following_url":"https://api.github.com/users/ReallyCoolName/following{/other_user}","gists_url":"https://api.github.com/users/ReallyCoolName/gists{/gist_id}","starred_url":"https://api.github.com/users/ReallyCoolName/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ReallyCoolName/subscriptions","organizations_url":"https://api.github.com/users/ReallyCoolName/orgs","repos_url":"https://api.github.com/users/ReallyCoolName/repos","events_url":"https://api.github.com/users/ReallyCoolName/events{/privacy}","received_events_url":"https://api.github.com/users/ReallyCoolName/received_events","type":"User","site_admin":false},"created_at":"2016-06-15T13:43:52Z","updated_at":"2016-06-15T13:43:52Z","body":"@nouiz  Not a recursion limit problem - I already had that problem (gave me a different error), and I increased the recursion limit.\r\n\r\n@fchollet length isn't that big, though I guess it was pretty dumb of me to assume `RepeatVector` has anything to do with the problem (though lately I haven't had much time to think)\r\n\r\nguess I should close this for now, since you're aware of the problem and it's not that important"}}
{"actor":{"display_login":"ReallyCoolName"}}