{"action":"closed","issue":{"url":"https://api.github.com/repos/nyu-dl/dl4mt-tutorial/issues/49","repository_url":"https://api.github.com/repos/nyu-dl/dl4mt-tutorial","labels_url":"https://api.github.com/repos/nyu-dl/dl4mt-tutorial/issues/49/labels{/name}","comments_url":"https://api.github.com/repos/nyu-dl/dl4mt-tutorial/issues/49/comments","events_url":"https://api.github.com/repos/nyu-dl/dl4mt-tutorial/issues/49/events","html_url":"https://github.com/nyu-dl/dl4mt-tutorial/issues/49","id":151595345,"number":49,"title":"I can't re-run a previous perfect train_nmt.py","user":{"login":"amirj","id":1645137,"avatar_url":"https://avatars.githubusercontent.com/u/1645137?v=3","gravatar_id":"","url":"https://api.github.com/users/amirj","html_url":"https://github.com/amirj","followers_url":"https://api.github.com/users/amirj/followers","following_url":"https://api.github.com/users/amirj/following{/other_user}","gists_url":"https://api.github.com/users/amirj/gists{/gist_id}","starred_url":"https://api.github.com/users/amirj/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/amirj/subscriptions","organizations_url":"https://api.github.com/users/amirj/orgs","repos_url":"https://api.github.com/users/amirj/repos","events_url":"https://api.github.com/users/amirj/events{/privacy}","received_events_url":"https://api.github.com/users/amirj/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":2,"created_at":"2016-04-28T09:49:08Z","updated_at":"2016-04-28T12:46:54Z","closed_at":"2016-04-28T12:46:54Z","body":"I had trained a NMT network about one month ago, it was great. \r\nToday, I can't run the same code for the same dataset. Here is the output:\r\n\r\n`Loading data\r\nBuilding model\r\nBuliding sampler\r\nBuilding f_log_probs... Done\r\nBuilding f_cost... Done\r\nComputing gradient... Done\r\nBuilding optimizers... Done\r\nOptimization\r\nTraceback (most recent call last):\r\n  File \"train_nmt.py\", line 54, in <module>\r\n    'learning-rate': [0.001]})\r\n  File \"train_nmt.py\", line 33, in main\r\n    use_dropout=params['use-dropout'][0])\r\n  File \"/Users/AmirHJ/projects/deep-learning/seq2seq/nmt2.py\", line 1176, in train\r\n    cost = f_grad_shared(x, x_mask, y, y_mask)\r\n  File \"/usr/local/lib/python2.7/site-packages/theano/compile/function_module.py\", line 871, in __call__\r\n    storage_map=getattr(self.fn, 'storage_map', None))\r\n  File \"/usr/local/lib/python2.7/site-packages/theano/gof/link.py\", line 314, in raise_with_op\r\n    reraise(exc_type, exc_value, exc_trace)\r\n  File \"/usr/local/lib/python2.7/site-packages/theano/compile/function_module.py\", line 859, in __call__\r\n    outputs = self.fn()\r\nValueError: all the input array dimensions except for the concatenation axis must match exactly\r\nApply node that caused the error: Join(TensorConstant{0}, InplaceDimShuffle{x,0,1,2}.0, InplaceDimShuffle{x,0,1,2}.0, InplaceDimShuffle{x,0,1,2}.0)\r\nToposort index: 999\r\nInputs types: [TensorType(int8, scalar), TensorType(float32, (True, False, False, False)), TensorType(float32, (True, False, False, False)), TensorType(float32, (True, False, False, False))]\r\nInputs shapes: [(), (1, 26, 32, 200), (1, 26, 32, 400), (1, 26, 32, 24)]\r\nInputs strides: [(), (665600, 25600, 800, 4), (1331200, 51200, 1600, 4), (79872, 3072, 96, 4)]\r\nInputs values: [array(0, dtype=int8), 'not shown', 'not shown', 'not shown']\r\nOutputs clients: [[InplaceDimShuffle{0,1,2,3}(Join.0)]]\r\n\r\nBacktrace when the node is created(use Theano flag traceback.limit=N to make it longer):\r\n  File \"train_nmt.py\", line 54, in <module>\r\n    'learning-rate': [0.001]})\r\n  File \"train_nmt.py\", line 33, in main\r\n    use_dropout=params['use-dropout'][0])\r\n  File \"/Users/AmirHJ/projects/deep-learning/seq2seq/nmt2.py\", line 1076, in train\r\n    build_model(tparams, model_options)\r\n  File \"/Users/AmirHJ/projects/deep-learning/seq2seq/nmt2.py\", line 639, in build_model\r\n    prefix='ff_logit_lstm', activ='linear')\r\n  File \"/Users/AmirHJ/projects/deep-learning/seq2seq/nmt2.py\", line 220, in fflayer\r\n    tensor.dot(state_below, tparams[_p(prefix, 'W')]) +\r\n\r\nHINT: Use the Theano flag 'exception_verbosity=high' for a debugprint and storage map footprint of this apply node.\r\n\r\nIf you suspect this is an IPython bug, please report it at:\r\n    https://github.com/ipython/ipython/issues\r\nor send an email to the mailing list at ipython-dev@scipy.org\r\n\r\nYou can print a more detailed traceback right now with \"%tb\", or use \"%debug\"\r\nto interactively debug it.\r\n\r\nExtra-detailed tracebacks for bug-reporting purposes can be enabled via:\r\n    %config Application.verbose_crash=True`"}}