{"action":"opened","issue":{"url":"https://api.github.com/repos/fchollet/keras/issues/4060","repository_url":"https://api.github.com/repos/fchollet/keras","labels_url":"https://api.github.com/repos/fchollet/keras/issues/4060/labels{/name}","comments_url":"https://api.github.com/repos/fchollet/keras/issues/4060/comments","events_url":"https://api.github.com/repos/fchollet/keras/issues/4060/events","html_url":"https://github.com/fchollet/keras/issues/4060","id":183068489,"number":4060,"title":"Vanilla Theano replication of a Keras model does not match as expected","user":{"login":"rawhazard","id":17744748,"avatar_url":"https://avatars.githubusercontent.com/u/17744748?v=3","gravatar_id":"","url":"https://api.github.com/users/rawhazard","html_url":"https://github.com/rawhazard","followers_url":"https://api.github.com/users/rawhazard/followers","following_url":"https://api.github.com/users/rawhazard/following{/other_user}","gists_url":"https://api.github.com/users/rawhazard/gists{/gist_id}","starred_url":"https://api.github.com/users/rawhazard/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/rawhazard/subscriptions","organizations_url":"https://api.github.com/users/rawhazard/orgs","repos_url":"https://api.github.com/users/rawhazard/repos","events_url":"https://api.github.com/users/rawhazard/events{/privacy}","received_events_url":"https://api.github.com/users/rawhazard/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2016-10-14T14:44:39Z","updated_at":"2016-10-14T14:44:39Z","closed_at":null,"body":"I find Keras a very useful and well done tool. It is perfect to start using Theano and it is really easily understandable and usable.\r\nNow, I realised a Keras model (using Theano interface), which works perfectly well and I would like to replicate it using only Theano code.\r\nSince Keras is actually using Theano code, I should be able, in principle, to do this.\r\nThe neural net is a convolutional neural network for a one output regression task, with the following layers: conv2d, maxpool2d, conv2d, maxpool2d, dense, dense, output and using Adam optimizer.\r\nUnfortunately, despite it seems to me that I implemented exactly the same neural network with vanilla Theano code, the performance is consistently different.\r\nSo, I guess I must be wrong somewhere, but I can't see where.\r\n\r\nI will put here a link to the codes I'm using, in order to make the post too long. They are short and simple codes, do not worry =)\r\n\r\n[Keras Impl.](http://pastebin.com/7eNubwxw)\r\n\r\n[Theano main](http://pastebin.com/Lvdn6UAc)\r\n[Dense layer with MSE loss function](http://pastebin.com/RyUH07Te)\r\n[Conv layer + max pooling](http://pastebin.com/VVmXm1Uk)\r\n[Updates rule]((http://pastebin.com/fp5Draq7)\r\n\r\n\r\nKeep in mind that the Theano code is mostly adapted from the Theano tutorial found on the website, and the update rules for Adam optimizer is adapted from Keras source code.\r\nI have a large training set, so I usually check after a few epochs the behaviour of the code and this is what I see:\r\nKeras Model: the validation error keeps decreasing, and already after two or three epochs I see a very good match between prediction and true values (points quite close around the bisector in the plot at the bottom of the code)\r\nVanilla Theano Impl: the validation decreases at first, but then some kind of oscillating/overfitting features appear (and the absolute value is 10 times higher than in the Keras impl.), and the match between prediction and true values is worse (points quite spread around the bisector in the plot at the bottom of the code)\r\n\r\nSo, is someone able to tell me where is the difference between the Keras model and my Theano implementation? Since I'm new to the field, I would really like to understand what I'm doing wrong that so strongly affects the performance of the network.\r\nAny help would be appreciated. Thanks"}}
{"actor":{"display_login":"rawhazard"}}