{"action":"opened","issue":{"url":"https://api.github.com/repos/fchollet/keras/issues/1687","repository_url":"https://api.github.com/repos/fchollet/keras","labels_url":"https://api.github.com/repos/fchollet/keras/issues/1687/labels{/name}","comments_url":"https://api.github.com/repos/fchollet/keras/issues/1687/comments","events_url":"https://api.github.com/repos/fchollet/keras/issues/1687/events","html_url":"https://github.com/fchollet/keras/issues/1687","id":132790029,"number":1687,"title":"How to manually reproduce the results of a linear siamese network?","user":{"login":"jsphon","id":5187727,"avatar_url":"https://avatars.githubusercontent.com/u/5187727?v=3","gravatar_id":"","url":"https://api.github.com/users/jsphon","html_url":"https://github.com/jsphon","followers_url":"https://api.github.com/users/jsphon/followers","following_url":"https://api.github.com/users/jsphon/following{/other_user}","gists_url":"https://api.github.com/users/jsphon/gists{/gist_id}","starred_url":"https://api.github.com/users/jsphon/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jsphon/subscriptions","organizations_url":"https://api.github.com/users/jsphon/orgs","repos_url":"https://api.github.com/users/jsphon/repos","events_url":"https://api.github.com/users/jsphon/events{/privacy}","received_events_url":"https://api.github.com/users/jsphon/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":0,"created_at":"2016-02-10T19:08:39Z","updated_at":"2016-02-10T19:08:39Z","closed_at":null,"body":"Hello,\r\n\r\nI am trying to recreate the calculations of a Siamese Network. The purpose of this exercise is to ensure that I understand what is happening in the background.\r\n\r\nIn the code below, I fit the Siamese Network with random data. Then I use the model to predict the outputs (y_hat1). Then I attempt to reproduce the predictions myself (y_hat2). However, the results are very different.\r\n\r\nPlease could you help me understand where I am going wrong.\r\n\r\n```\r\nfrom keras.models import Sequential\r\nfrom keras.layers.core import Dense, Dropout, Activation, Siamese, Layer\r\nfrom keras.optimizers import SGD\r\n\r\nimport matplotlib.pyplot as plt\r\n%matplotlib inline\r\n\r\nfrom pylab import rcParams\r\nrcParams['figure.figsize'] = 10, 8\r\n\r\nimport numpy as np\r\nimport keras\r\nN       = 100\r\nX_train = np.random.randn( 100, 5 )\r\ny_train = 0.1 * X_train[:,[0]] - 0.2 * X_train[:,[1]] + 0.01 * np.random.randn( 100, 1 )\r\n\r\nin_dim  = 5\r\ninput1 = Sequential()\r\ninput2 = Sequential()\r\ninput1.add(Layer(input_shape=(in_dim,)))\r\ninput2.add(Layer(input_shape=(in_dim,)))\r\nshared_layer      = Dense(1, activation='linear')\r\nsiamese_layer = Siamese(shared_layer, [input1, input2], merge_mode='concat', concat_axis=1)\r\n\r\noutput_layer = Dense( 1, activation='linear' )\r\nmodel = Sequential()\r\nmodel.add( siamese_layer )\r\nmodel.add( output_layer )\r\nmodel.compile(loss='mean_squared_error', optimizer='sgd')\r\nmodel.fit( [ X_train, X_train ], y_train, nb_epoch=10 )\r\n\r\n# Use the model\r\ny_hat1 = model.predict( [X_train, X_train])\r\n\r\n# Recreate the results manually\r\nslW      = shared_layer.W.get_value()\r\nslb      = shared_layer.b.get_value()\r\nww       = np.repeat( slW, 2, axis=0 )\r\nbb       = np.repeat( slb, 2, axis=0 )\r\n\r\nolW      = output_layer.W.get_value()\r\nolb      = output_layer.b.get_value()\r\n\r\nXX_train = np.c_[ X_train, X_train ]\r\nl1       = np.dot( XX_train, ww ) + bb\r\ny_hat2   = np.dot( l1, olW ) + olb\r\n\r\n# Compare the results\r\nprint( np.c_[ y_hat1, y_hat2 ] )\r\n\r\nplt.plot(y_hat1)\r\nplt.plot(y_hat2)\r\n\r\nnp.testing.assert_array_almost_equal( y_hat1, y_hat2 )\r\n``` \r\n\r\nThank you!\r\n\r\nJon."}}