{"action":"closed","issue":{"url":"https://api.github.com/repos/fchollet/keras/issues/1699","repository_url":"https://api.github.com/repos/fchollet/keras","labels_url":"https://api.github.com/repos/fchollet/keras/issues/1699/labels{/name}","comments_url":"https://api.github.com/repos/fchollet/keras/issues/1699/comments","events_url":"https://api.github.com/repos/fchollet/keras/issues/1699/events","html_url":"https://github.com/fchollet/keras/issues/1699","id":133143891,"number":1699,"title":"No effect of pretraining using autoencoder","user":{"login":"snurkabill","id":4818573,"avatar_url":"https://avatars.githubusercontent.com/u/4818573?v=3","gravatar_id":"","url":"https://api.github.com/users/snurkabill","html_url":"https://github.com/snurkabill","followers_url":"https://api.github.com/users/snurkabill/followers","following_url":"https://api.github.com/users/snurkabill/following{/other_user}","gists_url":"https://api.github.com/users/snurkabill/gists{/gist_id}","starred_url":"https://api.github.com/users/snurkabill/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/snurkabill/subscriptions","organizations_url":"https://api.github.com/users/snurkabill/orgs","repos_url":"https://api.github.com/users/snurkabill/repos","events_url":"https://api.github.com/users/snurkabill/events{/privacy}","received_events_url":"https://api.github.com/users/snurkabill/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":11,"created_at":"2016-02-12T02:46:30Z","updated_at":"2016-02-28T16:42:58Z","closed_at":"2016-02-28T16:42:58Z","body":"Hi, I may have doing something wrong here, but I am trying to follow steps in documentation. \r\n\r\nConsider this code: \r\n```\r\n(X_train, y_train), (X_test, y_test) = mnist.load_data()\r\n\r\nX_train = X_train.reshape(60000, 784)\r\nX_test = X_test.reshape(10000, 784)\r\n\r\nY_train = np_utils.to_categorical(y_train, 10)\r\nY_test = np_utils.to_categorical(y_test, 10)\r\n\r\nX_train = X_train.astype('float32')\r\nX_test = X_test.astype('float32')\r\nX_train /= 255\r\nX_test /= 255\r\n\r\ntopology = [len(X_train[0]), 200, 100, len(Y_train[0])]\r\nbatch_size = 128\r\nnb_pretraining_epochs = 00\r\nnb_epoch = 2\r\n\r\n\r\n\r\nautoencoder_0 = Sequential()\r\n\r\nencoder_0 = Sequential([Dense(output_dim = 200, input_dim = 784, activation = \"softplus\")])\r\ndecoder_0 = Sequential([Dense(output_dim = 784, input_dim = 200)])\r\n\r\nautoencoder_0.add(AutoEncoder(encoder = encoder_0, decoder = decoder_0, output_reconstruction = True))\r\nautoencoder_0.compile(optimizer = Adam(lr = 0.001), loss = 'mse')\r\nautoencoder_0.fit(X_train, X_train, batch_size = batch_size, nb_epoch = nb_pretraining_epochs)\r\n\r\ntemp_0 = Sequential()\r\ntemp_0.add(encoder_0)\r\ntemp_0.compile(loss = 'mse', optimizer = Adam(lr = 0.001))\r\n\r\nX_train_1 = temp_0.predict(X_train)\r\n\r\nautoencoder_1 = Sequential()\r\nencoder_1 = Sequential([Dense(input_dim = 200, output_dim = 100, activation = \"softplus\")])\r\ndecoder_1 = Sequential([Dense(input_dim = 100, output_dim = 200)])\r\n\r\nautoencoder_1.add(AutoEncoder(encoder = encoder_1, decoder = decoder_1, output_reconstruction = True))\r\nautoencoder_1.compile(optimizer = Adam(0.001), loss = 'mse')\r\nautoencoder_1.fit(X_train_1, X_train_1, batch_size = batch_size, nb_epoch =  nb_pretraining_epochs)\r\n\r\nmodel = Sequential()\r\nmodel.add(encoder_0)\r\nmodel.add(encoder_1)\r\nmodel.add(Dense(input_dim = 100, output_dim = 10, init = 'zero', activation = 'softmax'))\r\n\r\n\r\nadam = Adam(lr = 0.001)\r\nmodel.compile(optimizer = adam, loss = \"categorical_crossentropy\")\r\n\r\nmodel.fit(X = X_train,\r\n          y = Y_train,\r\n          batch_size = batch_size,\r\n          nb_epoch = nb_epoch,\r\n          show_accuracy = True,\r\n          verbose = 1,\r\n          validation_data = (X_test, Y_test),\r\n          shuffle = True)\r\n```\r\n\r\nnow, no matter how I setup nb_pretraining_epochs, the accuracies are the same after 2 fine-tuning epochs. What am I doing wrong? Do I pass weights from encoders correctly? "}}