{"action":"created","issue":{"url":"https://api.github.com/repos/fchollet/keras/issues/2841","repository_url":"https://api.github.com/repos/fchollet/keras","labels_url":"https://api.github.com/repos/fchollet/keras/issues/2841/labels{/name}","comments_url":"https://api.github.com/repos/fchollet/keras/issues/2841/comments","events_url":"https://api.github.com/repos/fchollet/keras/issues/2841/events","html_url":"https://github.com/fchollet/keras/issues/2841","id":157365105,"number":2841,"title":"Questions about this convolutional autoencoder","user":{"login":"planaria158","id":11464863,"avatar_url":"https://avatars.githubusercontent.com/u/11464863?v=3","gravatar_id":"","url":"https://api.github.com/users/planaria158","html_url":"https://github.com/planaria158","followers_url":"https://api.github.com/users/planaria158/followers","following_url":"https://api.github.com/users/planaria158/following{/other_user}","gists_url":"https://api.github.com/users/planaria158/gists{/gist_id}","starred_url":"https://api.github.com/users/planaria158/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/planaria158/subscriptions","organizations_url":"https://api.github.com/users/planaria158/orgs","repos_url":"https://api.github.com/users/planaria158/repos","events_url":"https://api.github.com/users/planaria158/events{/privacy}","received_events_url":"https://api.github.com/users/planaria158/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":1,"created_at":"2016-05-29T02:00:10Z","updated_at":"2016-05-29T02:50:15Z","closed_at":null,"body":"Hi All,\r\n\r\nI've been experimenting with autoencoders for reconstructing input images.  Had good success with MNIST digits.  Now wanting to try CIFAR10 images.\r\n\r\nAt the end of this message is the network I've coded up. \r\n\r\nIn this network below, I've experimented with:\r\n- number of filters (64, 32, 16, 8)\r\n- weight initialization for the Conv layers.\r\n- activation functions in the decoding layers.\r\n- overall depth of the network.\r\n- Sometimes I get nan's for loss.  More recently, this example below returns negative values for loss and has a great deal of trouble converging to anything (accuracies are only about 0.003! after 100 epochs and loss is large negative value).\r\n \r\n\r\nSome other details:\r\n- Training set is only 100 images\r\n- Running on my Macbook pro (El Capitan)\r\n- Theano backend\r\n- Using cpu (not gpu)\r\n- Keras 1.0.3\r\n- Theano 0.8.2\r\n- .theanorc: [global] floatX=float64  (I've tried both float32 and float64)  device=cpu  optimizer=fast_run  (fast_compile tends to create NANs)  mode=FAST_RUN  exception_verbosity=high\r\n\r\nHere is my fitting and the first 2 epochs:\r\n```sh\r\nautoencoder.fit(x_train, x_train, nb_epoch=2,  batch_size=20, \r\n          shuffle=True, verbose=1, validation_data=(x_test, x_test))\r\n\r\nTrain on 100 samples, validate on 20 samples\r\nEpoch 1/2\r\n100/100 - 0s - loss: -10.1022 - acc: 0.0515 - val_loss: -49.2346 - val_acc: 0.0464\r\nEpoch 2/2\r\n100/100- 0s - loss: -67.9034 - acc: 0.0470 - val_loss: -111.7369 - val_acc: 0.0464\r\n```\r\n\r\nIt basically moves very little from what you see above (even after a few hundred epochs).  Also, the negative loss values (based on log loss) trouble me since I believe the log loss value should always be > 0.\r\n\r\nVisualizing what the network predicts shows blank images. \r\n\r\nHere is the complete iPython notebook of my network:\r\n[CIFAR10.ipynb.zip](https://github.com/fchollet/keras/files/288093/CIFAR10.ipynb.zip)\r\n"},"comment":{"url":"https://api.github.com/repos/fchollet/keras/issues/comments/222339678","html_url":"https://github.com/fchollet/keras/issues/2841#issuecomment-222339678","issue_url":"https://api.github.com/repos/fchollet/keras/issues/2841","id":222339678,"user":{"login":"joelthchao","id":10974532,"avatar_url":"https://avatars.githubusercontent.com/u/10974532?v=3","gravatar_id":"","url":"https://api.github.com/users/joelthchao","html_url":"https://github.com/joelthchao","followers_url":"https://api.github.com/users/joelthchao/followers","following_url":"https://api.github.com/users/joelthchao/following{/other_user}","gists_url":"https://api.github.com/users/joelthchao/gists{/gist_id}","starred_url":"https://api.github.com/users/joelthchao/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/joelthchao/subscriptions","organizations_url":"https://api.github.com/users/joelthchao/orgs","repos_url":"https://api.github.com/users/joelthchao/repos","events_url":"https://api.github.com/users/joelthchao/events{/privacy}","received_events_url":"https://api.github.com/users/joelthchao/received_events","type":"User","site_admin":false},"created_at":"2016-05-29T02:50:15Z","updated_at":"2016-05-29T02:50:15Z","body":"- Input is not normalized\r\n```python\r\nx_train = x_train.astype('float32') / 255.\r\nx_test = x_test.astype('float32') / 255.\r\n```\r\n- Activation\r\n\r\n`relu` for decoder is ok for me. It's not necessary to do `sigmoid` in the intermediate layers.\r\n\r\n- Traing size\r\n\r\nLarger training set and batch size will be better for learning."}}
{"actor":{"display_login":"joelthchao"}}