{"action":"closed","issue":{"url":"https://api.github.com/repos/fchollet/keras/issues/4050","repository_url":"https://api.github.com/repos/fchollet/keras","labels_url":"https://api.github.com/repos/fchollet/keras/issues/4050/labels{/name}","comments_url":"https://api.github.com/repos/fchollet/keras/issues/4050/comments","events_url":"https://api.github.com/repos/fchollet/keras/issues/4050/events","html_url":"https://github.com/fchollet/keras/issues/4050","id":182869604,"number":4050,"title":"Automatic Image Inpainting with Keras and GpuElemwise Errors","user":{"login":"Neltherion","id":22426131,"avatar_url":"https://avatars.githubusercontent.com/u/22426131?v=3","gravatar_id":"","url":"https://api.github.com/users/Neltherion","html_url":"https://github.com/Neltherion","followers_url":"https://api.github.com/users/Neltherion/followers","following_url":"https://api.github.com/users/Neltherion/following{/other_user}","gists_url":"https://api.github.com/users/Neltherion/gists{/gist_id}","starred_url":"https://api.github.com/users/Neltherion/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/Neltherion/subscriptions","organizations_url":"https://api.github.com/users/Neltherion/orgs","repos_url":"https://api.github.com/users/Neltherion/repos","events_url":"https://api.github.com/users/Neltherion/events{/privacy}","received_events_url":"https://api.github.com/users/Neltherion/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2016-10-13T18:49:26Z","updated_at":"2016-10-14T06:03:26Z","closed_at":"2016-10-14T06:03:26Z","body":"@fchollet @EderSantana \r\nHi... Before anything I wanted to thank you for the hard work you've been putting into Keras... It's really become a splendid creation... Thanks!\r\n\r\nI have a Model => Input : Gray Image : (1, 224, 224) || Output : RGB Image: (3, 224, 224)\r\nand I want to predict pixel colors by giving it Grayscale images and getting RGB ones.\r\nI tried to make a network in Keras which mostly resembles [this one (which has been made in Tensorflow)](http://tinyclouds.org/colorize).\r\n\r\nHere's the Model code :\r\n\r\n```\r\nfirst_input = Input(batch_shape=(None, 1, 224, 224))\r\n\r\nconv0_1_3 = Convolution2D(3, 3, 3, activation='relu', name='conv0_1_3', border_mode='same')(first_input)\r\n\r\nconv1_1_64 = Convolution2D(64, 3, 3, activation='relu', name='conv1_1', border_mode='same')(conv0_1_3)\r\nconv1_2_64 = Convolution2D(64, 3, 3, activation='relu', name='conv1_2', border_mode='same')(conv1_1_64)\r\nconv1_2_64 = MaxPooling2D((2, 2))(conv1_2_64)\r\n\r\nconv2_1_128 = Convolution2D(128, 3, 3, activation='relu', name='conv2_1', border_mode='same')(conv1_2_64)\r\nconv2_2_128 = Convolution2D(128, 3, 3, activation='relu', name='conv2_2', border_mode='same')(conv2_1_128)\r\nconv2_2_128 = MaxPooling2D((2, 2))(conv2_2_128)\r\n\r\nconv3_1_256 = Convolution2D(256, 3, 3, activation='relu', name='conv3_1', border_mode='same')(conv2_2_128)\r\nconv3_2_256 = Convolution2D(256, 3, 3, activation='relu', name='conv3_2', border_mode='same')(conv3_1_256)\r\nconv3_3_256 = Convolution2D(256, 3, 3, activation='relu', name='conv3_3', border_mode='same')(conv3_2_256)\r\nconv3_3_256 = MaxPooling2D((2, 2))(conv3_3_256)\r\n\r\nconv4_1_512 = Convolution2D(512, 3, 3, activation='relu', name='conv4_1', border_mode='same')(conv3_3_256)\r\nconv4_2_512 = Convolution2D(512, 3, 3, activation='relu', name='conv4_2', border_mode='same')(conv4_1_512)\r\nconv4_3_512 = Convolution2D(512, 3, 3, activation='relu', name='conv4_3', border_mode='same')(conv4_2_512)\r\nconv4_3_512 = MaxPooling2D((2, 2))(conv4_3_512)\r\n\r\nresidual1 = BatchNormalization(axis=1, name='batch1')(conv4_3_512)\r\nresidual1 = Convolution2D(256, 3, 3, activation='relu', name='residual1', border_mode='same')(residual1)\r\nresidual1 = UpSampling2D(name='upsample1')(residual1)\r\n\r\nconv3_3_256_batch_norm = BatchNormalization(axis=1, name='batch2')(conv3_3_256)\r\nmerge1 = merge((conv3_3_256_batch_norm, residual1), mode='concat', name='merge1', concat_axis=0)\r\nresidual2 = Convolution2D(128, 3, 3, activation='relu', name='residual2', border_mode='same')(merge1)\r\nresidual2 = UpSampling2D(name='upsample2')(residual2)\r\n\r\nconv2_2_128_batch_norm = BatchNormalization(axis=1, name='batch3')(conv2_2_128)\r\nmerge2 = merge((conv2_2_128_batch_norm, residual2), mode='concat', name='merge2', concat_axis=0)\r\nresidual3 = Convolution2D(64, 3, 3, activation='relu', name='residual3', border_mode='same')(merge2)\r\nresidual3 = UpSampling2D(name='upsample3')(residual3)\r\n\r\nconv1_2_64_batch_norm = BatchNormalization(axis=1, name='batch4')(conv1_2_64)\r\nmerge3 = merge((conv1_2_64_batch_norm, residual3), mode='concat', name='merge3', concat_axis=0)\r\nresidual4 = Convolution2D(3, 3, 3, activation='relu', name='residual4', border_mode='same')(merge3)\r\nresidual4 = UpSampling2D(name='upsample4')(residual4)\r\n\r\nconv0_1_3_batch_norm = BatchNormalization(axis=1, name='batch5')(conv0_1_3)\r\nmerge4 = merge((conv0_1_3_batch_norm, residual4), mode='concat', name='merge4', concat_axis=0)\r\nresidual5 = Convolution2D(3, 1, 1, activation='relu', name='residual5', border_mode='same')(merge4)\r\n\r\nmodel = Model(input=first_input, output=residual5)\r\n```\r\n\r\nand here's the Model Summary:\r\n\r\n```\r\nLayer (type)                     Output Shape          Param #     Connected to                     \r\n====================================================================================================\r\ninput_1 (InputLayer)             (None, 1, 224, 224)   0                                            \r\n____________________________________________________________________________________________________\r\nconv0_1_3 (Convolution2D)        (None, 3, 224, 224)   30          input_1[0][0]                    \r\n____________________________________________________________________________________________________\r\nconv1_1 (Convolution2D)          (None, 64, 224, 224)  1792        conv0_1_3[0][0]                  \r\n____________________________________________________________________________________________________\r\nconv1_2 (Convolution2D)          (None, 64, 224, 224)  36928       conv1_1[0][0]                    \r\n____________________________________________________________________________________________________\r\nmaxpooling2d_1 (MaxPooling2D)    (None, 64, 112, 112)  0           conv1_2[0][0]                    \r\n____________________________________________________________________________________________________\r\nconv2_1 (Convolution2D)          (None, 128, 112, 112) 73856       maxpooling2d_1[0][0]             \r\n____________________________________________________________________________________________________\r\nconv2_2 (Convolution2D)          (None, 128, 112, 112) 147584      conv2_1[0][0]                    \r\n____________________________________________________________________________________________________\r\nmaxpooling2d_2 (MaxPooling2D)    (None, 128, 56, 56)   0           conv2_2[0][0]                    \r\n____________________________________________________________________________________________________\r\nconv3_1 (Convolution2D)          (None, 256, 56, 56)   295168      maxpooling2d_2[0][0]             \r\n____________________________________________________________________________________________________\r\nconv3_2 (Convolution2D)          (None, 256, 56, 56)   590080      conv3_1[0][0]                    \r\n____________________________________________________________________________________________________\r\nconv3_3 (Convolution2D)          (None, 256, 56, 56)   590080      conv3_2[0][0]                    \r\n____________________________________________________________________________________________________\r\nmaxpooling2d_3 (MaxPooling2D)    (None, 256, 28, 28)   0           conv3_3[0][0]                    \r\n____________________________________________________________________________________________________\r\nconv4_1 (Convolution2D)          (None, 512, 28, 28)   1180160     maxpooling2d_3[0][0]             \r\n____________________________________________________________________________________________________\r\nconv4_2 (Convolution2D)          (None, 512, 28, 28)   2359808     conv4_1[0][0]                    \r\n____________________________________________________________________________________________________\r\nconv4_3 (Convolution2D)          (None, 512, 28, 28)   2359808     conv4_2[0][0]                    \r\n____________________________________________________________________________________________________\r\nmaxpooling2d_4 (MaxPooling2D)    (None, 512, 14, 14)   0           conv4_3[0][0]                    \r\n____________________________________________________________________________________________________\r\nbatch1 (BatchNormalization)      (None, 512, 14, 14)   1024        maxpooling2d_4[0][0]             \r\n____________________________________________________________________________________________________\r\nresidual1 (Convolution2D)        (None, 256, 14, 14)   1179904     batch1[0][0]                     \r\n____________________________________________________________________________________________________\r\nbatch2 (BatchNormalization)      (None, 256, 28, 28)   512         maxpooling2d_3[0][0]             \r\n____________________________________________________________________________________________________\r\nupsample1 (UpSampling2D)         (None, 256, 28, 28)   0           residual1[0][0]                  \r\n____________________________________________________________________________________________________\r\nmerge1 (Merge)                   (None, 256, 28, 28)   0           batch2[0][0]                     \r\n                                                                   upsample1[0][0]                  \r\n____________________________________________________________________________________________________\r\nresidual2 (Convolution2D)        (None, 128, 28, 28)   295040      merge1[0][0]                     \r\n____________________________________________________________________________________________________\r\nbatch3 (BatchNormalization)      (None, 128, 56, 56)   256         maxpooling2d_2[0][0]             \r\n____________________________________________________________________________________________________\r\nupsample2 (UpSampling2D)         (None, 128, 56, 56)   0           residual2[0][0]                  \r\n____________________________________________________________________________________________________\r\nmerge2 (Merge)                   (None, 128, 56, 56)   0           batch3[0][0]                     \r\n                                                                   upsample2[0][0]                  \r\n____________________________________________________________________________________________________\r\nresidual3 (Convolution2D)        (None, 64, 56, 56)    73792       merge2[0][0]                     \r\n____________________________________________________________________________________________________\r\nbatch4 (BatchNormalization)      (None, 64, 112, 112)  128         maxpooling2d_1[0][0]             \r\n____________________________________________________________________________________________________\r\nupsample3 (UpSampling2D)         (None, 64, 112, 112)  0           residual3[0][0]                  \r\n____________________________________________________________________________________________________\r\nmerge3 (Merge)                   (None, 64, 112, 112)  0           batch4[0][0]                     \r\n                                                                   upsample3[0][0]                  \r\n____________________________________________________________________________________________________\r\nresidual4 (Convolution2D)        (None, 3, 112, 112)   1731        merge3[0][0]                     \r\n____________________________________________________________________________________________________\r\nbatch5 (BatchNormalization)      (None, 3, 224, 224)   6           conv0_1_3[0][0]                  \r\n____________________________________________________________________________________________________\r\nupsample4 (UpSampling2D)         (None, 3, 224, 224)   0           residual4[0][0]                  \r\n____________________________________________________________________________________________________\r\nmerge4 (Merge)                   (None, 3, 224, 224)   0           batch5[0][0]                     \r\n                                                                   upsample4[0][0]                  \r\n____________________________________________________________________________________________________\r\nresidual5 (Convolution2D)        (None, 3, 224, 224)   12          merge4[0][0]                     \r\n====================================================================================================\r\nTotal params: 9187699\r\n```\r\n\r\nI don't know what I'm doing wrong since the summary fits exactly with what I have in mind but no matter what, I keep getting this error :\r\n\r\n> ValueError: GpuElemwise. Input dimension mis-match. Input 2 (indices start at 0) has shape[0] == 1, but the output's size on that axis is 5.\r\n> Apply node that caused the error: GpuElemwise{Composite{((i0 * (i1 + Abs(i1))) - i2)},no_inplace}(CudaNdarrayConstant{[[[[ 0.5]]]]}, GpuElemwise{Add}[(0, 0)].0, GpuFromHost.0)\r\n> Toposort index: 916\r\n> Inputs types: [CudaNdarrayType(float32, (True, True, True, True)), CudaNdarrayType(float32, 4D), CudaNdarrayType(float32, 4D)]\r\n> Inputs shapes: [(1, 1, 1, 1), (5, 3, 224, 224), (1, 3, 224, 224)]\r\n> Inputs strides: [(0, 0, 0, 0), (150528, 50176, 224, 1), (0, 50176, 224, 1)]\r\n> Inputs values: [CudaNdarray([[[[ 0.5]]]]), 'not shown', 'not shown']\r\n> Inputs type_num: ['', '', '']\r\n\r\nI've included a Graph of the Model with this question too:\r\n![model](https://cloud.githubusercontent.com/assets/22426131/19362112/3368c77c-9192-11e6-99d9-643600b77c4b.png)\r\n\r\nDebugging this is a nightmare... most of the other errors are pretty easy to understand and fix but these errors are really hard to understand... and unfortunately this isn't the first time I've had these errors with Keras.\r\n\r\nPlease! what is wrong with this model?! am I doing something completely wrong or perhaps this model shouldn't be designed this way?\r\n\r\nThanks so much..."}}
{"actor":{"display_login":"Neltherion"}}