{"action":"created","issue":{"url":"https://api.github.com/repos/tensorflow/tensorflow/issues/18660","repository_url":"https://api.github.com/repos/tensorflow/tensorflow","labels_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/18660/labels{/name}","comments_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/18660/comments","events_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/18660/events","html_url":"https://github.com/tensorflow/tensorflow/issues/18660","id":315538573,"number":18660,"title":"Bug: tf.keras.estimator.model_to_estimator() API giving error when Keras model contains Lambda layer.","user":{"login":"sibyjackgrove","id":25213730,"avatar_url":"https://avatars0.githubusercontent.com/u/25213730?v=4","gravatar_id":"","url":"https://api.github.com/users/sibyjackgrove","html_url":"https://github.com/sibyjackgrove","followers_url":"https://api.github.com/users/sibyjackgrove/followers","following_url":"https://api.github.com/users/sibyjackgrove/following{/other_user}","gists_url":"https://api.github.com/users/sibyjackgrove/gists{/gist_id}","starred_url":"https://api.github.com/users/sibyjackgrove/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/sibyjackgrove/subscriptions","organizations_url":"https://api.github.com/users/sibyjackgrove/orgs","repos_url":"https://api.github.com/users/sibyjackgrove/repos","events_url":"https://api.github.com/users/sibyjackgrove/events{/privacy}","received_events_url":"https://api.github.com/users/sibyjackgrove/received_events","type":"User","site_admin":false},"labels":[{"id":404586594,"url":"https://api.github.com/repos/tensorflow/tensorflow/labels/stat:awaiting%20tensorflower","name":"stat:awaiting tensorflower","color":"b60205","default":false},{"id":473172988,"url":"https://api.github.com/repos/tensorflow/tensorflow/labels/type:bug/performance","name":"type:bug/performance","color":"fbca04","default":false}],"state":"open","locked":false,"assignee":{"login":"fchollet","id":710255,"avatar_url":"https://avatars3.githubusercontent.com/u/710255?v=4","gravatar_id":"","url":"https://api.github.com/users/fchollet","html_url":"https://github.com/fchollet","followers_url":"https://api.github.com/users/fchollet/followers","following_url":"https://api.github.com/users/fchollet/following{/other_user}","gists_url":"https://api.github.com/users/fchollet/gists{/gist_id}","starred_url":"https://api.github.com/users/fchollet/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fchollet/subscriptions","organizations_url":"https://api.github.com/users/fchollet/orgs","repos_url":"https://api.github.com/users/fchollet/repos","events_url":"https://api.github.com/users/fchollet/events{/privacy}","received_events_url":"https://api.github.com/users/fchollet/received_events","type":"User","site_admin":false},"assignees":[{"login":"fchollet","id":710255,"avatar_url":"https://avatars3.githubusercontent.com/u/710255?v=4","gravatar_id":"","url":"https://api.github.com/users/fchollet","html_url":"https://github.com/fchollet","followers_url":"https://api.github.com/users/fchollet/followers","following_url":"https://api.github.com/users/fchollet/following{/other_user}","gists_url":"https://api.github.com/users/fchollet/gists{/gist_id}","starred_url":"https://api.github.com/users/fchollet/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/fchollet/subscriptions","organizations_url":"https://api.github.com/users/fchollet/orgs","repos_url":"https://api.github.com/users/fchollet/repos","events_url":"https://api.github.com/users/fchollet/events{/privacy}","received_events_url":"https://api.github.com/users/fchollet/received_events","type":"User","site_admin":false},{"login":"angersson","id":32465472,"avatar_url":"https://avatars2.githubusercontent.com/u/32465472?v=4","gravatar_id":"","url":"https://api.github.com/users/angersson","html_url":"https://github.com/angersson","followers_url":"https://api.github.com/users/angersson/followers","following_url":"https://api.github.com/users/angersson/following{/other_user}","gists_url":"https://api.github.com/users/angersson/gists{/gist_id}","starred_url":"https://api.github.com/users/angersson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/angersson/subscriptions","organizations_url":"https://api.github.com/users/angersson/orgs","repos_url":"https://api.github.com/users/angersson/repos","events_url":"https://api.github.com/users/angersson/events{/privacy}","received_events_url":"https://api.github.com/users/angersson/received_events","type":"User","site_admin":false}],"milestone":null,"comments":1,"created_at":"2018-04-18T15:48:33Z","updated_at":"2018-05-05T18:31:17Z","closed_at":null,"author_association":"CONTRIBUTOR","body":"------------------------\r\n\r\n### System information\r\n- **Have I written custom code (as opposed to using a stock example script provided in TensorFlow)**: Custom code\r\n- **OS Platform and Distribution (e.g., Linux Ubuntu 16.04)**: Windows\r\n- **TensorFlow installed from (source or binary)**: Binary\r\n- **TensorFlow version (use command below)**: 1.7\r\n- **Python version**:  3.6\r\n- **Bazel version (if compiling from source)**: N/A\r\n- **GCC/Compiler version (if compiling from source)**: N/A\r\n- **CUDA/cuDNN version**: N/A\r\n- **GPU model and memory**: N/A\r\n- **Exact command to reproduce**:\r\n`estimator_vae = tf.keras.estimator.model_to_estimator(keras_model=vae)`\r\n\r\n### Describe the problem\r\n `tf.keras.estimator.model_to_estimator()` API is failing when Keras model contains `Lambda ` layer.\r\nThe error I am getting is `SystemError: unknown opcode`\r\nThe problem seems to be only there when I am using custom functions inside the Keras mode.\r\n\r\n### Source code / logs\r\nI implemented a VAE in Keras and was trying to convert it into an TF estimator model. The model works and trains using Keras. The Keras model has functions for gaussian sampling and VAE training loss. The code and  error trace back is given below.\r\n\r\n```\r\n#Encoder network, mapping inputs to our latent distribution parameters:\r\nx = Input(batch_shape=(batch_size, original_dim),name='encoder_input')\r\nencoded = Dense(intermediate_dim, activation='relu',name='encoder_dense_1')(x)\r\nz_mean = Dense(latent_dim,name='z_mean')(encoded)\r\nz_log_var = Dense(latent_dim,name='z_log_var')(encoded)\r\n\r\n# Sampling from Gaussian\r\ndef sampling(args):\r\n    \r\n    z_mean, z_log_var = args\r\n\r\n    epsilon = tf.random_normal(shape=(batch_size, latent_dim),\r\n                               mean=0., stddev=epsilon_std) \r\n    \r\n    return z_mean + tf.exp(z_log_var/2) * epsilon\r\nz = Lambda(sampling,name='z')([z_mean, z_log_var]) \r\n\r\n#Compute VAE loss\r\ndef vae_loss(x, x_decoded):\r\n    x_mse_loss = original_dim*tf.keras.losses.mean_squared_error(tf.layers.flatten(x), tf.layers.flatten(x_decoded))\r\n    beta = 4.0\r\n    kl_loss = - 0.5*beta* tf.reduce_sum(1 + z_log_var - tf.square(z_mean) - tf.exp(z_log_var), axis=-1)\r\n    return tf.reduce_mean(x_mse_loss + kl_loss)\r\n\r\n# Map these sampled latent points back to reconstructed inputs:\r\n#Decoder network layers\r\ndecoder_dense_1 = Dense(intermediate_dim, activation='relu',name='decoder_dense_1')\r\ndecoder_output = Dense(48, activation='relu',name='decoder_output')\r\ndecoded = decoder_dense_1(z)\r\nx_decoded = decoder_output(decoded)\r\n\r\n# end-to-end autoencoder\r\nvae = Model(x, x_decoded)\r\nvae.compile(optimizer='adam',loss=vae_loss) \r\nvae.summary()\r\n#Converting to tf estimator\r\nestimator_vae = tf.keras.estimator.model_to_estimator(keras_model=vae)\r\n```\r\nOutput:\r\n\r\n```\r\n__________________________________________________________________________________________________\r\nLayer (type)                    Output Shape         Param #     Connected to                     \r\n==================================================================================================\r\nencoder_input (InputLayer)      (50, 48)             0                                            \r\n__________________________________________________________________________________________________\r\nencoder_dense_1 (Dense)         (50, 24)             1176        encoder_input[0][0]              \r\n__________________________________________________________________________________________________\r\nz_mean (Dense)                  (50, 10)             250         encoder_dense_1[0][0]            \r\n__________________________________________________________________________________________________\r\nz_log_var (Dense)               (50, 10)             250         encoder_dense_1[0][0]            \r\n__________________________________________________________________________________________________\r\nz (Lambda)                      (50, 10)             0           z_mean[0][0]                     \r\n                                                                 z_log_var[0][0]                  \r\n__________________________________________________________________________________________________\r\ndecoder_dense_1 (Dense)         (50, 24)             264         z[0][0]                          \r\n__________________________________________________________________________________________________\r\ndecoder_output (Dense)          (50, 48)             1200        decoder_dense_1[0][0]            \r\n==================================================================================================\r\nTotal params: 3,140\r\nTrainable params: 3,140\r\nNon-trainable params: 0\r\n__________________________________________________________________________________________________\r\nINFO:tensorflow:Using the Keras model provided.\r\nINFO:tensorflow:Using default config.\r\nWARNING:tensorflow:Using temporary folder as model directory: C:\\Users\\SPLATH~1\\AppData\\Local\\Temp\\tmps7nagdhz\r\nINFO:tensorflow:Using config: {'_model_dir': 'C:\\\\Users\\\\SPLATH~1\\\\AppData\\\\Local\\\\Temp\\\\tmps7nagdhz', '_tf_random_seed': None, '_save_summary_steps': 100, '_save_checkpoints_steps': None, '_save_checkpoints_secs': 600, '_session_config': None, '_keep_checkpoint_max': 5, '_keep_checkpoint_every_n_hours': 10000, '_log_step_count_steps': 100, '_service': None, '_cluster_spec': <tensorflow.python.training.server_lib.ClusterSpec object at 0x00000238136D9DA0>, '_task_type': 'worker', '_task_id': 0, '_global_id_in_cluster': 0, '_master': '', '_evaluation_master': '', '_is_chief': True, '_num_ps_replicas': 0, '_num_worker_replicas': 1}\r\n---------------------------------------------------------------------------\r\nSystemError                               Traceback (most recent call last)\r\n<ipython-input-10-77cc01c33881> in <module>()\r\n     13 vae.summary()\r\n     14 #Converting to tf estimator\r\n---> 15 estimator_vae = tf.keras.estimator.model_to_estimator(keras_model=vae)\r\n\r\n~\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\tensorflow\\python\\keras\\_impl\\keras\\estimator.py in model_to_estimator(keras_model, keras_model_path, custom_objects, model_dir, config)\r\n    481                            estimator,\r\n    482                            custom_objects,\r\n--> 483                            keras_weights)\r\n    484   elif keras_model.built:\r\n    485     logging.warning('You are creating an Estimator from a Keras model '\r\n\r\n~\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\tensorflow\\python\\keras\\_impl\\keras\\estimator.py in _save_first_checkpoint(keras_model, estimator, custom_objects, keras_weights)\r\n    396       training_util.create_global_step()\r\n    397       model = _clone_and_build_model(model_fn_lib.ModeKeys.TRAIN, keras_model,\r\n--> 398                                      custom_objects)\r\n    399       if isinstance(model, models.Sequential):\r\n    400         model = model.model\r\n\r\n~\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\tensorflow\\python\\keras\\_impl\\keras\\estimator.py in _clone_and_build_model(mode, keras_model, custom_objects, features, labels)\r\n    270         model = models.clone_model(keras_model, input_tensors=input_tensors)\r\n    271     else:\r\n--> 272       model = models.clone_model(keras_model, input_tensors=input_tensors)\r\n    273   else:\r\n    274     model = keras_model\r\n\r\n~\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\tensorflow\\python\\keras\\_impl\\keras\\models.py in clone_model(model, input_tensors)\r\n    261     return _clone_sequential_model(model, input_tensors=input_tensors)\r\n    262   else:\r\n--> 263     return _clone_functional_model(model, input_tensors=input_tensors)\r\n\r\n~\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\tensorflow\\python\\keras\\_impl\\keras\\models.py in _clone_functional_model(model, input_tensors)\r\n    166               kwargs['mask'] = computed_masks\r\n    167           output_tensors = generic_utils.to_list(layer(computed_tensors,\r\n--> 168                                                        **kwargs))\r\n    169           output_masks = generic_utils.to_list(\r\n    170               layer.compute_mask(computed_tensors, computed_masks))\r\n\r\n~\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\tensorflow\\python\\keras\\_impl\\keras\\engine\\base_layer.py in __call__(self, inputs, **kwargs)\r\n    237     \"\"\"\r\n    238     # Actually call the layer (optionally building it).\r\n--> 239     output = super(Layer, self).__call__(inputs, **kwargs)\r\n    240     if context.executing_eagerly():\r\n    241       return output\r\n\r\n~\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\tensorflow\\python\\layers\\base.py in __call__(self, inputs, *args, **kwargs)\r\n    712 \r\n    713         if not in_deferred_mode:\r\n--> 714           outputs = self.call(inputs, *args, **kwargs)\r\n    715           if outputs is None:\r\n    716             raise ValueError('A layer\\'s `call` method should return a Tensor '\r\n\r\n~\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\tensorflow\\python\\keras\\_impl\\keras\\layers\\core.py in call(self, inputs, mask)\r\n    640     if has_arg(self.function, 'mask'):\r\n    641       arguments['mask'] = mask\r\n--> 642     return self.function(inputs, **arguments)\r\n    643 \r\n    644   def compute_mask(self, inputs, mask=None):\r\n\r\n~\\AppData\\Local\\Continuum\\anaconda3\\lib\\site-packages\\tensorflow\\python\\keras\\_impl\\keras\\layers\\core.py in sampling(args)\r\n     16 def sampling(args):\r\n     17     #import tensorflow as tf\r\n---> 18     z_mean, z_log_var = args\r\n     19 \r\n     20     epsilon = tf.random_normal(shape=(batch_size, latent_dim),\r\n\r\nSystemError: unknown opcode\r\n```\r\n"},"comment":{"url":"https://api.github.com/repos/tensorflow/tensorflow/issues/comments/386825779","html_url":"https://github.com/tensorflow/tensorflow/issues/18660#issuecomment-386825779","issue_url":"https://api.github.com/repos/tensorflow/tensorflow/issues/18660","id":386825779,"user":{"login":"tensorflowbutler","id":28546240,"avatar_url":"https://avatars2.githubusercontent.com/u/28546240?v=4","gravatar_id":"","url":"https://api.github.com/users/tensorflowbutler","html_url":"https://github.com/tensorflowbutler","followers_url":"https://api.github.com/users/tensorflowbutler/followers","following_url":"https://api.github.com/users/tensorflowbutler/following{/other_user}","gists_url":"https://api.github.com/users/tensorflowbutler/gists{/gist_id}","starred_url":"https://api.github.com/users/tensorflowbutler/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/tensorflowbutler/subscriptions","organizations_url":"https://api.github.com/users/tensorflowbutler/orgs","repos_url":"https://api.github.com/users/tensorflowbutler/repos","events_url":"https://api.github.com/users/tensorflowbutler/events{/privacy}","received_events_url":"https://api.github.com/users/tensorflowbutler/received_events","type":"User","site_admin":false},"created_at":"2018-05-05T18:31:16Z","updated_at":"2018-05-05T18:31:16Z","author_association":"MEMBER","body":"Nagging Assignees @fchollet, @angersson: It has been 14 days with no activity and this issue has an assignee. Please update the label and/or status accordingly."}}
{"actor":{"display_login":"tensorflowbutler"}}