{"action":"opened","issue":{"url":"https://api.github.com/repos/lvapeab/nmt-keras/issues/77","repository_url":"https://api.github.com/repos/lvapeab/nmt-keras","labels_url":"https://api.github.com/repos/lvapeab/nmt-keras/issues/77/labels{/name}","comments_url":"https://api.github.com/repos/lvapeab/nmt-keras/issues/77/comments","events_url":"https://api.github.com/repos/lvapeab/nmt-keras/issues/77/events","html_url":"https://github.com/lvapeab/nmt-keras/issues/77","id":363829313,"node_id":"MDU6SXNzdWUzNjM4MjkzMTM=","number":77,"title":"Error during training","user":{"login":"parmpreet","id":4441356,"node_id":"MDQ6VXNlcjQ0NDEzNTY=","avatar_url":"https://avatars3.githubusercontent.com/u/4441356?v=4","gravatar_id":"","url":"https://api.github.com/users/parmpreet","html_url":"https://github.com/parmpreet","followers_url":"https://api.github.com/users/parmpreet/followers","following_url":"https://api.github.com/users/parmpreet/following{/other_user}","gists_url":"https://api.github.com/users/parmpreet/gists{/gist_id}","starred_url":"https://api.github.com/users/parmpreet/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/parmpreet/subscriptions","organizations_url":"https://api.github.com/users/parmpreet/orgs","repos_url":"https://api.github.com/users/parmpreet/repos","events_url":"https://api.github.com/users/parmpreet/events{/privacy}","received_events_url":"https://api.github.com/users/parmpreet/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":0,"created_at":"2018-09-26T02:01:18Z","updated_at":"2018-09-26T02:01:18Z","closed_at":null,"author_association":"NONE","body":"Hi,\r\nI am trying to setup nmt-keras and train the model using the default values provided. I followed the instructions from https://nmt-keras.readthedocs.io/en/latest/requirements.html#requirements\r\nI am using `Python 2.7.14 :: Anaconda, Inc.`\r\nMacbook pro osx `10.13.6 (17G65)`\r\nlatest tensorflow version using `pip install tensorflow`\r\n\r\nI am getting the following error while trying to start training:\r\n`[26/09/2018 13:29:49] Preparing optimizer: Adam [LR: 0.001 - LOSS: categorical_crossentropy - CLIP_C 5.0 - CLIP_V  0.0 - LR_OPTIMIZER_DECAY 0.0] and compiling.\r\n[26/09/2018 13:29:49] Starting training!\r\n[26/09/2018 13:29:49] <<< Training model >>>\r\n[26/09/2018 13:29:49] Training parameters: {'reload_epoch': 0, 'wo_da_patch_type': 'whole', 'shuffle': True, 'verbose': 1, 'data_augmentation': False, 'class_weights': None, 'epochs_for_save': 1, 'lr_reducer_exp_base': -0.5, 'lr_gamma': 0.8, 'start_eval_on_epoch': 1, 'n_epochs': 500, 'patience': 10, 'tensorboard': True, 'epoch_offset': 0, 'da_patch_type': 'resize_and_rndcrop', 'mean_substraction': False, 'normalize': True, 'n_gpus': 1, 'n_parallel_loaders': 1, 'each_n_epochs': 1, 'patience_check_split': 'val', 'da_enhance_list': [], 'reduce_each_epochs': False, 'lr_warmup_exp': -1.5, 'lr_reducer_type': 'exponential', 'tensorboard_params': {'batch_size': 50, 'write_grads': False, 'write_images': False, 'label_word_embeddings_with_vocab': True, 'embeddings_metadata': None, 'write_graph': True, 'embeddings_layer_names': ['source_word_embedding', 'target_word_embedding'], 'word_embeddings_labels': ['source_text', 'target_text'], 'embeddings_freq': 1, 'log_dir': 'tensorboard_logs', 'histogram_freq': 0}, 'metric_check': 'Bleu_4', 'start_reduction_on_epoch': 0, 'homogeneous_batches': False, 'batch_size': 50, 'num_iterations_val': None, 'normalization_type': '(-1)-1', 'initial_lr': 0.001, 'lr_half_life': 100, 'lr_decay': None, 'joint_batches': 4, 'maxlen': 50, 'eval_on_epochs': True, 'eval_on_sets': [], 'extra_callbacks': [<keras_wrapper.extra.callbacks.EvalPerformance object at 0x1c23378050>, <keras_wrapper.extra.callbacks.Sample object at 0x1c23378710>]}\r\n[26/09/2018 13:29:49] <<< creating directory trained_models/EuTrans_esen_AttentionRNNEncoderDecoder_src_emb_32_bidir_True_enc_LSTM_32_dec_ConditionalLSTM_32_deepout_linear_trg_emb_32_Adam_0.001//tensorboard_logs ... >>>\r\n2018-09-26 13:29:53.546212: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA\r\nTraceback (most recent call last):\r\n  File \"main.py\", line 49, in <module>\r\n    train_model(parameters, args.dataset)\r\n  File \"/Users/parm/Documents/learning/nmt/nmt-keras/nmt_keras/training.py\", line 158, in train_model\r\n    nmt_model.trainNet(dataset, training_params)\r\n  File \"/Users/parm/Documents/learning/nmt/nmt-keras/src/keras-wrapper/keras_wrapper/cnn_model.py\", line 886, in trainNet\r\n    self.__train(ds, params)\r\n  File \"/Users/parm/Documents/learning/nmt/nmt-keras/src/keras-wrapper/keras_wrapper/cnn_model.py\", line 1111, in __train\r\n    initial_epoch=params['epoch_offset'])\r\n  File \"/Users/parm/Documents/learning/nmt/nmt-keras/src/keras/keras/legacy/interfaces.py\", line 91, in wrapper\r\n    return func(*args, **kwargs)\r\n  File \"/Users/parm/Documents/learning/nmt/nmt-keras/src/keras/keras/engine/training.py\", line 1475, in fit_generator\r\n    initial_epoch=initial_epoch)\r\n  File \"/Users/parm/Documents/learning/nmt/nmt-keras/src/keras/keras/engine/training_generator.py\", line 94, in fit_generator\r\n    callbacks.set_model(callback_model)\r\n  File \"/Users/parm/Documents/learning/nmt/nmt-keras/src/keras/keras/callbacks.py\", line 54, in set_model\r\n    callback.set_model(model)\r\n  File \"/Users/parm/Documents/learning/nmt/nmt-keras/src/keras/keras/callbacks.py\", line 771, in set_model\r\n    self.sess = K.get_session()\r\n  File \"/Users/parm/Documents/learning/nmt/nmt-keras/src/keras/keras/backend/tensorflow_backend.py\", line 223, in get_session\r\n    [tf.is_variable_initialized(v) for v in candidate_vars])\r\n  File \"/Users/parm/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/client/session.py\", line 877, in run\r\n    run_metadata_ptr)\r\n  File \"/Users/parm/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/client/session.py\", line 1100, in _run\r\n    feed_dict_tensor, options, run_metadata)\r\n  File \"/Users/parm/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/client/session.py\", line 1272, in _do_run\r\n    run_metadata)\r\n  File \"/Users/parm/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/client/session.py\", line 1291, in _do_call\r\n    raise type(e)(node_def, op, message)\r\ntensorflow.python.framework.errors_impl.InvalidArgumentError: No OpKernel was registered to support Op 'CudnnRNN' with these attrs.  Registered devices: [CPU], Registered kernels:\r\n  <no registered kernels>\r\n\r\n\t [[Node: bidirectional_encoder_LSTM/CudnnRNN = CudnnRNN[T=DT_FLOAT, direction=\"unidirectional\", dropout=0, input_mode=\"linear_input\", is_training=true, rnn_mode=\"lstm\", seed=87654321, seed2=0](bidirectional_encoder_LSTM/transpose, bidirectional_encoder_LSTM/ExpandDims_1, bidirectional_encoder_LSTM/ExpandDims_2, bidirectional_encoder_LSTM/concat)]]\r\n\r\nCaused by op u'bidirectional_encoder_LSTM/CudnnRNN', defined at:\r\n  File \"main.py\", line 49, in <module>\r\n    train_model(parameters, args.dataset)\r\n  File \"/Users/parm/Documents/learning/nmt/nmt-keras/nmt_keras/training.py\", line 81, in train_model\r\n    clear_dirs=clear_dirs)\r\n  File \"/Users/parm/Documents/learning/nmt/nmt-keras/nmt_keras/model_zoo.py\", line 139, in __init__\r\n    eval('self.' + model_type + '(params)')\r\n  File \"<string>\", line 1, in <module>\r\n  File \"/Users/parm/Documents/learning/nmt/nmt-keras/nmt_keras/model_zoo.py\", line 364, in AttentionRNNEncoderDecoder\r\n    merge_mode=params.get('BIDIRECTIONAL_MERGE_MODE', 'concat'))(src_embedding)\r\n  File \"/Users/parm/Documents/learning/nmt/nmt-keras/src/keras/keras/layers/wrappers.py\", line 430, in __call__\r\n    return super(Bidirectional, self).__call__(inputs, **kwargs)\r\n  File \"/Users/parm/Documents/learning/nmt/nmt-keras/src/keras/keras/engine/base_layer.py\", line 457, in __call__\r\n    output = self.call(inputs, **kwargs)\r\n  File \"/Users/parm/Documents/learning/nmt/nmt-keras/src/keras/keras/layers/wrappers.py\", line 525, in call\r\n    y = self.forward_layer.call(inputs, **kwargs)\r\n  File \"/Users/parm/Documents/learning/nmt/nmt-keras/src/keras/keras/layers/cudnn_recurrent.py\", line 90, in call\r\n    output, states = self._process_batch(inputs, initial_state)\r\n  File \"/Users/parm/Documents/learning/nmt/nmt-keras/src/keras/keras/layers/cudnn_recurrent.py\", line 517, in _process_batch\r\n    is_training=True)\r\n  File \"/Users/parm/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/contrib/cudnn_rnn/python/ops/cudnn_rnn_ops.py\", line 1544, in __call__\r\n    input_data, input_h, input_c, params, is_training=is_training)\r\n  File \"/Users/parm/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/contrib/cudnn_rnn/python/ops/cudnn_rnn_ops.py\", line 1435, in __call__\r\n    seed=self._seed)\r\n  File \"/Users/parm/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/contrib/cudnn_rnn/python/ops/cudnn_rnn_ops.py\", line 922, in _cudnn_rnn\r\n    outputs, output_h, output_c, _ = gen_cudnn_rnn_ops.cudnn_rnn(**args)\r\n  File \"/Users/parm/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/ops/gen_cudnn_rnn_ops.py\", line 115, in cudnn_rnn\r\n    is_training=is_training, name=name)\r\n  File \"/Users/parm/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/framework/op_def_library.py\", line 787, in _apply_op_helper\r\n    op_def=op_def)\r\n  File \"/Users/parm/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/util/deprecation.py\", line 454, in new_func\r\n    return func(*args, **kwargs)\r\n  File \"/Users/parm/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/framework/ops.py\", line 3155, in create_op\r\n    op_def=op_def)\r\n  File \"/Users/parm/anaconda3/envs/py27/lib/python2.7/site-packages/tensorflow/python/framework/ops.py\", line 1717, in __init__\r\n    self._traceback = tf_stack.extract_stack()\r\n\r\nInvalidArgumentError (see above for traceback): No OpKernel was registered to support Op 'CudnnRNN' with these attrs.  Registered devices: [CPU], Registered kernels:\r\n  <no registered kernels>\r\n\r\n\t [[Node: bidirectional_encoder_LSTM/CudnnRNN = CudnnRNN[T=DT_FLOAT, direction=\"unidirectional\", dropout=0, input_mode=\"linear_input\", is_training=true, rnn_mode=\"lstm\", seed=87654321, seed2=0](bidirectional_encoder_LSTM/transpose, bidirectional_encoder_LSTM/ExpandDims_1, bidirectional_encoder_LSTM/ExpandDims_2, bidirectional_encoder_LSTM/concat)]]`\r\n\r\n\r\n\r\nCould you please help me fix this issue?"}}
{"actor":{"display_login":"parmpreet"}}