title: Keras hanging during training when paired with celery
body: <p>I have code that trains a neural network using Keras.</p>
  
  <p>First, I run it through a grid search-like algorithm to get the best parameters and then I use the best parameters to do the real predictions.</p>
  
  <p>The code works completely fine on its own; it's only after I run it with celery (Django backend) that I start to have the problem stated in the title.</p>
  
  <p>To clarify, training works completely fine during the grid search, but when it's done and runs again on the best parameters, it just hangs on 'Epoch 1/1'.</p>
  
  <p>Upon research, I read that I need to keep the keras imports to one area, as it doesn't work with multiprocessing. However, I made sure that was the case and even tried putting the keras imports inside the function that trains it, and still having the same problem.</p>
  
  <p>EDIT: I tried the following and still having the same problem...</p>
  
  <pre><code>@worker_process_init.connect()
  def init_worker_process(**kwargs):
  
      import tensorflow as tf
      session_conf = tf.ConfigProto(
         intra_op_parallelism_threads=1,
         inter_op_parallelism_threads=1
      )
  
      import keras
      from keras import backend as K
      K.clear_session()
      tf.reset_default_graph()
      tf.set_random_seed(7)
      sess = tf.Session(graph=tf.get_default_graph(), config=session_conf)
      K.set_session(sess)
  
      from keras.models import Sequential
      from keras.layers import Dense, Dropout, LSTM, GRU
      from keras.callbacks import EarlyStopping
      from keras.optimizers import Adam, Nadam, Adamax, RMSprop
      from keras import regularizers
      from keras.initializers import glorot_uniform, Orthogonal
  
  @shared_task
  def myfunc(inputs...):
      #...call function that performs everything, including training with keras
  </code></pre>
  
  <p>My question is: how do I get Keras to work properly with celery?</p>