title: Hyperparameter Optimization in Tensorflow
body: <p>I am doing hyperparameter optimization using Bayesian Optimization in Tensorflow for my Convolutional Neural Network (CNN). And I am getting this error:</p>
  
  <blockquote>
    <p>ResourceExhaustedError (see above for traceback): OOM when allocating
    tensor with shape[4136,1,180,432] and type float on
    /job:localhost/replica:0/task:0/device:GPU:0 by allocator GPU_0_bfc</p>
  </blockquote>
  
  <p>I optimized these hyperparameters:</p>
  
  <pre><code>dim_batch_size = Integer(low=1, high=50, name='batch_size')
  dim_kernel_size1 = Integer(low=1, high=75, name='kernel_size1')
  dim_kernel_size2 = Integer(low=1, high=50, name='kernel_size2')
  dim_depth = Integer(low=1, high=100, name='depth')
  dim_num_hidden = Integer(low=5, high=1500, name='num_hidden')
  dim_num_dense_layers = Integer(low=1, high=5, name='num_dense_layers')
  dim_learning_rate = Real(low=1e-6, high=1e-2, prior='log-uniform',
                           name='learning_rate')
  dim_activation = Categorical(categories=['relu', 'sigmoid'],
                               name='activation')
  dim_max_pool = Integer(low=1, high=100, name='max_pool')
  
  dimensions = [dim_batch_size,
                dim_kernel_size1,
                dim_kernel_size2,
                dim_depth,
                dim_num_hidden,
                dim_num_dense_layers,
                dim_learning_rate,
                dim_activation,
                dim_max_pool]
  </code></pre>
  
  <p>It says resource is exhausted. Why is this?</p>
  
  <p>Is it because I optimized too many hyperparameters? Or there is some dimension mismatch? Or did I assign a hyperparameter range that is beyond the allowed range for correct operation?</p>