title: caffe: "Check failed: status == CUDNN_STATUS_SUCCESS (3 vs. 0) CUDNN_STATUS_BAD_PARAM" during training
body: <p>I am getting into the programming of networks with caffe and since I am used to more comfortable and "lazy" solutions I am a bit overwhelmed by the problems that can occur.</p>
  
  <p>Right now I am getting the error 
  <code>Check failed: status == CUDNN_STATUS_SUCCESS (3 vs. 0)  CUDNN_STATUS_BAD_PARAM</code></p>
  
  <p>This one is quite well known to be produced by bad cuda or cudnn versions.
  So i checked those and they are up to date. (Cuda: 8.0.61 Cudnn: 6.0.21)</p>
  
  <p>Since I will only get this error when I add this ReLU layer I suppose it is caused by me confusing a parameter:</p>
  
  <pre><code>layer{
  name: "relu1"
  type: "ReLU"
  bottom: "pool1"
  top: "relu1"
  }
  </code></pre>
  
  <p>And to give you all the information, here is the error message I get:</p>
  
  <pre><code>I0319 09:41:09.484148  6909 solver.cpp:44] Initializing solver from parameters:
  test_iter: 10
  test_interval: 1000
  base_lr: 0.001
  display: 20
  max_iter: 800
  lr_policy: "step"
  gamma: 0.1
  momentum: 0.9
  weight_decay: 0.04
  stepsize: 200
  snapshot: 10000
  snapshot_prefix: "models/train"
  solver_mode: GPU
  net: "train_val.prototxt"
  I0319 09:41:09.484392  6909 solver.cpp:87] Creating training net from net file: train_val.prototxt
  I0319 09:41:09.485164  6909 net.cpp:294] The NetState phase (0) differed from the phase (1) specified by a rule in layer feed2
  I0319 09:41:09.485183  6909 net.cpp:51] Initializing net from parameters:
  name: "CaffeNet"
  state {
    phase: TRAIN
  }
  layer {
    name: "feed"
    type: "HDF5Data"
    top: "data"
    top: "label"
    include {
      phase: TRAIN
    }
    hdf5_data_param {
      source: "train_h5_list.txt"
      batch_size: 50
    }
  }
  layer {
    name: "conv1"
    type: "Convolution"
    bottom: "data"
    top: "conv1"
    param {
      lr_mult: 1
    }
    param {
      lr_mult: 2
    }
    convolution_param {
      num_output: 1
      kernel_size: 3
      stride: 1
      weight_filler {
        type: "gaussian"
      }
      bias_filler {
        type: "constant"
      }
    }
  }
  layer {
    name: "pool1"
    type: "Pooling"
    bottom: "conv1"
    top: "pool1"
    pooling_param {
      pool: MAX
      kernel_size: 2
      stride: 1
    }
  }
  layer {
    name: "relu1"
    type: "ReLU"
    bottom: "pool1"
    top: "relu1"
  }
  layer {
    name: "conv2"
    type: "Convolution"
    bottom: "relu1"
    top: "conv2"
    param {
      lr_mult: 1
    }
    param {
      lr_mult: 2
    }
    convolution_param {
      num_output: 1
      kernel_size: 3
      stride: 1
      weight_filler {
        type: "gaussian"
      }
      bias_filler {
        type: "constant"
      }
    }
  }
  layer {
    name: "ip2"
    type: "InnerProduct"
    bottom: "conv2"
    top: "ip2"
    param {
      lr_mult: 1
      decay_mult: 1
    }
    inner_product_param {
      num_output: 1
      weight_filler {
        type: "gaussian"
        std: 0.01
      }
      bias_filler {
        type: "constant"
        value: 0
      }
    }
  }
  layer {
    name: "sig1"
    type: "Sigmoid"
    bottom: "ip2"
    top: "sig1"
  }
  layer {
    name: "loss"
    type: "EuclideanLoss"
    bottom: "sig1"
    bottom: "label"
    top: "loss"
  }
  I0319 09:41:09.485752  6909 layer_factory.hpp:77] Creating layer feed
  I0319 09:41:09.485780  6909 net.cpp:84] Creating Layer feed
  I0319 09:41:09.485792  6909 net.cpp:380] feed -&gt; data
  I0319 09:41:09.485819  6909 net.cpp:380] feed -&gt; label
  I0319 09:41:09.485836  6909 hdf5_data_layer.cpp:80] Loading list of HDF5 filenames from: train_h5_list.txt
  I0319 09:41:09.485860  6909 hdf5_data_layer.cpp:94] Number of HDF5 files: 1
  I0319 09:41:09.486469  6909 hdf5.cpp:32] Datatype class: H5T_FLOAT
  I0319 09:41:09.500986  6909 net.cpp:122] Setting up feed
  I0319 09:41:09.501011  6909 net.cpp:129] Top shape: 50 227 227 3 (7729350)
  I0319 09:41:09.501027  6909 net.cpp:129] Top shape: 50 1 (50)
  I0319 09:41:09.501039  6909 net.cpp:137] Memory required for data: 30917600
  I0319 09:41:09.501051  6909 layer_factory.hpp:77] Creating layer conv1
  I0319 09:41:09.501080  6909 net.cpp:84] Creating Layer conv1
  I0319 09:41:09.501087  6909 net.cpp:406] conv1 &lt;- data
  I0319 09:41:09.501101  6909 net.cpp:380] conv1 -&gt; conv1
  I0319 09:41:09.880740  6909 net.cpp:122] Setting up conv1
  I0319 09:41:09.880765  6909 net.cpp:129] Top shape: 50 1 225 1 (11250)
  I0319 09:41:09.880781  6909 net.cpp:137] Memory required for data: 30962600
  I0319 09:41:09.880808  6909 layer_factory.hpp:77] Creating layer pool1
  I0319 09:41:09.880836  6909 net.cpp:84] Creating Layer pool1
  I0319 09:41:09.880846  6909 net.cpp:406] pool1 &lt;- conv1
  I0319 09:41:09.880861  6909 net.cpp:380] pool1 -&gt; pool1
  I0319 09:41:09.880888  6909 net.cpp:122] Setting up pool1
  I0319 09:41:09.880899  6909 net.cpp:129] Top shape: 50 1 224 0 (0)
  I0319 09:41:09.880913  6909 net.cpp:137] Memory required for data: 30962600
  I0319 09:41:09.880921  6909 layer_factory.hpp:77] Creating layer relu1
  I0319 09:41:09.880934  6909 net.cpp:84] Creating Layer relu1
  I0319 09:41:09.880941  6909 net.cpp:406] relu1 &lt;- pool1
  I0319 09:41:09.880952  6909 net.cpp:380] relu1 -&gt; relu1
  F0319 09:41:09.881192  6909 cudnn.hpp:80] Check failed: status == CUDNN_STATUS_SUCCESS (3 vs. 0)  CUDNN_STATUS_BAD_PARAM
  </code></pre>
  
  <p>EDIT: Tried setting the solver mode to CPU, I still get this error. </p>