title: My Keras Conv1D model doesn't recognize a substring, and instead produces the same value for every input
body: <p>I want to use a Con1D model to categorize DNA sequences, and as a starting point, I'd like to build a model that recognizes a specific sub-sequence in randomly generated DNA sequences.</p>
  
  <p><strong>Test Data</strong></p>
  
  <p>DNA consists of a sequence of one of 4 possible bases, A, T, G, and C.  I represent my DNA in "one hot" encoding -- a sequence consisting of a single base, A, would therefore be represented as <code>[1,0,0,0]</code> -- yielding a matrix with dimensions <code>SAMPLE_CT x SEQ_LENGTH x 4</code>.  For my test set, I generate a number of random sequences, and in a fraction of those sequences, I overwrite a portion of the sequence with a fixed "signal" sequence.  Sequences are scored based on whether the signal is present or not -- 1 for sequences containing the signal, 0 for sequences lacking it:</p>
  
  <pre><code>SIGNAL_LEN = 80
  COUNT, LENGTH, FRAC_PRESENT = 20000, 5000, 0.3
  
  X = np.zeros((COUNT, LENGTH, 4), dtype='uint8')
  Y = np.zeros((COUNT,), dtype='float16')
  
  # random sub-sequence:
  signal = [random.choice([0,1,2,3]) for _ in range(SIGNAL_LEN)]
  
  for seq_i in range(COUNT):
      for idx in range(LENGTH):
          # flip random index to 1
          X[ seq_i, idx, random.choice([0,1,2,3]) ] = 1
  
      if random.random() &lt; FRAC_PRESENT:
          start_pos = random.randint(100, LENGTH - (100 + SIGNAL_LEN))
          for idx in range(SIGNAL_LEN):
              X[ seq_i, start_pos + idx, : ] = 0
              X[ seq_i, start_pos + idx, signal[idx] ] = 1
  
          # a score of 1.0 means present; 0.0 means absent
          Y[ seq_id ] = 1.0
  </code></pre>
  
  <p><strong>Keras Model</strong></p>
  
  <p>I cribbed my model from <a href="https://github.com/fchollet/keras/issues/2483" rel="nofollow noreferrer">examples</a> I found online.  The output is the <code>score</code> described earlier, 1 for present and 0 for absent.</p>
  
  <pre><code>WINDOW = 20
  sample_ct, width, depth = X.shape
  
  # define base model
  def baseline_model():
      model = Sequential()
      model.add( Conv1D(filters=256, kernel_size=20,
                        input_shape=(width, depth),
                        kernel_initializer= 'uniform',
                        activation= 'relu') )
      model.add(Activation('sigmoid'))
      model.add(MaxPooling1D(pool_length=4))
      model.add( Conv1D(filters=128, kernel_size=20) )
      model.add(Activation('sigmoid'))
      model.add(MaxPooling1D(pool_length=4))
      model.add( Conv1D(1, kernel_size=20,
                        kernel_initializer= 'uniform',
                        activation= 'relu') )
      model.add( Flatten() )
      model.add( Dense(128, kernel_initializer='normal', activation='relu') )
      model.add( Dense(1, activation='sigmoid', name='output') )
  
      model.compile(loss='mean_squared_error', optimizer=Adam(lr=1e04))
  
      model.summary()
      return model
  </code></pre>
  
  <p>The <code>model.summary</code> looks like this:</p>
  
  <pre><code>_________________________________________________________________
  Layer (type)                 Output Shape              Param #
  =================================================================
  conv1d_1 (Conv1D)            (None, 4981, 256)         20736
  _________________________________________________________________
  activation_1 (Activation)    (None, 4981, 256)         0
  _________________________________________________________________
  max_pooling1d_1 (MaxPooling1 (None, 1245, 256)         0
  _________________________________________________________________
  conv1d_2 (Conv1D)            (None, 1226, 128)         655488
  _________________________________________________________________
  activation_2 (Activation)    (None, 1226, 128)         0
  _________________________________________________________________
  max_pooling1d_2 (MaxPooling1 (None, 306, 128)          0
  _________________________________________________________________
  conv1d_3 (Conv1D)            (None, 287, 1)            2561
  _________________________________________________________________
  flatten_1 (Flatten)          (None, 287)               0
  _________________________________________________________________
  dense_1 (Dense)              (None, 128)               36864
  _________________________________________________________________
  output (Dense)               (None, 1)                 129
  =================================================================
  Total params: 715,778
  Trainable params: 715,778
  Non-trainable params: 0
  </code></pre>
  
  <p>I then fit my model against half my test data, and predict the other half:</p>
  
  <pre><code>    from numpy.random import rand
      model = baseline_model()
  
      # initialize a random index
      idx = rand(sample_ct) &gt; 0.5
  
      X_train, X_test = X[idx], X[~ idx]
      Y_train, Y_test = Y[idx], Y[~ idx]
  
      model.fit( X_train,
                {'output': Y_train},
                epochs=3)
  
      pred = model.predict( X_test )
  </code></pre>
  
  <p>However, all my predictions are 0, with a MSE of 0.2994 after 3 epochs.  What am I doing wrong?</p>