title: Multi-dimensional regression with Keras
body: <p>I want to use Keras to train a neural network for 2-dimensional regression.</p>
  <p>My input is a single number, and my output has two numbers:</p>
  <pre><code>model = Sequential()
  model.add(Dense(16, input_shape=(1,), kernel_initializer=initializers.constant(0.0), bias_initializer=initializers.constant(0.0)))
  model.add(Activation('relu'))
  model.add(Dense(16, input_shape=(1,), kernel_initializer=initializers.constant(0.0), bias_initializer=initializers.constant(0.0)))
  model.add(Activation('relu'))
  model.add(Dense(2, kernel_initializer=initializers.constant(0.0), bias_initializer=initializers.constant(0.0)))
  adam = Adam(lr=0.001, beta_1=0.9, beta_2=0.999, epsilon=1e-08, decay=0.0)
  model.compile(loss='mean_squared_error', optimizer=adam)
  </code></pre>
  <p>I then created some dummy data for training:</p>
  <pre><code>inputs = np.zeros((10, 1), dtype=np.float32)
  targets = np.zeros((10, 2), dtype=np.float32)
  
  for i in range(10):
      inputs[i] = i / 10.0
      targets[i, 0] = 0.1
      targets[i, 1] = 0.01 * i
  </code></pre>
  <p>And finally, I trained with minibatches in a loop, whilst testing on the training data:</p>
  <pre><code>while True:
  
      loss = model.train_on_batch(inputs, targets)
  
      test_outputs = model.predict(inputs)
  
      print test_outputs
  </code></pre>
  <p>The problem is, the outputs printed out are as follows:</p>
  <pre><code>[0.1, 0.045]
  [0.1, 0.045]
  [0.1, 0.045]
  .....
  .....
  .....
  </code></pre>
  <p>So, whilst the first dimension is correct (0.1), the second dimension is not correct. The second dimension should be [0.01, 0.02, 0.03, .....]. So in fact, the output from the network (0.45) is simply the average of what all the values in the second dimension should be.</p>
  <p>What am I doing wrong?</p>