title: What is the structure of a Keras model if input_shape is omitted and why does it perform better?
body: <p>I omitted the <code>input_shape</code> in the first layer of my Keras model by mistake. Eventually I noticed this and fixed it – and my model's performance dropped dramatically.</p>
  
  <p>Looking at the structure of the model with and without <code>input_shape</code>, I discovered that the better-performing model has the output shape of <code>multiple</code>. Moreover, plotting it with <code>plot_model</code> shows no connections between the layers:</p>
  
  <p><a href="https://i.stack.imgur.com/EUlOR.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/EUlOR.png" alt="enter image description here"></a></p>
  
  <p>When it comes to performance, the model I understand (with input_shape) achieves a validation loss of 4.0513 (MSE) after 10 epochs with my test code (below), while the "weird" model manages 1.3218 – and the difference only increases with more epochs.</p>
  
  <p>Model definition:</p>
  
  <pre><code>model = keras.Sequential()
  model.add(keras.layers.Dense(64, activation=tf.nn.relu, input_shape=(1001,)))
  #                                   add or remove this  ^^^^^^^^^^^^^^^^^^^
  model.add(keras.layers.Dropout(0.05))
  ...
  </code></pre>
  
  <p>(never mind the details, this is just a model that demonstrates the difference in performance with and without input_shape)</p>
  
  <p><strong>So what is happening in the better-performing model?</strong> What is <code>multiple</code>? How are the layers really connected? How could I build this same model while also specifying <code>input_shape</code>?</p>
  
  <p>Complete script:</p>
  
  <pre><code>import tensorflow as tf
  from tensorflow import keras
  import numpy as np
  from collections import deque
  import math, random
  
  def func(x):
      return math.sin(x)*5 + math.sin(x*1.8)*4 + math.sin(x/4)*5
  
  def get_data():
      x = 0
      dx = 0.1
      q = deque()
      r = 0
      data = np.zeros((100000, 1002), np.float32)
      while True:
          x = x + dx
          sig = func(x)
          q.append(sig)
          if len(q) &lt; 1000:
              continue
  
          arr = np.array(q, np.float32)
  
          for k in range(10):
              xx = random.uniform(0.1, 9.9)
              data[r, :1000] = arr[:1000]
              data[r, 1000] = 5*xx #scale for easier fitting
              data[r, 1001] = func(x + xx)
              r = r + 1
              if r &gt;= data.shape[0]:
                  break
  
          if r &gt;= data.shape[0]:
              break
  
          q.popleft()
  
      inputs = data[:, :1001]
      outputs = data[:, 1001]
      return (inputs, outputs)
  
  np.random.seed(1)
  tf.set_random_seed(1)
  random.seed(1)
  
  model = keras.Sequential()
  model.add(keras.layers.Dense(64, activation=tf.nn.relu, input_shape=(1001,)))
  #                                   add or remove this  ^^^^^^^^^^^^^^^^^^^
  model.add(keras.layers.Dropout(0.05))
  model.add(keras.layers.Dense(64, activation=tf.nn.relu))
  model.add(keras.layers.Dropout(0.05))
  model.add(keras.layers.Dense(64, activation=tf.nn.relu))
  model.add(keras.layers.Dropout(0.05))
  model.add(keras.layers.Dense(64, activation=tf.nn.relu))
  model.add(keras.layers.Dropout(0.05))
  model.add(keras.layers.Dense(1))
  
  model.compile(
      loss = 'mse',
      optimizer = tf.train.RMSPropOptimizer(0.0005),
      metrics = ['mae', 'mse'])
  
  inputs, outputs = get_data()
  
  hist = model.fit(inputs, outputs, epochs=10, validation_split=0.1)
  
  print("Final val_loss is", hist.history['val_loss'][-1])
  </code></pre>