<p>Your first question:<br>
Yes, <code>input_shape</code> must be specified in the first layer, which is ,in your case, the <code>Flatten</code> layer. It is because different #param (weights &amp; bias) needed to be initialised after compiling the model. In you case, the #pram of your first <code>Dense</code> layer will depend on the <code>input_shape</code> you specified in <code>Flatten</code> layer.  </p>

<p>Second question:<br>
If the <code>Desnse</code> layer is the last layer of your model, the <code>unit</code> should clearly be #classes / #outputs you want the model to perform. However, when it comes to <code>hidden layers</code>, as far as I know, there is no such a universal rule/formula to guarantee an optimal number of <code>unit</code>. It really depends on the data you feed into the model &amp; the complexity of the task &amp; etc... I would say it should be chosen on trial and error basis.  </p>

<p>Edit:<br>
Here I found some info for your second question if you really cannot be satisfied by my answer.<br>
<a href="https://arxiv.org/abs/1905.11946" rel="nofollow noreferrer">EfficientNet paper</a>: but it is for Convolutional Neural Networks<br>
<a href="https://www.r-bloggers.com/selecting-the-number-of-neurons-in-the-hidden-layer-of-a-neural-network/" rel="nofollow noreferrer">"Rlue of thumb"</a>: ...</p>
Knowing what to put into a Keras Dense layer
<p>I'm new to Keras and am doing a basic Kaggle tutorial (<a href="https://www.kaggle.com/c/digit-recognizer" rel="nofollow noreferrer">The Digit Recognizer</a>). I am struggling to understand what to actually put into a <code>Dense</code> layer. I have found <a href="https://stackoverflow.com/questions/44747343/keras-input-explanation-input-shape-units-batch-size-dim-etc">this</a> post to be very helpful, but my understanding isn't quite there yet.</p>

<p>In my <code>Sequential</code> model, I'm starting off with a <code>Dense</code> layer. But, I see some posts saying that the first layer <em>must</em> have an <code>input_shape</code> whereas I see plenty of Kaggle submissions and other examples that don't adhere to this.</p>

<ol>
<li>Is an <code>input_shape</code> actually required in the first layer? Is it required at all?</li>
<li>A <code>Dense</code> layers first argument is <code>units</code>. For the life of me, I cannot find a solid explanation on what this argument should actually be. Is there some formula to run here based on the <code>shape</code> of your data? Sometimes I see rather large numbers (something like 784) in the first <code>Dense</code> layer whereas in other cases it's small (something like 10). Or, is it a total guess?</li>
</ol>

<p>I understand that there isn't a "this is what you do for this type of data" approach to building a predictive model, but I can't understand how to even take an educated guess at what numbers to plug in here.</p>

<p>Here is my current model:</p>

<pre><code>model = Sequential()
model.add(Flatten())
model.add(Dense(64, activation=tf.nn.relu))
model.add(Dense(10, activation=tf.nn.softmax))
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy', metrics=['accuracy'])
x_val = x_train[:10000]
y_val = labels[:10000]
model.fit(x_train, y_train, epochs=3) # , validation_data=(x_val, y_val))
</code></pre>

<p>My model performs <em>decent</em> (I think) as I only have about 350 misses out of 8400 images. I've got it down to about 220 with adding more layers, changing numbers, using <code>Dropout</code>, etc...</p>

<p>I'd really like to get a better understanding about the best way to understand what numbers I should be plugging in. And also, do I need an <code>input_shape</code>?</p>