title: How to decide the size of layers in Keras' Dense method?
body: <p>Below is the simple example of multi-class classification task with 
  IRIS data. </p>
  
  <pre><code>import seaborn as sns
  import numpy as np
  from sklearn.cross_validation import train_test_split
  from keras.models import Sequential
  from keras.layers.core import Dense, Activation, Dropout
  from keras.regularizers import l2
  from keras.utils import np_utils
  
  
  #np.random.seed(1335)
  
  # Prepare data
  iris = sns.load_dataset("iris")
  iris.head()
  X = iris.values[:, 0:4]
  y = iris.values[:, 4]
  
  
  # Make test and train set
  train_X, test_X, train_y, test_y = train_test_split(X, y, train_size=0.5, random_state=0)
  
  
  ################################
  # Evaluate Keras Neural Network
  ################################
  
  
  # Make ONE-HOT
  def one_hot_encode_object_array(arr):
      '''One hot encode a numpy array of objects (e.g. strings)'''
      uniques, ids = np.unique(arr, return_inverse=True)
      return np_utils.to_categorical(ids, len(uniques))
  
  train_y_ohe = one_hot_encode_object_array(train_y)
  test_y_ohe = one_hot_encode_object_array(test_y)
  
  
  model = Sequential()
  model.add(Dense(16, input_shape=(4,),
        activation="tanh",
        W_regularizer=l2(0.001)))
  model.add(Dropout(0.5))
  model.add(Dense(3, activation='sigmoid'))
  model.compile(loss='categorical_crossentropy', metrics=['accuracy'], optimizer='adam')
  
  
  # Actual modelling
  # If you increase the epoch the accuracy will increase until it drop at
  # certain point. Epoch 50 accuracy 0.99, and after that drop to 0.977, with
  # epoch 70 
  hist = model.fit(train_X, train_y_ohe, verbose=0,   nb_epoch=100,  batch_size=1)
  
  
  score, accuracy = model.evaluate(test_X, test_y_ohe, batch_size=16, verbose=0)
  print("Test fraction correct (NN-Score) = {:.2f}".format(score))
  print("Test fraction correct (NN-Accuracy) = {:.2f}".format(accuracy))
  </code></pre>
  
  <p>My question is how do people usually decide the size of layers?
  For example based on code above we have:</p>
  
  <pre><code>model.add(Dense(16, input_shape=(4,),
        activation="tanh",
        W_regularizer=l2(0.001)))
  model.add(Dense(3, activation='sigmoid'))
  </code></pre>
  
  <p>Where first parameter of <code>Dense</code> is 16 and second is 3.</p>
  
  <ul>
  <li>Why two layers uses two different values for Dense? </li>
  <li>How do we choose what's the best value for Dense?</li>
  </ul>