title: Keras Neural Network model for 'nim' game
body: <p>The nim game is a game where there is 21 matches and 2 oppenents,</p>
  <p>one after the other, we take up 1 to 3 matches.
  the goal is to make the other oppenent take the last matche.</p>
  <p>This problem seems not to hard to solve with a Q Table method.</p>
  <p>I would like to try it with a Neural Network (i'ld like to make Deep Q Learning after this).</p>
  <p>At the moment, I made a program which simulate 10.000 games and store every moves in an array,
  if we loose the game, we deletes the moves from the array, if we win the game, we keep the moves we did in the game.</p>
  <p>So, I've got 2 arrays:</p>
  <ul>
  <li>The first one is an array of the matches numbers there was before we make a move (between 21 and 2)</li>
  <li>The other one is an array which contains the number of matches we took and led us to the victory of the game (it's should be between 1 and 3, but i normalized it between 0 and 1, dividing it by 3).</li>
  </ul>
  <p>Here's all my Code:</p>
  <pre><code>def game():
  
  remaining = 21
  history=[]
  playerPlaying = random.randint(0,1)
  while remaining &gt; 0:
      move = random.randint(1,3)
      remaining-= move
      if playerPlaying:
          history= history+[(remaining+move,move)]
      playerPlaying = not playerPlaying
  #return loosing player
  return (not playerPlaying, history)
          
  xs=[]
  ys=[]
  for i in range(10000):
      res = game()
      if not res[0] == True:
          for e in res[1]:
              xs+=[e[0]]
              ys+=[e[1]/3]
              
  xs=np.array([xs])
  ys=np.array([ys])
              
          
  model = Sequential()
  model.add(Dense(8, input_dim=1, activation='relu'))
  model.add(Dense(8, activation='relu'))
  model.add(Dense(8, activation='relu'))
  model.add(Dense(8, activation='relu'))
  model.add(Dense(1, activation='sigmoid'))
  
  model.compile(loss='mean_squared_error',
                optimizer='adam',
                metrics=['accuracy'])
  
  
  model.fit(xs,ys, epochs=1000, verbose=2)
  </code></pre>
  <p>print (model.predict([2,3,4,5,6,7]))</p>
  <p>My problem is that this model doesn't seems to improve. it's so far from a good result and give this shape of training output :</p>
  <pre><code>Epoch 996/1000
  807/807 - 1s - loss: 0.0708 - accuracy: 0.3222
  Epoch 997/1000
  807/807 - 1s - loss: 0.0708 - accuracy: 0.3222
  Epoch 998/1000
  807/807 - 1s - loss: 0.0708 - accuracy: 0.3222
  Epoch 999/1000
  807/807 - 1s - loss: 0.0708 - accuracy: 0.3222
  Epoch 1000/1000
  807/807 - 1s - loss: 0.0708 - accuracy: 0.3222
  </code></pre>
  <p>Is there a big mistake i did ? i don't understand why my Xs and Yx does not make the model learn.</p>
  <p>I just have the basics about Deep Learning. I'm not really familiar yet with the optimizers and the way to design Hidden layers in a NN.</p>