title: Tic-tac-toe ANN player with TensorFlow learn?
body: <p>I am experimenting with ANNs as tic-tac-toe players using <a href="http://terrytangyuan.github.io/2016/03/14/scikit-flow-intro/" rel="nofollow">TensorFlow learn</a> (formerly Scikit Flow)</p>
  
  <p>The simulator code is <a href="https://bitbucket.org/bachi76/tic-tac-toe" rel="nofollow">here</a>. My current simple nn-based player scores around 90% vs. a random player, but cannot defend at all, as I don't know how to feed into the NN what it should <em>not</em> do (wrong moves if the opponent has two in a row). Also, it only trains on the last board state and winning move.</p>
  
  <p>Questions: </p>
  
  <ol>
  <li><p>What would be a better approach to solve this problem (using an ANN and zero game rule knowledge)? </p></li>
  <li><p>I assume an RNN/LSTM NN would help. There's a short example on the TensorFlow Learn site:</p></li>
  </ol>
  
  <p><code>
  classifier = skflow.TensorFlowRNNClassifier(rnn_size=EMBEDDING_SIZE, 
      n_classes=15, cell_type='gru', input_op_fn=input_op_fn,
      num_layers=1, bidirectional=False, sequence_length=None,
      steps=1000, optimizer='Adam', learning_rate=0.01, continue_training=True)
  </code></p>
  
  <p>But I'm at a loss here - how does this work for the tic-tac-toe example? What would the <code>rnn_size=EMBEDDING_SIZE</code> and <code>input_op_fn params</code> be?</p>
  
  <p>Edit: The training code looks like this (<a href="https://bitbucket.org/bachi76/tic-tac-toe/src/5d901d3886983ea0f3ac7f91e02290d092c64ab6/AIPlayer.py?fileviewer=file-view-default" rel="nofollow">Player source</a>):</p>
  
  <pre><code>def train(self, history):
  
      X = np.zeros((len(history), 18))
      y = np.zeros((len(history), 1))
      i = 0
  
      for game in history:
  
          # Train only on wins of player A (+1 values)
          if game.state == game.WIN_P1:
  
              # Get the second last board states (that lead to the winning move)
              X[i] = np.concatenate([
                  game.history[-2],
                  game.history[-3]
              ]).flatten()
  
              y[i] = game.last_move
  
          # TODO: How can we train lost games with a DNN classifier?
          # if game.state == game.WIN_PMINUS1:
  
          i += 1
  
      self.classifier.fit(X, y)
  </code></pre>
  
  <p>So a row in X contains the last two game states (3x3 nparrays containing 0, 1, -1 for empty, player 1, player 2 respectively), and y contains the last move (that lead to the victory).</p>