title: How to implement a neural network to play an endless runner game?
body: <p>After learning the basics of neural networks I want to create a small but an unique project (something more difficult than handwritten digits etc). 
  I chose to implement an endless runner game, record my moves and train a neural network with the collected data.</p>
  
  <p><strong>Game looks like this:</strong></p>
  
  <p><a href="https://i.stack.imgur.com/jE3dx.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/jE3dx.png" alt="enter image description here"></a></p>
  
  <p>Player can move to right and to left (arrows) , the goal is to avoid asteroids. 
  When player presses left arrow, coordinates of all asteroids, player position and left move are recorded.
  When player presses right arrow, the same thing happens but "move" is set to right. 
  I had a problem with figuring out how to record no movement so I coded it to record no movement (straight) when i press up arrow. So when I wasn't moving, i was pressing up arrow (at a medium pace)</p>
  
  <p>There can be maximum of 20 asteroids but I collected data for a smaller data set, to see how it works.</p>
  
  <p><strong>Collected data looks like this:</strong></p>
  
  <p><a href="https://i.stack.imgur.com/dgbQG.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/dgbQG.png" alt="enter image description here"></a></p>
  
  <p>As you can see in the first row, there are 42 columns.
  20 columns for x-coordinates of all asteroids on the board.
  20 columns for y-coordinates of all asteroids on the board. 
  1 column for x-coordinate of a player
  1 column for a move (0 - left; 1 - right; 2 - straight, no movement)</p>
  
  <p>I collected 4042 samples:
  1140 left
  1122 right
  1780 straight</p>
  
  <p>As I said before, I prepared it to handle 20 asteroids but I collected data for a only of few of them so the rest of the columns are filled with zeros.</p>
  
  <p><strong>I used tensorflow to train a neural network</strong></p>
  
  <pre><code>data = pd.read_csv('collected_data.csv')
  y = data['move']
  x = data.drop('move', axis=1)
  x_train = x.to_numpy()
  y_train = y.to_numpy()
  
  x_train = tf.keras.utils.normalize(x_train, axis=1)
  
  model = tf.keras.Sequential([
      tf.keras.layers.Dense(41,activation='hard_sigmoid',input_shape=x_train[0].shape),
      tf.keras.layers.Dense(41, activation='hard_sigmoid'),
      tf.keras.layers.Dense(41, activation='hard_sigmoid'),
      tf.keras.layers.Dense(3,activation='hard_sigmoid')
  ])
  model.compile(optimizer='adam',
                loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True), metrics=['accuracy'])
  
  model.fit(x_train, y_train, epochs=10)
  
  model.save('model.h5')
  </code></pre>
  
  <p><strong>Result:</strong></p>
  
  <p><a href="https://i.stack.imgur.com/8bR2J.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/8bR2J.png" alt="enter image description here"></a></p>
  
  <p>My model doesn't fit at all. When i run the game, spaceship is not moving at all, always chosing to go straight. I tried different neural network architectures, more layers, fewer layers etc. but it almost doesn't change the result. Do I have not enough data ? Data is too complex? Did I choose the wrong model? 
  I will be grateful for any advice. Thanks!</p>