title: Self Driving Car (Game) using Convolutional Neural Network in Python
body: <p>I am using a convolutional neural network (tensorflow, alexnet) to drive a car in a game (Need For Speed). I'm just a student who wants to work with Machine Learning. I'm a total beginner at this.</p>
  
  <p>Here's what my plan is:</p>
  
  <ol>
  <li>Get training data</li>
  <li>Balance  and shuffle it</li>
  <li>Train a model</li>
  <li>Test a model</li>
  </ol>
  
  <p>The problem is that I am not finding a way to log or detect KeyPresses that I do while playing the game. I would like python to detect which keys I press, and store them in a npy array along with the game's frame image. </p>
  
  <p>I did come across this piece of code which  logs KeyPresses, but only the alphabet keys. I want python to also detect arrow keys, space bar, etc.</p>
  
  <pre><code>import win32api as wapio
  import time
  
  keyList = ["\b"]
  for char in "ABCDEFGHIJKLMNOPQRSTUVWXYZ 123456789,.'£$/\\":
      keyList.append(char)
  
  def key_check():
      keys = []
      for key in keyList:
          if wapi.GetAsyncKeyState(ord(key)):
              keys.append(key)
      return keys
  </code></pre>
  
  <p>I would also like a way to simulate KeyPresses, so that my model could actually drive a car. </p>
  
  <p>I do have this piece of code, which works perfectly fine. I'm just asking for suggestions on this one.</p>
  
  <pre><code>import ctypes
  import time
  
  SendInput = ctypes.windll.user32.SendInput
  
  W = 0x11
  A = 0x1E
  S = 0x1F
  D = 0x20
  
  # C struct redefinitions 
  PUL = ctypes.POINTER(ctypes.c_ulong)
  class KeyBdInput(ctypes.Structure):
      _fields_ = [("wVk", ctypes.c_ushort),
                  ("wScan", ctypes.c_ushort),
                  ("dwFlags", ctypes.c_ulong),
                  ("time", ctypes.c_ulong),
                  ("dwExtraInfo", PUL)]
  
  class HardwareInput(ctypes.Structure):
      _fields_ = [("uMsg", ctypes.c_ulong),
                  ("wParamL", ctypes.c_short),
                  ("wParamH", ctypes.c_ushort)]
  
  class MouseInput(ctypes.Structure):
      _fields_ = [("dx", ctypes.c_long),
                  ("dy", ctypes.c_long),
                  ("mouseData", ctypes.c_ulong),
                  ("dwFlags", ctypes.c_ulong),
                  ("time",ctypes.c_ulong),
                  ("dwExtraInfo", PUL)]
  
  class Input_I(ctypes.Union):
      _fields_ = [("ki", KeyBdInput),
                   ("mi", MouseInput),
                   ("hi", HardwareInput)]
  
  class Input(ctypes.Structure):
      _fields_ = [("type", ctypes.c_ulong),
                  ("ii", Input_I)]
  
  # Actuals Functions
  
  def PressKey(hexKeyCode):
      extra = ctypes.c_ulong(0)
      ii_ = Input_I()
      ii_.ki = KeyBdInput( 0, hexKeyCode, 0x0008, 0, ctypes.pointer(extra) )
      x = Input( ctypes.c_ulong(1), ii_ )
      ctypes.windll.user32.SendInput(1, ctypes.pointer(x), ctypes.sizeof(x))
  
  def ReleaseKey(hexKeyCode):
      extra = ctypes.c_ulong(0)
      ii_ = Input_I()
      ii_.ki = KeyBdInput( 0, hexKeyCode, 0x0008 | 0x0002, 0, 
      ctypes.pointer(extra) )
      x = Input( ctypes.c_ulong(1), ii_ )
      ctypes.windll.user32.SendInput(1, ctypes.pointer(x), ctypes.sizeof(x))
  
  
  if __name__ == '__main__':
      while (True):
          PressKey(0x11)
          time.sleep(1)
          ReleaseKey(0x11)
          time.sleep(1)
  </code></pre>
  
  <p>The next step is to actually collect the frame (image) of the game. I use cv2 to capture a region of my screen, where I play the game. I then:</p>
  
  <ol>
  <li><p>convert it to grayscale</p></li>
  <li><p>resize the image itself</p></li>
  <li><p>save it as a numpy file (for every 1000 frames captured)</p></li>
  </ol>
  
  <p>The data is captured perfectly fine, but once I have around 50 000 frames (or around 500 mb of data), it takes a long time to save the data. Sometimes, Python crashes while the numpy file is saving, resulting in a loss of all my training data.</p>
  
  <p>Here's my code for capturing data:</p>
  
  <pre><code>import numpy as np
  import cv2
  import time
  from grabscreen import grab_screen
  from getkeys import key_check
  import os
  
  
  def keys_to_output(keys):
      output = [0, 0, 0]
  
      if 'A' in keys:
          output[0] = 1
      elif 'D' in keys:
          output[2] = 1
      else:
          output[1] = 1
  
      return output
  
  
  file_name = 'training_data.npy'
  
  if os.path.isfile(file_name):
      print('File exists, loading previous data!')
      training_data = list(np.load(file_name))
  else:
      print('File does not exist, starting fresh!')
      training_data = []
  
  
  def main():
      for i in list(range(5))[::-1]:
          print(i + 1)
          time.sleep(1)
  
      if os.path.isfile(file_name):
          print('Existing Training Data:' + str(len(training_data)))
          print('Capturing Data!')   
      else:
          print('Capturing Data Freshly!') 
  
  
      while True:
          screen = grab_screen(region=(40, 250, 860, 560))
          screen = cv2.resize(screen, (120, 56))
          screen = cv2.cvtColor(screen, cv2.COLOR_BGR2GRAY)
  
          keys = key_check()
          output = keys_to_output(keys)
          training_data.append([screen, output])
  
  
          if len(training_data) % 1000 == 0:
              print('New Training Data: ' + str(len(training_data)))
              print('Saving Data!')
              np.save(file_name, training_data)
              print('Data saved succesfully! You can quit now.')
              print('Capturing data!')
  
  
  main()
  </code></pre>
  
  <p>Can anyone suggest a better way to get my training data. Also, I heard about using PyTables, but I'm not sure of how to use it for my program.</p>
  
  <p>After this, I shuffle my data, and balance it:</p>
  
  <pre><code>import numpy as np
  import pandas as pd
  from collections import Counter
  from random import shuffle
  
  train_data = np.load('training_data.npy')
  
  print('Training Data: ' + str(len(train_data)))
  df = pd.DataFrame(train_data)
  print(df.head())
  print(Counter(df[1].apply(str)))
  
  lefts = []
  rights = []
  forwards = []
  
  shuffle(train_data)
  
  for data in train_data:
      img = data[0]
      choice = data[1]
  
      if choice == [1, 0, 0]:
          lefts.append([img, choice])
      elif choice == [0, 1, 0]:
          forwards.append([img, choice])
      elif choice == [0, 0, 1]:
          rights.append([img, choice])
      else:
          print('no matches!!!')
  
  
  forwards = forwards[:len(lefts)][:len(rights)]
  lefts = lefts[:len(forwards)]
  rights = rights[:len(forwards)]
  
  final_data = forwards + lefts + rights
  
  shuffle(final_data)
  print('Final Balanced Data: ' + str(len(final_data)))
  np.save('training_data_balanced.npy', final_data)
  </code></pre>
  
  <p>Model:</p>
  
  <pre><code>import tflearn
  from tflearn.layers.conv import conv_2d, max_pool_2d
  from tflearn.layers.core import input_data, dropout, fully_connected
  from tflearn.layers.estimator import regression
  from tflearn.layers.normalization import local_response_normalization
  
  def alexnet(width, height, lr):
      network = input_data(shape=[None, width, height, 1], name='input')
      network = conv_2d(network, 96, 11, strides=4, activation='relu')
      network = max_pool_2d(network, 3, strides=2)
      network = local_response_normalization(network)
      network = conv_2d(network, 256, 5, activation='relu')
      network = max_pool_2d(network, 3, strides=2)
      network = local_response_normalization(network)
      network = conv_2d(network, 384, 3, activation='relu')
      network = conv_2d(network, 384, 3, activation='relu')
      network = conv_2d(network, 256, 3, activation='relu')
      network = max_pool_2d(network, 3, strides=2)
      network = local_response_normalization(network)
      network = fully_connected(network, 4096, activation='tanh')
      network = dropout(network, 0.5)
      network = fully_connected(network, 4096, activation='tanh')
      network = dropout(network, 0.5)
      network = fully_connected(network, 3, activation='softmax')
      network = regression(network, optimizer='momentum',
                           loss='categorical_crossentropy',
                           learning_rate=lr, name='targets')
  
      model = tflearn.DNN(network, checkpoint_path='model_alexnet',
                          max_checkpoints=1, tensorboard_verbose=2, tensorboard_dir='log')
  
      return model
  </code></pre>
  
  <p>Training the model:</p>
  
  <pre><code>import numpy as np
  from alexnet import alexnet
  
  WIDTH = 120
  HEIGHT = 56
  LR = 1e-3
  EPOCHS = 15
  MODEL_NAME = 'nfs-car-{}-{}-epochs.model'.format(LR, EPOCHS)
  
  model = alexnet(WIDTH, HEIGHT, LR)
  
  # for every epoch finished, save the model.
  # therefore, if accuracy drops or loss increases, we can terminate the script.
  # we will have a trained model with the best accuracy and previously saved epoch.
  
  for i in range(EPOCHS):
      train_data = np.load('training_data_balanced.npy')
  
      train = train_data[:-10] # 80% of balanced data
      test = train_data[-10:] # 20% of balanced data
  
      X = np.array([i[0] for i in train]).reshape(-1, WIDTH, HEIGHT, 1)
      Y = [i[1] for i in train]
  
      test_x = np.array([i[0] for i in test]).reshape(-1, WIDTH, HEIGHT, 1)
      test_y = [i[1] for i in test]
  
      model.fit({'input': X}, {'targets': Y}, n_epoch=1,
                validation_set=({'input': test_x}, {'targets': test_y}),
                snapshot_step=500, show_metric=True, run_id=MODEL_NAME)
  
      model.save(MODEL_NAME)
      print('Saved epoch: ' + str(i + 1))
  </code></pre>
  
  <p>Then of course, I test the model:</p>
  
  <pre><code>import numpy as np
  import cv2
  import time
  from grabscreen import grab_screen
  from getkeys import key_check
  from alexnet import alexnet
  from directkeys import PressKey, ReleaseKey, W, A, D
  
  WIDTH = 120
  HEIGHT = 56
  LR = 1e-3
  EPOCHS = 15
  MODEL_NAME = 'nfs-car-{}-{}-epochs.model'.format(LR, EPOCHS)
  
  
  def straight():
      PressKey(W)
      ReleaseKey(A)
      ReleaseKey(D)
  
  
  def left():
      PressKey(W)
      PressKey(A)
      ReleaseKey(D) # added
      time.sleep(0.09)
      ReleaseKey(A)
  
  
  def right():
      PressKey(W)
      PressKey(D)
      ReleaseKey(A) # added
      time.sleep(0.09)
      ReleaseKey(D)
  
  
  model = alexnet(WIDTH, HEIGHT, LR)
  model.load(MODEL_NAME)
  
  
  def main():
      for i in list(range(5))[::-1]:
          print(i + 1)
          time.sleep(1)
  
  
      paused = False
  
      while True:
  
          if not paused:
              screen = grab_screen(region=(40, 250, 860, 560))
              screen = cv2.cvtColor(screen, cv2.COLOR_BGR2GRAY)
              screen = cv2.resize(screen, (120, 56))
  
              prediction = model.predict([screen.reshape(WIDTH, HEIGHT, 1)])[0]
              print(prediction)
  
              turn_thrush = .75
              fwd_thrush = 0.70
  
              if prediction[0] &gt; turn_thrush:
                  left()
              elif prediction[1] &gt; fwd_thrush:
                  straight()
              elif prediction[2] &gt; turn_thrush:
                  right()
              else:
                  straight()
  
              keys = key_check()
  
              if 'T' in keys:
                  if paused:
                      paused = False
                      print('Unpaused!')
                      time.sleep(1)
                  else:
                      print('Pausing!')
                      paused = True
                      ReleaseKey(A)
                      ReleaseKey(W)
                      ReleaseKey(D)
                      time.sleep(1)
  
  
  main()
  </code></pre>
  
  <p>So yeah, this is what my idea is. Yet, I'm not able to fully apply this concept due to lack of proper information.</p>