title: OpenAI/Tensorflow Custom Game Environment Instead of using 'gym.make()'
body: <p><strong>[Introduction]</strong>
  I have a custom made Python game which uses 'w' 's' keys for moving and 'space' key for shooting as inputs. I've found a reinforcement learning algorithm which I would like to try implement into the game.  </p>
  
  <p>However, the RL algorithm uses openAI's atari games as the environment with command 'gym.make (env_name)'. I'm on Windows OS so can't experiment on the code as gym[atari] does not work for me. </p>
  
  <pre><code>class Agent:
      def __init__(self, env_name, training, render=False, use_logging=True):
  
          self.env = gym.make(env_name)
  </code></pre>
  
  <p><strong>[Question]</strong>
  Is there another command I could use instead of 'gym.make()' in this class to implement the RL algorithm to train on my custom made game, or is the only option to make my own gym environment? 
  Would 'pygame.surfarray.array2d()' return something similar to 'gym.make()'?</p>
  
  <p>Please tell me if additional information is required, I'm new to gym and tensorflow so my understanding may be flawed.</p>
  
  <p><strong>[Edit]</strong>
  I made my game using functions, if I was to convert the game into a gym environment, would the only option be to convert the functions to classes? As an example of how my code looks like, here is the game loop: (I can't post the whole code as it's a controlled assessment for end of year grade so would like to avoid any plagiarisation problems)</p>
  
  <pre><code>def game_loop():
  global pause
  x = (display_width * 0.08)
  y = (display_height * 0.2)
  
  x_change = 0
  y_change = 0
  
  blob_speed = 2
  
  velocity = [2, 2]
  
  score = 0
  lives = 3
  
  pos_x = display_width/1.2
  pos_y = display_height/1.2
  
  previous_time = pygame.time.get_ticks()
  previous_time2 = pygame.time.get_ticks()
  
  gameExit = False
  while not gameExit:
      for event in pygame.event.get():#monitors hardware movement/ clicks
          if event.type == pygame.QUIT:
              pygame.quit()
              quit()
  
      pos_x += velocity[0]
      pos_y += velocity[1]
  
      if pos_x + blob_width &gt; display_width or pos_x &lt; 601:
          velocity[0] = -velocity[0]
  
      if pos_y + blob_height &gt; display_height or pos_y &lt; 0:
          velocity[1] = -velocity[1]
  
      for b in range(len(bullets2)):
          bullets2[b][0] -= 6
  
      for bullet in bullets2:
          if bullet[0] &lt; 0:
              bullets2.remove(bullet)
  
  
      current_time2 = pygame.time.get_ticks()
      #ready to fire when 500 ms have passed.
      if current_time2 - previous_time2 &gt; 500:
          previous_time2 = current_time2
          bullets2.append([pos_x+25, pos_y+24])
  
      keys = pygame.key.get_pressed()
  
      for b in range(len(bullets)):
          bullets[b][0] += 6
  
      for bullet in bullets:
          if bullet[0] &gt; 1005:
              bullets.remove(bullet)
  
      if keys[pygame.K_SPACE]:
          current_time = pygame.time.get_ticks()
          #ready to fire when 500 ms have passed.
          if current_time - previous_time &gt; 600:
              previous_time = current_time
              bullets.append([x+25, y+24])
  
  
      if x &lt; 0:
          x = 0
      if keys[pygame.K_a]:
          x_change = -blob_speed
      if x &gt; 401 - blob_width:
          x = 401 - blob_width
      if keys[pygame.K_d]:
          x_change = blob_speed
      if keys[pygame.K_p]:
          pause = True
          paused()
  
  
      if keys[pygame.K_a] and keys[pygame.K_d]:
          x_change = 0
      if not keys[pygame.K_a] and not keys[pygame.K_d]:
          x_change = 0
  
      if y &lt; 0:
          y = 0
      if keys[pygame.K_w]:
          y_change = -blob_speed
      if y &gt; display_height - blob_height:
          y = display_height - blob_height
      if keys[pygame.K_s]:
          y_change = blob_speed
  
  
      if keys[pygame.K_w] and keys[pygame.K_s]:
          y_change = 0
      if not keys[pygame.K_w] and not keys[pygame.K_s]:
          y_change = 0
  
  
      #print(event)
      # Reset x and y to new position
      x += x_change
      y += y_change
  
      gameDisplay.fill(blue)  #changes background surface
      bullets_hit(score)
      player_lives(lives)
      pygame.draw.line(gameDisplay, black, (601, display_height), (601, 0), 3)
      pygame.draw.line(gameDisplay, black, (401, display_height), (401, 0), 3)
      blob(pos_x, pos_y)
      blob(x, y)
  
      for bullet in bullets:
          gameDisplay.blit(bulletpicture, pygame.Rect(bullet[0], bullet[1], 0, 0))
          if bullet[0] &gt; pos_x and bullet[0] &lt; pos_x + blob_width:
              if bullet[1] &gt; pos_y and bullet[1] &lt; pos_y + blob_height or bullet[1] + bullet_height &gt; pos_y and bullet[1] + bullet_height &lt; pos_y + blob_height:
                  bullets.remove(bullet)
                  score+=1
  
      for bullet in bullets2:
          gameDisplay.blit(bulletpicture, pygame.Rect(bullet[0], bullet[1], 0, 0))
          if bullet[0] + bullet_width &lt; x + blob_width and bullet[0] &gt; x:
              if bullet[1] &gt; y and bullet[1] &lt; y + blob_height or bullet[1] + bullet_height &gt; y and bullet[1] + bullet_height &lt; y + blob_height:
                  bullets2.remove(bullet)
                  lives-=1
  
      if lives == 0:
          game_over()
  
  
      pygame.display.update() #update screen
      clock.tick(120)#moves frame on (fps in parameters)
  </code></pre>