title: Rendering multiple agents in the same Gym environment using NEAT algorithm
body: <p>Is it possible to render multiple agents in the same gym environment to visualize the physical training process when using the NEAT algorithm? E.g. when training the cart and pole I would like to see all the generations moving in different directions in the same environment/rendering window.</p>
  <p>Below is my example code which renders every agent but in separate windows.</p>
  <pre><code>    import multiprocessing
      import os
      import pickle
      
      import neat
      import numpy as np
      import visualize
      import gym
      
      runs_per_net = 2
      
      # Use the NN network phenotype and the discrete actuator force function.
      def eval_genome(genome, config):
          net = neat.nn.FeedForwardNetwork.create(genome, config)
      
          fitnesses = []
      
          for runs in range(runs_per_net):
              env = gym.make(&quot;CartPole-v1&quot;)
              observation= env.reset()
        
              # Run the given simulation for up to num_steps time steps.
              fitness = 0.0
              done = False
              while not done:
      
                  action = np.argmax(net.activate(observation))
                  
                  observation, reward, done, info = env.step(action)
                  
                  env.render() # renders each Cart and pole agent of each generation but in separate windows/environments
                  
                  fitness += reward
      
              fitnesses.append(fitness)
      
          return np.mean(fitnesses)
  
      def eval_genomes(genomes, config):
      for genome_id, genome in genomes:
          genome.fitness = eval_genome(genome, config)
  
  
  def run():
      # Load the config file, which is assumed to live in
      # the same directory as this script.
      local_dir = os.path.dirname(__file__)
      config_path = os.path.join(local_dir, 'config-feedforward.txt')
      config = neat.Config(neat.DefaultGenome, neat.DefaultReproduction,
                           neat.DefaultSpeciesSet, neat.DefaultStagnation,
                           config_path)
  
      pop = neat.Population(config)
      stats = neat.StatisticsReporter()
      pop.add_reporter(stats)
      pop.add_reporter(neat.StdOutReporter(True))
  
      pe = neat.ParallelEvaluator(multiprocessing.cpu_count(), eval_genome)
      winner = pop.run(pe.evaluate)
  
      # Save the winner.
      with open('winner-feedforward', 'wb') as f:
          pickle.dump(winner, f)
  
      print(winner)
  
  if __name__ == '__main__':
      run()
  </code></pre>
  <p>This is purely for visualization of how generations physically are performing when using NEAT algorithm on gym environments.</p>