original_docstring: 
          Train the experiment's agent for a certain number of episodes per training/validation volley and a certain
          number of volleys. Training and validation volleys are always run in sequence (training first then validation)
          and in the same amount.
  
          If a restore path is given the model is restored from that (according to the way the agent restores its inner
          model), otherwise a new model is used. Also if restoration fails a new model is used.
  
          This generates a tensorflow session and initialize both environment and agent associated with the experiment.
          After this is run all training/validation results are accessible as properties.
  
          :param logger: the logger used to print the experiment information, warnings and errors
          :param training_volleys_episodes: the number of training episodes per training volley, must be greater than zero
          :param validation_volleys_episodes: the number of validation episodes per validation volley, must be greater than zero
          :param volleys: the number of training/validation volleys, must be greater than zero
          :param episode_length: the max length in steps of each episode, must be greater than zero
          :param render_training: boolean flag to render or not the environment during training volleys execution
          :param render_validation: boolean flag to render or not the environment during validation volleys execution
          :param restore_path: the path of the model to restore is stored, if None or empty a new model is built
          
docstring: Train the experiment's agent for a certain number of episodes per training/validation volley and a certain
  number of volleys. Training and validation volleys are always run in sequence (training first then validation)
  and in the same amount.
  
  If a restore path is given the model is restored from that (according to the way the agent restores its inner
  model), otherwise a new model is used. Also if restoration fails a new model is used.
  
  This generates a tensorflow session and initialize both environment and agent associated with the experiment.
  After this is run all training/validation results are accessible as properties.
docstring_tokens: Train the experiment ' s agent for a certain number of episodes per training / validation volley and a certain number of volleys . Training and validation volleys are always run in sequence ( training first then validation ) and in the same amount . If a restore path is given the model is restored from that ( according to the way the agent restores its inner model ) otherwise a new model is used . Also if restoration fails a new model is used . This generates a tensorflow session and initialize both environment and agent associated with the experiment . After this is run all training / validation results are accessible as properties .
code: def train(self,
                logger: logging.Logger,
                training_volleys_episodes: int, validation_volleys_episodes: int, volleys: int,
                episode_length: int,
                render_training: bool = False, render_validation: bool = False,
                restore_path: str = None):
          """
          Train the experiment's agent for a certain number of episodes per training/validation volley and a certain
          number of volleys. Training and validation volleys are always run in sequence (training first then validation)
          and in the same amount.
  
          If a restore path is given the model is restored from that (according to the way the agent restores its inner
          model), otherwise a new model is used. Also if restoration fails a new model is used.
  
          This generates a tensorflow session and initialize both environment and agent associated with the experiment.
          After this is run all training/validation results are accessible as properties.
  
          :param logger: the logger used to print the experiment information, warnings and errors
          :param training_volleys_episodes: the number of training episodes per training volley, must be greater than zero
          :param validation_volleys_episodes: the number of validation episodes per validation volley, must be greater than zero
          :param volleys: the number of training/validation volleys, must be greater than zero
          :param episode_length: the max length in steps of each episode, must be greater than zero
          :param render_training: boolean flag to render or not the environment during training volleys execution
          :param render_validation: boolean flag to render or not the environment during validation volleys execution
          :param restore_path: the path of the model to restore is stored, if None or empty a new model is built
          """
          # Make sure parameters are valid
          assert(training_volleys_episodes > 0 and validation_volleys_episodes > 0 and volleys > 0 and episode_length > 0)
          # Start experiment training process
          logger.info("Experiment " + self._id + ": training process started")
          # Define the session
          with tensorflow.Session(config=self._tensorflow_config) as session:
              # Initialize the environment and the agent
              logger.info("Initializing the environment...")
              self._environment.initialize(logger, session)
              logger.info("Environment initialized")
              logger.info("Initializing the agent..")
              self._agent.initialize(logger, session)
              logger.info("Agent initialized")
              # Restore the agent if possible
              if restore_path is not None and restore_path:
                  logger.info("Restoring agent graph with checkpoint path: " + restore_path)
                  if self._agent.restore(logger, session, restore_path):
                      logger.info("Agent restored")
              # Execute warm-up if the agents require so
              if self._agent.warmup_steps > 0:
                  # Generate warm-up volley
                  self._warmup_volley = StepVolley(self._environment, self._agent, self._interface, self._parallel,
                                                   self._agent.warmup_steps, episode_length)
                  # Setup and run warm-up volley
                  logger.info("Setup of warm-up volley...")
                  if not self._warmup_volley.setup():
                      logger.error("Warm-up volley failed to setup. Exiting...")
                      return
                  logger.info("Setup of warm-up volley completed")
                  self._warmup_volley.run(logger, session)
                  logger.info("Warm-up volley is completed")
              # Generate training and validation volleys
              self._training_volley = EpisodeVolley(self._environment, self._agent, self._interface, self._parallel,
                                                    EpisodeVolleyType.training, self._plots_path, self._plots_dpi,
                                                    training_volleys_episodes, episode_length)
              self._validation_volley = EpisodeVolley(self._environment, self._agent, self._interface, self._parallel,
                                                      EpisodeVolleyType.validation, self._plots_path, self._plots_dpi,
                                                      validation_volleys_episodes, episode_length)
              # Execute training/validation volleys
              start: int = self._training_validation_volley_counter
              for volley_index in range(start, volleys):
                  # Setup and run training volley
                  logger.info("_____________________________________________________________________________________")
                  logger.info("Setup of training volley " + str(self._training_validation_volley_counter) + "...")
                  if not self._training_volley.setup(self._training_validation_volley_counter,
                                                     self._trained_steps, self._trained_episodes):
                      logger.error("Training volley failed to setup. Exiting...")
                      return
                  logger.info("Setup of training volley " + str(self._training_validation_volley_counter) + " completed")
                  self._training_volley.run(logger, session, render_training)
                  # Update counter of trained steps and episodes
                  self._trained_steps += self._training_volley.steps
                  self._trained_episodes += self._training_volley.episodes
                  # Store training volley statistics
                  self._training_rewards.append(self._training_volley.rewards.copy())
                  self._training_total_rewards.append(self._training_volley.total_rewards.copy())
                  self._training_avg_total_rewards.append(self._training_volley.avg_total_reward)
                  self._training_scaled_rewards.append(self._training_volley.scaled_rewards.copy())
                  self._training_avg_scaled_rewards.append(self._training_volley.avg_scaled_reward)
                  self._training_std_total_rewards.append(self._training_volley.std_total_reward)
                  self._training_std_scaled_rewards.append(self._training_volley.std_scaled_reward)
                  self._training_episode_lengths.append(self._training_volley.episode_lengths.copy())
                  self._training_avg_episode_lengths.append(self._training_volley.avg_episode_length)
                  self._training_avg_actions_durations.append(self._training_volley.avg_action_duration)
                  # Print training volley results
                  logger.info("Training volley " + str(self._training_validation_volley_counter) + " is completed")
                  logger.info("Training steps up to now: " + str(self._trained_steps))
                  # Save/update plots for all training volleys up to now (except at first volley, just one volley is not enough to plot anything)
                  if self._training_validation_volley_counter > 0:
                      if self._training_validation_volley_counter == 0:
                          logger.info("Save all training volleys plots...")
                      else:
                          logger.info("Update all training volleys plots...")
                      self._save_plots(EpisodeVolleyType.training)
                      if self._training_validation_volley_counter == 0:
                          logger.info("Plots saved successfully")
                      else:
                          logger.info("Plots updated successfully")
                  logger.info("_____________________________________________________________________________________")
                  # Setup and run validation volley
                  logger.info("---------------------------------------------------------------------------------------")
                  logger.info("Setup of validation volley " + str(self._training_validation_volley_counter) + "...")
                  if not self._validation_volley.setup(self._training_validation_volley_counter):
                      logger.error("Validation volley failed to setup. Exiting...")
                      return
                  logger.info("Setup of validation volley " + str(self._training_validation_volley_counter) + " completed")
                  self._validation_volley.run(logger, session, render_validation)
                  # Store validation volley statistics
                  self._validation_rewards.append(self._validation_volley.rewards.copy())
                  self._validation_total_rewards.append(self._validation_volley.total_rewards.copy())
                  self._validation_avg_total_rewards.append(self._validation_volley.avg_total_reward)
                  self._validation_scaled_rewards.append(self._validation_volley.scaled_rewards.copy())
                  self._validation_avg_scaled_rewards.append(self._validation_volley.avg_scaled_reward)
                  self._validation_std_total_rewards.append(self._validation_volley.std_total_reward)
                  self._validation_std_scaled_rewards.append(self._validation_volley.std_scaled_reward)
                  self._validation_episode_lengths.append(self._validation_volley.episode_lengths.copy())
                  self._validation_avg_episode_lengths.append(self._validation_volley.avg_episode_length)
                  self._validation_avg_actions_durations.append(self._validation_volley.avg_action_duration)
                  # Print validation volley results
                  logger.info("Validation volley " + str(self._training_validation_volley_counter) + " is completed")
                  # Save/update plots for all training volleys up to now (except at first volley, just one volley is not enough to plot anything)
                  if self._training_validation_volley_counter > 0:
                      if self._training_validation_volley_counter == 0:
                          logger.info("Save all validation volleys plots...")
                      else:
                          logger.info("Update all validation volleys plots...")
                      self._save_plots(EpisodeVolleyType.validation)
                      if self._training_validation_volley_counter == 1:
                          logger.info("Plots saved successfully")
                      else:
                          logger.info("Plots updated successfully")
                  logger.info("---------------------------------------------------------------------------------------")
                  # Save the agent metagraph
                  self._agent.save(logger, session)
                  # Update the training/validation volley counter
                  self._training_validation_volley_counter += 1
                  # Check for validation
                  self._validated = self._is_validated(logger)
                  if self._validated:
                      logger.info("Validation of the agent is successful")
                      break
              logger.info("End of training")
code_tokens: def train ( self , logger : logging . Logger , training_volleys_episodes : int , validation_volleys_episodes : int , volleys : int , episode_length : int , render_training : bool = False , render_validation : bool = False , restore_path : str = None ) : assert ( training_volleys_episodes > 0 and validation_volleys_episodes > 0 and volleys > 0 and episode_length > 0 ) logger . info ( "Experiment " + self . _id + ": training process started" ) with tensorflow . Session ( config = self . _tensorflow_config ) as session : logger . info ( "Initializing the environment..." ) self . _environment . initialize ( logger , session ) logger . info ( "Environment initialized" ) logger . info ( "Initializing the agent.." ) self . _agent . initialize ( logger , session ) logger . info ( "Agent initialized" ) if restore_path is not None and restore_path : logger . info ( "Restoring agent graph with checkpoint path: " + restore_path ) if self . _agent . restore ( logger , session , restore_path ) : logger . info ( "Agent restored" ) if self . _agent . warmup_steps > 0 : self . _warmup_volley = StepVolley ( self . _environment , self . _agent , self . _interface , self . _parallel , self . _agent . warmup_steps , episode_length ) logger . info ( "Setup of warm-up volley..." ) if not self . _warmup_volley . setup ( ) : logger . error ( "Warm-up volley failed to setup. Exiting..." ) return logger . info ( "Setup of warm-up volley completed" ) self . _warmup_volley . run ( logger , session ) logger . info ( "Warm-up volley is completed" ) self . _training_volley = EpisodeVolley ( self . _environment , self . _agent , self . _interface , self . _parallel , EpisodeVolleyType . training , self . _plots_path , self . _plots_dpi , training_volleys_episodes , episode_length ) self . _validation_volley = EpisodeVolley ( self . _environment , self . _agent , self . _interface , self . _parallel , EpisodeVolleyType . validation , self . _plots_path , self . _plots_dpi , validation_volleys_episodes , episode_length ) start : int = self . _training_validation_volley_counter for volley_index in range ( start , volleys ) : logger . info ( "_____________________________________________________________________________________" ) logger . info ( "Setup of training volley " + str ( self . _training_validation_volley_counter ) + "..." ) if not self . _training_volley . setup ( self . _training_validation_volley_counter , self . _trained_steps , self . _trained_episodes ) : logger . error ( "Training volley failed to setup. Exiting..." ) return logger . info ( "Setup of training volley " + str ( self . _training_validation_volley_counter ) + " completed" ) self . _training_volley . run ( logger , session , render_training ) self . _trained_steps += self . _training_volley . steps self . _trained_episodes += self . _training_volley . episodes self . _training_rewards . append ( self . _training_volley . rewards . copy ( ) ) self . _training_total_rewards . append ( self . _training_volley . total_rewards . copy ( ) ) self . _training_avg_total_rewards . append ( self . _training_volley . avg_total_reward ) self . _training_scaled_rewards . append ( self . _training_volley . scaled_rewards . copy ( ) ) self . _training_avg_scaled_rewards . append ( self . _training_volley . avg_scaled_reward ) self . _training_std_total_rewards . append ( self . _training_volley . std_total_reward ) self . _training_std_scaled_rewards . append ( self . _training_volley . std_scaled_reward ) self . _training_episode_lengths . append ( self . _training_volley . episode_lengths . copy ( ) ) self . _training_avg_episode_lengths . append ( self . _training_volley . avg_episode_length ) self . _training_avg_actions_durations . append ( self . _training_volley . avg_action_duration ) logger . info ( "Training volley " + str ( self . _training_validation_volley_counter ) + " is completed" ) logger . info ( "Training steps up to now: " + str ( self . _trained_steps ) ) if self . _training_validation_volley_counter > 0 : if self . _training_validation_volley_counter == 0 : logger . info ( "Save all training volleys plots..." ) else : logger . info ( "Update all training volleys plots..." ) self . _save_plots ( EpisodeVolleyType . training ) if self . _training_validation_volley_counter == 0 : logger . info ( "Plots saved successfully" ) else : logger . info ( "Plots updated successfully" ) logger . info ( "_____________________________________________________________________________________" ) logger . info ( "---------------------------------------------------------------------------------------" ) logger . info ( "Setup of validation volley " + str ( self . _training_validation_volley_counter ) + "..." ) if not self . _validation_volley . setup ( self . _training_validation_volley_counter ) : logger . error ( "Validation volley failed to setup. Exiting..." ) return logger . info ( "Setup of validation volley " + str ( self . _training_validation_volley_counter ) + " completed" ) self . _validation_volley . run ( logger , session , render_validation ) self . _validation_rewards . append ( self . _validation_volley . rewards . copy ( ) ) self . _validation_total_rewards . append ( self . _validation_volley . total_rewards . copy ( ) ) self . _validation_avg_total_rewards . append ( self . _validation_volley . avg_total_reward ) self . _validation_scaled_rewards . append ( self . _validation_volley . scaled_rewards . copy ( ) ) self . _validation_avg_scaled_rewards . append ( self . _validation_volley . avg_scaled_reward ) self . _validation_std_total_rewards . append ( self . _validation_volley . std_total_reward ) self . _validation_std_scaled_rewards . append ( self . _validation_volley . std_scaled_reward ) self . _validation_episode_lengths . append ( self . _validation_volley . episode_lengths . copy ( ) ) self . _validation_avg_episode_lengths . append ( self . _validation_volley . avg_episode_length ) self . _validation_avg_actions_durations . append ( self . _validation_volley . avg_action_duration ) logger . info ( "Validation volley " + str ( self . _training_validation_volley_counter ) + " is completed" ) if self . _training_validation_volley_counter > 0 : if self . _training_validation_volley_counter == 0 : logger . info ( "Save all validation volleys plots..." ) else : logger . info ( "Update all validation volleys plots..." ) self . _save_plots ( EpisodeVolleyType . validation ) if self . _training_validation_volley_counter == 1 : logger . info ( "Plots saved successfully" ) else : logger . info ( "Plots updated successfully" ) logger . info ( "---------------------------------------------------------------------------------------" ) self . _agent . save ( logger , session ) self . _training_validation_volley_counter += 1 self . _validated = self . _is_validated ( logger ) if self . _validated : logger . info ( "Validation of the agent is successful" ) break logger . info ( "End of training" )
short_docstring: Train the experiment's agent for a certain number of episodes per training/validation volley and a certain
  number of volleys.
short_docstring_tokens: Train the experiment ' s agent for a certain number of episodes per training / validation volley and a certain number of volleys .
comment: """
          Train the experiment's agent for a certain number of episodes per training/validation volley and a certain
          number of volleys. Training and validation volleys are always run in sequence (training first then validation)
          and in the same amount.
  
          If a restore path is given the model is restored from that (according to the way the agent restores its inner
          model), otherwise a new model is used. Also if restoration fails a new model is used.
  
          This generates a tensorflow session and initialize both environment and agent associated with the experiment.
          After this is run all training/validation results are accessible as properties.
  
          :param logger: the logger used to print the experiment information, warnings and errors
          :param training_volleys_episodes: the number of training episodes per training volley, must be greater than zero
          :param validation_volleys_episodes: the number of validation episodes per validation volley, must be greater than zero
          :param volleys: the number of training/validation volleys, must be greater than zero
          :param episode_length: the max length in steps of each episode, must be greater than zero
          :param render_training: boolean flag to render or not the environment during training volleys execution
          :param render_validation: boolean flag to render or not the environment during validation volleys execution
          :param restore_path: the path of the model to restore is stored, if None or empty a new model is built
          """$SEP$# Make sure parameters are valid$SEP$# Start experiment training process$SEP$# Define the session$SEP$# Initialize the environment and the agent$SEP$# Restore the agent if possible$SEP$# Execute warm-up if the agents require so$SEP$# Generate warm-up volley$SEP$# Setup and run warm-up volley$SEP$# Generate training and validation volleys$SEP$# Execute training/validation volleys$SEP$# Setup and run training volley$SEP$# Update counter of trained steps and episodes$SEP$# Store training volley statistics$SEP$# Print training volley results$SEP$# Save/update plots for all training volleys up to now (except at first volley, just one volley is not enough to plot anything)$SEP$# Setup and run validation volley$SEP$# Store validation volley statistics$SEP$# Print validation volley results$SEP$# Save/update plots for all training volleys up to now (except at first volley, just one volley is not enough to plot anything)$SEP$# Save the agent metagraph$SEP$# Update the training/validation volley counter$SEP$# Check for validation