original_docstring: Executes a step in the pipeline, managing generic logging and reuse of intermediate results
  
      Parameters
      ----------
      step_tuple : tuple
          step_name : str
              The name of the step to execute
          experiment_func : function
              Function to run on each experiment. Should take experiment string as argument and return
          batch_func : function
              Function to run on the results of all experiments for this step
          get_preexisting_func : function
              Function to get a preexisting result (if needed for batch) when experiment_func is skipped on an experiment
          use_existing : bool
              Whether this step can skipped using preexisting results
      experiments : list
          The experiments used in this run
      batch_outdir : str
          The path to the batch output directory
      config : dict
          The config for this run
      
docstring: Executes a step in the pipeline, managing generic logging and reuse of intermediate results
  Parameters
  
  step_tuple : tuple
  step_name : str
  The name of the step to execute
  experiment_func : function
  Function to run on each experiment.
docstring_tokens: Executes a step in the pipeline managing generic logging and reuse of intermediate results Parameters step_tuple : tuple step_name : str The name of the step to execute experiment_func : function Function to run on each experiment .
code: def pipeline_run_step(step_tuple, experiments, batch_outdir, config):
      """Executes a step in the pipeline, managing generic logging and reuse of intermediate results
  
      Parameters
      ----------
      step_tuple : tuple
          step_name : str
              The name of the step to execute
          experiment_func : function
              Function to run on each experiment. Should take experiment string as argument and return
          batch_func : function
              Function to run on the results of all experiments for this step
          get_preexisting_func : function
              Function to get a preexisting result (if needed for batch) when experiment_func is skipped on an experiment
          use_existing : bool
              Whether this step can skipped using preexisting results
      experiments : list
          The experiments used in this run
      batch_outdir : str
          The path to the batch output directory
      config : dict
          The config for this run
      """
  
      # Parse step tuple
      step = step_tuple[0]
      experiment_func = step_tuple[1]
      batch_func = step_tuple[2]
      get_preexisting_func = step_tuple[3]
      use_preexisting =  step_tuple[4]
  
      logging.info("\x1b[1mBeginning {} step...\x1b[0m".format(step))  # Bold font
      st = timeit.default_timer()
      outputs = []
      # Run per experiment steps (if any)
      if experiment_func:
          for experiment in experiments:
              # Skip running on an experiment if we can use a pre-existing result
              if not should_run(step, use_preexisting, experiment, config):
                  logging.info("Using cached {} result for experiment {}".format(step, experiment))
                  if get_preexisting_func:
                      outputs.append(get_preexisting_func(experiment, config))
                  continue
              # Else run and timestamp
              result = experiment_func(experiment, config)
              ct = int(datetime.utcnow().timestamp())
              timestamp_dir = get_exp_subdir('timestamp_dir', experiment, config)
              with open(op.join(timestamp_dir, step), 'w') as ts_file:
                  ts_file.write(str(ct))
              #TODO: Dump subset of config depended on by this step
              if result:
                  outputs.append(result)
  
      # Run any batch operations on the outputs of all experiments
      if batch_func:
          batch_func(outputs, experiments, batch_outdir, config)
  
      logging.info("Finished {} step. Elapsed time = {time:.2f} s".format(
          step, time=timeit.default_timer() - st))
code_tokens: def pipeline_run_step ( step_tuple , experiments , batch_outdir , config ) : step = step_tuple [ 0 ] experiment_func = step_tuple [ 1 ] batch_func = step_tuple [ 2 ] get_preexisting_func = step_tuple [ 3 ] use_preexisting = step_tuple [ 4 ] logging . info ( "\x1b[1mBeginning {} step...\x1b[0m" . format ( step ) ) st = timeit . default_timer ( ) outputs = [ ] if experiment_func : for experiment in experiments : if not should_run ( step , use_preexisting , experiment , config ) : logging . info ( "Using cached {} result for experiment {}" . format ( step , experiment ) ) if get_preexisting_func : outputs . append ( get_preexisting_func ( experiment , config ) ) continue result = experiment_func ( experiment , config ) ct = int ( datetime . utcnow ( ) . timestamp ( ) ) timestamp_dir = get_exp_subdir ( 'timestamp_dir' , experiment , config ) with open ( op . join ( timestamp_dir , step ) , 'w' ) as ts_file : ts_file . write ( str ( ct ) ) if result : outputs . append ( result ) if batch_func : batch_func ( outputs , experiments , batch_outdir , config ) logging . info ( "Finished {} step. Elapsed time = {time:.2f} s" . format ( step , time = timeit . default_timer ( ) - st ) )
short_docstring: Executes a step in the pipeline, managing generic logging and reuse of intermediate results
  Parameters
short_docstring_tokens: Executes a step in the pipeline managing generic logging and reuse of intermediate results Parameters
comment: """Executes a step in the pipeline, managing generic logging and reuse of intermediate results
  
      Parameters
      ----------
      step_tuple : tuple
          step_name : str
              The name of the step to execute
          experiment_func : function
              Function to run on each experiment. Should take experiment string as argument and return
          batch_func : function
              Function to run on the results of all experiments for this step
          get_preexisting_func : function
              Function to get a preexisting result (if needed for batch) when experiment_func is skipped on an experiment
          use_existing : bool
              Whether this step can skipped using preexisting results
      experiments : list
          The experiments used in this run
      batch_outdir : str
          The path to the batch output directory
      config : dict
          The config for this run
      """$SEP$# Parse step tuple$SEP$# Bold font$SEP$# Run per experiment steps (if any)$SEP$# Skip running on an experiment if we can use a pre-existing result$SEP$# Else run and timestamp$SEP$#TODO: Dump subset of config depended on by this step$SEP$# Run any batch operations on the outputs of all experiments