title: Redirect stdout and stderr of an external library in Python
body: <p>I'm trying to collect the log of the search of the Google OR-tools library. Specifically, I'm using python to write a program that uses the library to solve a TSP instance. What I'm trying to do is to make the library solve the same instance with different methods and collect the output of the searching (which the library writes to stderr). The problem is that the library was written and compiled in C++ and only the wrappers for python have been written. I tried to use <code>redirect_stderr</code> and also the <code>sys</code> module to set <code>stderr</code> to another file, but the result is the same: the output still gets written on stderr and shown on the console. Because I've written a multithreaded program, shell redirection is not really an option. Is there a way to solve this issue?</p>
  <p>The script below was taken from the Google OR-Tools documentation and slightly modified to include the specification of the methods to use <em>and</em> my attempt to redirect the srderr file:</p>
  <pre><code>&quot;&quot;&quot;Simple travelling salesman problem between cities.&quot;&quot;&quot;
  
  from __future__ import print_function
  from ortools.constraint_solver import routing_enums_pb2
  from ortools.constraint_solver import pywrapcp
  from contextlib import redirect_stderr, redirect_stdout
  import sys
  
  
  def create_data_model():
      &quot;&quot;&quot;Stores the data for the problem.&quot;&quot;&quot;
      data = {}
      data['distance_matrix'] = [
          [0, 2451, 713, 1018, 1631, 1374, 2408, 213, 2571, 875, 1420, 2145, 1972],
          [2451, 0, 1745, 1524, 831, 1240, 959, 2596, 403, 1589, 1374, 357, 579],
          [713, 1745, 0, 355, 920, 803, 1737, 851, 1858, 262, 940, 1453, 1260],
          [1018, 1524, 355, 0, 700, 862, 1395, 1123, 1584, 466, 1056, 1280, 987],
          [1631, 831, 920, 700, 0, 663, 1021, 1769, 949, 796, 879, 586, 371],
          [1374, 1240, 803, 862, 663, 0, 1681, 1551, 1765, 547, 225, 887, 999],
          [2408, 959, 1737, 1395, 1021, 1681, 0, 2493, 678, 1724, 1891, 1114, 701],
          [213, 2596, 851, 1123, 1769, 1551, 2493, 0, 2699, 1038, 1605, 2300, 2099],
          [2571, 403, 1858, 1584, 949, 1765, 678, 2699, 0, 1744, 1645, 653, 600],
          [875, 1589, 262, 466, 796, 547, 1724, 1038, 1744, 0, 679, 1272, 1162],
          [1420, 1374, 940, 1056, 879, 225, 1891, 1605, 1645, 679, 0, 1017, 1200],
          [2145, 357, 1453, 1280, 586, 887, 1114, 2300, 653, 1272, 1017, 0, 504],
          [1972, 579, 1260, 987, 371, 999, 701, 2099, 600, 1162, 1200, 504, 0],
      ]  # yapf: disable
      data['num_vehicles'] = 1
      data['depot'] = 0
      return data
  
  
  def print_solution(manager, routing, solution):
      &quot;&quot;&quot;Prints solution on console.&quot;&quot;&quot;
      print('Objective: {} miles'.format(solution.ObjectiveValue()))
      index = routing.Start(0)
      plan_output = 'Route for vehicle 0:\n'
      route_distance = 0
      while not routing.IsEnd(index):
          plan_output += ' {} -&gt;'.format(manager.IndexToNode(index))
          previous_index = index
          index = solution.Value(routing.NextVar(index))
          route_distance += routing.GetArcCostForVehicle(previous_index, index, 0)
      plan_output += ' {}\n'.format(manager.IndexToNode(index))
      plan_output += 'Route distance: {}miles\n'.format(route_distance)
      print(plan_output)
  
  
  def main():
      &quot;&quot;&quot;Entry point of the program.&quot;&quot;&quot;
      # Instantiate the data problem.
      data = create_data_model()
  
      # Create the routing index manager.
      manager = pywrapcp.RoutingIndexManager(len(data['distance_matrix']),
                                             data['num_vehicles'], data['depot'])
  
      # Create Routing Model.
      routing = pywrapcp.RoutingModel(manager)
  
  
      def distance_callback(from_index, to_index):
          &quot;&quot;&quot;Returns the distance between the two nodes.&quot;&quot;&quot;
          # Convert from routing variable Index to distance matrix NodeIndex.
          from_node = manager.IndexToNode(from_index)
          to_node = manager.IndexToNode(to_index)
          return data['distance_matrix'][from_node][to_node]
  
      transit_callback_index = routing.RegisterTransitCallback(distance_callback)
  
      # Define cost of each arc.
      routing.SetArcCostEvaluatorOfAllVehicles(transit_callback_index)
  
      # Setting first solution heuristic.
      search_parameters = pywrapcp.DefaultRoutingSearchParameters()
      search_parameters.first_solution_strategy = (
          routing_enums_pb2.FirstSolutionStrategy.PATH_CHEAPEST_ARC)
      search_parameters.local_search_metaheuristic = (routing_enums_pb2.LocalSearchMetaheuristic.TABU_SEARCH)
      search_parameters.time_limit.seconds = 1
      search_parameters.log_search = True
  
      # This is the first approach and it does not work
      # (Yes, I'm sure that it's routing.SolveWithParameters() that logs the search
      # parameters to stderr.
      with open(&quot;output.txt&quot;, &quot;wt&quot;) as f:
          with redirect_stderr(f):
              solution = routing.SolveWithParameters(search_parameters)
  
      # Print solution on console.
      if solution:
          print_solution(manager, routing, solution)
  
  
  if __name__ == '__main__':
      main()
  </code></pre>
  <p>The second approach was:</p>
  <pre><code>#[code omitted: see above]
      search_parameters.time_limit.seconds = 1
      search_parameters.log_search = True
  
      # This is the second approach and it does not work
      # (Yes, I'm sure that it's routing.SolveWithParameters() that logs the search
      # parameters to stderr.
      sys.stderr = open(&quot;output.txt&quot;, &quot;wt&quot;)
      solution = routing.SolveWithParameters(search_parameters)
  #[code omitted: see above]
  </code></pre>