title: Logging separate files for different Processes in Python
body: <p>I've seen a few questions regarding putting logs from different processes together when using the multiprocessing module in Python. I would like to do the opposite, produce separate log files for different processes, and they should log everything that happens when calling other modules without being mangled. In the example below I have a main program (main.py) and two modules (module1.py and module2.py), and I want the main logger (mainlog) to write to stdout, which it does fine. I also want a separate file for each process including logging from module1 and module2.</p>
  
  <p>main.py:</p>
  
  <pre><code>import logging
  import multiprocessing as mpr
  import module1
  import sys
  
  mainlog = logging.getLogger("main")
  h = logging.StreamHandler(sys.stdout)
  mainlog.addHandler(h)
  logging.root.setLevel(logging.DEBUG)
  
  for i in xrange(0,3):
      mainlog.info("Starting process ... %s", i)
      log = logging.getLogger("module1")
      h = logging.FileHandler("process_{0}.log".format(i))
      fmt = logging.Formatter(fmt="%(levelname)-10s:%(filename)-20s:%(message)s")
      h.setFormatter(fmt)
      log.addHandler(h)
      log.setLevel(logging.DEBUG)
      p = mpr.Process(target=module1.do_something, args=(i,))
      p.start()
  </code></pre>
  
  <p>A module1.py:</p>
  
  <pre><code>import logging
  import module2
  
  log = logging.getLogger("module1")
  
  def do_something(i):
  
      for j in xrange(0,100):
          log.debug("do something. process %2s. iteration %2s", i,j)
          module2.multiply(j,2)
  </code></pre>
  
  <p>And a module2.py:</p>
  
  <pre><code>import logging
  log = logging.getLogger("module2")
  
  def multiply(x,y):
      log.debug("... multiplying %s x %s = %s", x,y, x*y)
      return x*y
  </code></pre>
  
  <p>Instead I get the following output:</p>
  
  <pre><code>Starting process ... 0
  Starting process ... 1
  No handlers could be found for logger "module2"
  Starting process ... 2
  No handlers could be found for logger "module2"
  No handlers could be found for logger "module2"
  </code></pre>
  
  <p>And 3 individual logging files (process_0.log, ...) that contain the messages from all processes together, instead of only one. Nothing from module2.py is logged. What am I doing wrong?</p>