title: Python Celery - How to separate log files
body: <p>By default the python celery package centralizes logs.
  In my design specs, I would like to have the logs of each worker written to a separate file where the filename would match unique ids sent to the task.
  Centrilized logs shows which worker the log is from but I could not figure out how to map the id to worker number.</p>
  
  <p>To decentralize the logs, I tried:</p>
  
  <pre><code>import logging
  CELERYD_HIJACK_ROOT_LOGGER = False
  app = Celery()
  @app.task
  def my_task(id):
      logging.basicConfig(filename='/tmp/%i.log' % id)
      my_package.do_somthing()
      logging.info('hi')
  </code></pre>
  
  <p>but this is not creating any log file.
  Is there a way to separate the logs or a way to map worker numbers to ids so that I can generate separate logs?</p>
  
  <hr>
  
  <p><strong>update:</strong></p>
  
  <p>Reading <a href="http://docs.celeryproject.org/en/latest/userguide/tasks.html#names" rel="nofollow">http://docs.celeryproject.org/en/latest/userguide/tasks.html#names</a>
  I tried this approach that still fails to make separate logs:</p>
  
  <p>celeryconfig.py:</p>
  
  <p>CELERYD_HIJACK_ROOT_LOGGER = False</p>
  
  <p>main.py</p>
  
  <pre><code>import logging
  app = Celery()
  app.config_from_object('celeryconfig')
  @app.task(bind=True)
  def my_task(self, id):
      self.request.logfile = '/tmp/%i.log' % id
      my_package.do_somthing()
      logging.info('hi')
  </code></pre>
  
  <p>Also, I discovered a blog: <a href="http://echorand.me/2012/08/14/celery-and-python-logging/" rel="nofollow">http://echorand.me/2012/08/14/celery-and-python-logging/</a>
  and tried it. It only would write the log message 'hi' to separate files but will continute to send logs from </p>
  
  <pre><code>my_package.do_somthing()
  </code></pre>
  
  <p>to standard output meaning that the logger is not fixed globally and passing around the logger is going to get very messy so that will not do.</p>
  
  <hr>
  
  <p><strong>update</strong></p>
  
  <p>I tried the code from the blog:
  <a href="https://github.com/kouroshparsa/celery_separate_logging/tree/master/celery_example" rel="nofollow">https://github.com/kouroshparsa/celery_separate_logging/tree/master/celery_example</a></p>
  
  <p>when calling mylib/fire.py: burn method, the logging is not stored anywhere.</p>