title: Celery periodic_task running multiple times in parallel
body: <p>I have some very simple periodic code using Celery's threading; it simply prints "Pre" and "Post" and sleep in between. It is adapted from <a href="https://stackoverflow.com/questions/4095940/running-unique-tasks-with-celery">this StackOverflow question</a> and <a href="http://ask.github.com/celery/cookbook/tasks.html" rel="nofollow noreferrer">this linked website</a></p>
  
  <pre><code>from celery.task import task
  from celery.task import periodic_task
  from django.core.cache import cache
  from time import sleep
  import main
  import cutout_score
  from threading import Lock
  
  import socket
  from datetime import timedelta
  from celery.decorators import task, periodic_task
  
  def single_instance_task(timeout):
    def task_exc(func):
      def wrapper(*args, **kwargs):
          lock_id = "celery-single-instance-" + func.__name__
          acquire_lock = lambda: cache.add(lock_id, "true", timeout)
          release_lock = lambda: cache.delete(lock_id)
          if acquire_lock():
              try:
                  func()
              finally:
                  release_lock()
      return wrapper
    return task_exc
  
  LOCK_EXPIRE = 60 * 5 # Lock expires in 5 minutes
  @periodic_task(run_every = timedelta(seconds=2))
  def test():
      lock_id = "lock"
  
      # cache.add fails if if the key already exists
      acquire_lock = lambda: cache.add(lock_id, "true", LOCK_EXPIRE)
      # memcache delete is very slow, but we have to use it to take
      # advantage of using add() for atomic locking
      release_lock = lambda: cache.delete(lock_id)
  
      if acquire_lock():
          try:
              print 'pre'
              sleep(20)
              print 'post'
          finally:
              release_lock()
          return
      print 'already in use...'
  </code></pre>
  
  <p>This code <em>never</em> prints <code>'already in use...'</code>; the same phenomenon occurs when I use the <code>@single_instance_task</code> decorator.</p>
  
  <p>Do you know what's wrong?</p>
  
  <p><strong>Edit: I've simplified the question so that it doesn't write to memory (using a global or the django cache); I still never see <code>'already in use...'</code></strong></p>
  
  <hr>
  
  <p><strong>Edit: When I add the following code to my Django settings.py file (by changing the code from <a href="https://docs.djangoproject.com/en/dev/topics/cache/" rel="nofollow noreferrer">https://docs.djangoproject.com/en/dev/topics/cache/</a> everything works as hoped, <em>but only when I use port 11211</em> (oddly enough, my server is on port 8000)</strong></p>
  
  <pre><code>CACHES = {
      'default': {
          'BACKEND': 'django.core.cache.backends.memcached.MemcachedCache',
          'LOCATION': [
              '127.0.0.1:11211'
          ]
      }
  }
  </code></pre>