title: How do you deal with an exception raised by celery (not your code)?
body: <p>So in my flask app right now I am using Celery to deploy servers on remote machines. Right now, I have an enum, status, which indicates the lifecycle of my deployment process:</p>
  <pre><code>@celery.task(bind=True)
  def deploy_server(self, server_id):
    server = Server.query.get(server_id)
    if not server.can_launch():
      return
  
    try:
      server.status = RemoteStatus.LAUNCHING
      db.session.commit()
  
      verify_DNS(server)
  
      host = server.server.ssh_user + '@' + server.server.ip
      execute(fabric_deploy_server, self, server, hosts=host)
  
      server.status = RemoteStatus.LAUNCHED
      db.session.commit()
    except Exception as e:
      server.status = RemoteStatus.ERROR
      db.session.commit()
      traceback.print_exc()
      raise e
  </code></pre>
  <p>As you can see, when a server is being deployed, its status is changed to &quot;Launching&quot;. If there is an exception, it will be changed to ERROR.</p>
  <p>I found one exception which completely bypasses this bloc of code: when I overloaded my celery server with too many requests, I get this exception:</p>
  <pre><code>[2017-07-09 18:00:03,127: WARNING/PoolWorker-3] /app/.heroku/python/lib/python2.7/site-packages/celery/app/trace.py:542: RuntimeWarning: Exception raised outside body: ConnectionError('max number of clients reached',): 
  Traceback (most recent call last): 
    File &quot;/app/.heroku/python/lib/python2.7/site-packages/celery/app/trace.py&quot;, line 427, in trace_task 
      uuid, retval, task_request, publish_result, 
    File &quot;/app/.heroku/python/lib/python2.7/site-packages/celery/backends/base.py&quot;, line 152, in mark_as_done 
      self.store_result(task_id, result, state, request=request) 
    File &quot;/app/.heroku/python/lib/python2.7/site-packages/celery/backends/base.py&quot;, line 309, in store_result 
      request=request, **kwargs) 
    File &quot;/app/.heroku/python/lib/python2.7/site-packages/celery/backends/base.py&quot;, line 652, in _store_result 
      self.set(self.get_key_for_task(task_id), self.encode(meta)) 
    File &quot;/app/.heroku/python/lib/python2.7/site-packages/celery/backends/redis.py&quot;, line 204, in set 
      return self.ensure(self._set, (key, value), **retry_policy) 
    File &quot;/app/.heroku/python/lib/python2.7/site-packages/celery/backends/redis.py&quot;, line 194, in ensure 
      **retry_policy) 
    File &quot;/app/.heroku/python/lib/python2.7/site-packages/kombu/utils/functional.py&quot;, line 333, in retry_over_time 
      return fun(*args, **kwargs) 
    File &quot;/app/.heroku/python/lib/python2.7/site-packages/celery/backends/redis.py&quot;, line 213, in _set 
      pipe.execute() 
    File &quot;/app/.heroku/python/lib/python2.7/site-packages/redis/client.py&quot;, line 2641, in execute 
      return execute(conn, stack, raise_on_error) 
    File &quot;/app/.heroku/python/lib/python2.7/site-packages/redis/client.py&quot;, line 2495, in _execute_transaction 
      connection.send_packed_command(all_cmds) 
    File &quot;/app/.heroku/python/lib/python2.7/site-packages/redis/connection.py&quot;, line 538, in send_packed_command 
      self.connect() 
    File &quot;/app/.heroku/python/lib/python2.7/site-packages/redis/connection.py&quot;, line 446, in connect 
      self.on_connect() 
    File &quot;/app/.heroku/python/lib/python2.7/site-packages/redis/connection.py&quot;, line 514, in on_connect 
      if nativestr(self.read_response()) != 'OK': 
    File &quot;/app/.heroku/python/lib/python2.7/site-packages/redis/connection.py&quot;, line 577, in read_response 
      response = self._parser.read_response() 
    File &quot;/app/.heroku/python/lib/python2.7/site-packages/redis/connection.py&quot;, line 255, in read_response 
      raise error 
  ConnectionError: max number of clients reached 
    exc, exc_info.traceback))) 
  </code></pre>
  <p>My biggest problem with this is that this error is raised somewhere outside of my Try/Catch bloc. Hence, when this exception occurs, all my servers remain in the &quot;Launching&quot; mode rather than &quot;Error&quot;.</p>
  <h2>How can I catch this exception and modify Server.status?</h2>