{"action":"created","issue":{"url":"https://api.github.com/repos/celery/kombu/issues/573","repository_url":"https://api.github.com/repos/celery/kombu","labels_url":"https://api.github.com/repos/celery/kombu/issues/573/labels{/name}","comments_url":"https://api.github.com/repos/celery/kombu/issues/573/comments","events_url":"https://api.github.com/repos/celery/kombu/issues/573/events","html_url":"https://github.com/celery/kombu/issues/573","id":140895353,"number":573,"title":"python exceptions are not json serializable.","user":{"login":"jayendra13","id":651057,"avatar_url":"https://avatars.githubusercontent.com/u/651057?v=3","gravatar_id":"","url":"https://api.github.com/users/jayendra13","html_url":"https://github.com/jayendra13","followers_url":"https://api.github.com/users/jayendra13/followers","following_url":"https://api.github.com/users/jayendra13/following{/other_user}","gists_url":"https://api.github.com/users/jayendra13/gists{/gist_id}","starred_url":"https://api.github.com/users/jayendra13/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jayendra13/subscriptions","organizations_url":"https://api.github.com/users/jayendra13/orgs","repos_url":"https://api.github.com/users/jayendra13/repos","events_url":"https://api.github.com/users/jayendra13/events{/privacy}","received_events_url":"https://api.github.com/users/jayendra13/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"milestone":null,"comments":7,"created_at":"2016-03-15T07:21:41Z","updated_at":"2016-03-15T18:07:33Z","closed_at":"2016-03-15T18:07:33Z","body":"I have some celery tasks, which may raise exception, if required. The problem is python Exception object is not json-serialization, yet though yaml can serialize python Exception objects but I don't have control to change this configuration for celery.  \r\n\r\nCan we in celery(more specifically in kombu) internally handle, serialization of exception object ? \r\n\r\nHere is a snippet which will generate error.\r\n\r\n```\r\nfrom kombu import Connection\r\n\r\nex = None\r\ntry:\r\n    a = 1/0\r\nexcept ZeroDivisionError, e:\r\n    ex = e\r\nprint ex\r\n\r\nconn = Connection('amqp://guest:guest@localhost:5672//')\r\nsimple_queue = conn.SimpleQueue('simple_queue')\r\n\r\n# No error as expected\r\nsimple_queue.put('Hello')\r\n\r\n# No error as expected\r\nsimple_queue.put(str(ex))\r\n\r\n# json cannot serliaze exception\r\n# hence celery gives error\r\nsimple_queue.put(ex)\r\n```\r\n\r\nHere is error which I get.\r\n\r\n```\r\n---------------------------------------------------------------------------\r\nEncodeError                               Traceback (most recent call last)\r\n<ipython-input-22-9667148d81ba> in <module>()\r\n----> 1 sq.put(ex)\r\n\r\n/usr/local/lib/python2.7/dist-packages/kombu/simple.pyc in put(self, message, serializer, headers, compression, routing_key, **kwargs)\r\n     70                               headers=headers,\r\n     71                               compression=compression,\r\n---> 72                               **kwargs)\r\n     73 \r\n     74     def clear(self):\r\n\r\n/usr/local/lib/python2.7/dist-packages/kombu/messaging.pyc in publish(self, body, routing_key, delivery_mode, mandatory, immediate, priority, content_type, content_encoding, serializer, headers, compression, exchange, retry, retry_policy, declare, expiration, **properties)\r\n    163         body, content_type, content_encoding = self._prepare(\r\n    164             body, serializer, content_type, content_encoding,\r\n--> 165             compression, headers)\r\n    166 \r\n    167         publish = self._publish\r\n\r\n/usr/local/lib/python2.7/dist-packages/kombu/messaging.pyc in _prepare(self, body, serializer, content_type, content_encoding, compression, headers)\r\n    239             serializer = serializer or self.serializer\r\n    240             (content_type, content_encoding,\r\n--> 241              body) = dumps(body, serializer=serializer)\r\n    242         else:\r\n    243             # If the programmer doesn't want us to serialize,\r\n\r\n/usr/local/lib/python2.7/dist-packages/kombu/serialization.pyc in dumps(self, data, serializer)\r\n    162 \r\n    163         with _reraise_errors(EncodeError):\r\n--> 164             payload = encoder(data)\r\n    165         return content_type, content_encoding, payload\r\n    166     encode = dumps  # XXX compat\r\n\r\n/usr/lib/python2.7/contextlib.pyc in __exit__(self, type, value, traceback)\r\n     33                 value = type()\r\n     34             try:\r\n---> 35                 self.gen.throw(type, value, traceback)\r\n     36                 raise RuntimeError(\"generator didn't stop after throw()\")\r\n     37             except StopIteration, exc:\r\n\r\n/usr/local/lib/python2.7/dist-packages/kombu/serialization.pyc in _reraise_errors(wrapper, include, exclude)\r\n     57         raise\r\n     58     except include as exc:\r\n---> 59         reraise(wrapper, wrapper(exc), sys.exc_info()[2])\r\n     60 \r\n     61 \r\n\r\n/usr/local/lib/python2.7/dist-packages/kombu/serialization.pyc in _reraise_errors(wrapper, include, exclude)\r\n     53                     include=(Exception, ), exclude=(SerializerNotInstalled, )):\r\n     54     try:\r\n---> 55         yield\r\n     56     except exclude:\r\n     57         raise\r\n\r\n/usr/local/lib/python2.7/dist-packages/kombu/serialization.pyc in dumps(self, data, serializer)\r\n    162 \r\n    163         with _reraise_errors(EncodeError):\r\n--> 164             payload = encoder(data)\r\n    165         return content_type, content_encoding, payload\r\n    166     encode = dumps  # XXX compat\r\n\r\n/usr/local/lib/python2.7/dist-packages/anyjson/__init__.pyc in dumps(value)\r\n    139     def dumps(value):\r\n    140         \"\"\"Deserialize JSON-encoded object to a Python object.\"\"\"\r\n--> 141         return implementation.dumps(value)\r\n    142     serialize = dumps\r\n\r\n/usr/local/lib/python2.7/dist-packages/anyjson/__init__.pyc in dumps(self, data)\r\n     85         TypeError if the object could not be serialized.\"\"\"\r\n     86         try:\r\n---> 87             return self._encode(data)\r\n     88         except self._encode_error, exc:\r\n     89             raise TypeError, TypeError(*exc.args), sys.exc_info()[2]\r\n\r\n/usr/local/lib/python2.7/dist-packages/simplejson-3.8.1-py2.7-linux-x86_64.egg/simplejson/__init__.pyc in dumps(obj, skipkeys, ensure_ascii, check_circular, allow_nan, cls, indent, separators, encoding, default, use_decimal, namedtuple_as_object, tuple_as_array, bigint_as_string, sort_keys, item_sort_key, for_json, ignore_nan, int_as_string_bitcount, iterable_as_array, **kw)\r\n    378         and not kw\r\n    379     ):\r\n--> 380         return _default_encoder.encode(obj)\r\n    381     if cls is None:\r\n    382         cls = JSONEncoder\r\n\r\n/usr/local/lib/python2.7/dist-packages/simplejson-3.8.1-py2.7-linux-x86_64.egg/simplejson/encoder.pyc in encode(self, o)\r\n    273         # exceptions aren't as detailed.  The list call should be roughly\r\n    274         # equivalent to the PySequence_Fast that ''.join() would do.\r\n--> 275         chunks = self.iterencode(o, _one_shot=True)\r\n    276         if not isinstance(chunks, (list, tuple)):\r\n    277             chunks = list(chunks)\r\n\r\n/usr/local/lib/python2.7/dist-packages/simplejson-3.8.1-py2.7-linux-x86_64.egg/simplejson/encoder.pyc in iterencode(self, o, _one_shot)\r\n    355                 self.iterable_as_array, Decimal=decimal.Decimal)\r\n    356         try:\r\n--> 357             return _iterencode(o, 0)\r\n    358         finally:\r\n    359             key_memo.clear()\r\n\r\n/usr/local/lib/python2.7/dist-packages/simplejson-3.8.1-py2.7-linux-x86_64.egg/simplejson/encoder.pyc in default(self, o)\r\n    250 \r\n    251         \"\"\"\r\n--> 252         raise TypeError(repr(o) + \" is not JSON serializable\")\r\n    253 \r\n    254     def encode(self, o):\r\n\r\n```"},"comment":{"url":"https://api.github.com/repos/celery/kombu/issues/comments/196953343","html_url":"https://github.com/celery/kombu/issues/573#issuecomment-196953343","issue_url":"https://api.github.com/repos/celery/kombu/issues/573","id":196953343,"user":{"login":"ask","id":1558,"avatar_url":"https://avatars.githubusercontent.com/u/1558?v=3","gravatar_id":"","url":"https://api.github.com/users/ask","html_url":"https://github.com/ask","followers_url":"https://api.github.com/users/ask/followers","following_url":"https://api.github.com/users/ask/following{/other_user}","gists_url":"https://api.github.com/users/ask/gists{/gist_id}","starred_url":"https://api.github.com/users/ask/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ask/subscriptions","organizations_url":"https://api.github.com/users/ask/orgs","repos_url":"https://api.github.com/users/ask/repos","events_url":"https://api.github.com/users/ask/events{/privacy}","received_events_url":"https://api.github.com/users/ask/received_events","type":"User","site_admin":false},"created_at":"2016-03-15T18:07:33Z","updated_at":"2016-03-15T18:07:33Z","body":"Kombu doesn't care about the content of your messages, celery has a specific task protocol and tools to deal with exceptions.\r\n\r\nCelery task results can definitely deal with exceptions using json, in the case where it don't it would be a bug and should be reported to the celery issue tracker!"}}