{"action":"created","issue":{"url":"https://api.github.com/repos/jazzband/django-silk/issues/265","repository_url":"https://api.github.com/repos/jazzband/django-silk","labels_url":"https://api.github.com/repos/jazzband/django-silk/issues/265/labels{/name}","comments_url":"https://api.github.com/repos/jazzband/django-silk/issues/265/comments","events_url":"https://api.github.com/repos/jazzband/django-silk/issues/265/events","html_url":"https://github.com/jazzband/django-silk/issues/265","id":291769827,"node_id":"MDU6SXNzdWUyOTE3Njk4Mjc=","number":265,"title":"DB Deadlock when stress testing with silk","user":{"login":"selimt","id":15021762,"node_id":"MDQ6VXNlcjE1MDIxNzYy","avatar_url":"https://avatars2.githubusercontent.com/u/15021762?v=4","gravatar_id":"","url":"https://api.github.com/users/selimt","html_url":"https://github.com/selimt","followers_url":"https://api.github.com/users/selimt/followers","following_url":"https://api.github.com/users/selimt/following{/other_user}","gists_url":"https://api.github.com/users/selimt/gists{/gist_id}","starred_url":"https://api.github.com/users/selimt/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/selimt/subscriptions","organizations_url":"https://api.github.com/users/selimt/orgs","repos_url":"https://api.github.com/users/selimt/repos","events_url":"https://api.github.com/users/selimt/events{/privacy}","received_events_url":"https://api.github.com/users/selimt/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":3,"created_at":"2018-01-26T01:11:07Z","updated_at":"2018-12-13T12:57:15Z","closed_at":"2018-01-28T23:14:39Z","author_association":"NONE","body":"I got a deadlock error when testing with the latest version of silk (1.1.0).\r\n\r\nHere is the DB output:\r\n\r\n```\r\n        Process 149: DELETE FROM \"silk_response\" WHERE \"silk_response\".\"request_id\" IN ('629f685a-26dc-41ca-ba9d-cc8af1998ef9', '8336effc-217c-47a7-8cc2-6944b8aede7f', 'aa12852b-5f5b-40fa-a647-cc3ddd2ef4ba', '2b2d5e84-e595-4ccf-b06c-78db181b4f24', '3f94c00d-f0c3-4021-8def-703e363e4f8f', 'e88b9162-0455-4e78-ba7b-029694b493d8')\r\n        Process 150: DELETE FROM \"silk_response\" WHERE \"silk_response\".\"request_id\" IN ('629f685a-26dc-41ca-ba9d-cc8af1998ef9', '9ad245bb-4b6a-49cf-b32c-1a547ee8c11f', '93c50572-f6ec-416c-a9d6-42798f1631dd', '1ffd6a86-f93e-48f8-a593-86184e26c417', '3f94c00d-f0c3-4021-8def-703e363e4f8f')\r\n        Process 151: DELETE FROM \"silk_response\" WHERE \"silk_response\".\"request_id\" IN ('629f685a-26dc-41ca-ba9d-cc8af1998ef9', '9ad245bb-4b6a-49cf-b32c-1a547ee8c11f', 'aa12852b-5f5b-40fa-a647-cc3ddd2ef4ba', '8336effc-217c-47a7-8cc2-6944b8aede7f', '2b2d5e84-e595-4ccf-b06c-78db181b4f24', '93c50572-f6ec-416c-a9d6-42798f1631dd', '1ffd6a86-f93e-48f8-a593-86184e26c417', '3f94c00d-f0c3-4021-8def-703e363e4f8f', 'e88b9162-0455-4e78-ba7b-029694b493d8')\r\n        Process 144: DELETE FROM \"silk_response\" WHERE \"silk_response\".\"request_id\" IN ('629f685a-26dc-41ca-ba9d-cc8af1998ef9', '9ad245bb-4b6a-49cf-b32c-1a547ee8c11f', 'aa12852b-5f5b-40fa-a647-cc3ddd2ef4ba', '8336effc-217c-47a7-8cc2-6944b8aede7f', '2b2d5e84-e595-4ccf-b06c-78db181b4f24', '93c50572-f6ec-416c-a9d6-42798f1631dd', '1ffd6a86-f93e-48f8-a593-86184e26c417', '3f94c00d-f0c3-4021-8def-703e363e4f8f', 'e88b9162-0455-4e78-ba7b-029694b493d8')\r\n2018-01-26 01:03:55.996 UTC [148] HINT:  See server log for query details.\r\n2018-01-26 01:03:55.996 UTC [148] STATEMENT:  DELETE FROM \"silk_response\" WHERE \"silk_response\".\"request_id\" IN ('629f685a-26dc-41ca-ba9d-cc8af1998ef9', '9ad245bb-4b6a-49cf-b32c-1a547ee8c11f', 'aa12852b-5f5b-40fa-a647-cc3ddd2ef4ba', '8336effc-217c-47a7-8cc2-6944b8aede7f', '2b2d5e84-e595-4ccf-b06c-78db181b4f24', '1ffd6a86-f93e-48f8-a593-86184e26c417', '3f94c00d-f0c3-4021-8def-703e363e4f8f', 'e88b9162-0455-4e78-ba7b-029694b493d8')\r\n2018-01-26 01:03:56.024 UTC [144] ERROR:  deadlock detected\r\n2018-01-26 01:03:56.024 UTC [144] DETAIL:  Process 144 waits for AccessExclusiveLock on tuple (443,8) of relation 16579 of database 12994; blocked by process 149.\r\n        Process 149 waits for ShareLock on transaction 152027; blocked by process 150.\r\n        Process 150 waits for AccessExclusiveLock on tuple (440,10) of relation 16579 of database 12994; blocked by process 151.\r\n        Process 151 waits for ShareLock on transaction 152109; blocked by process 144.\r\n```\r\n\r\nOutput from pip freeze:\r\n\r\n```\r\namqp==2.2.2\r\nautopep8==1.3.4\r\nbilliard==3.5.0.3\r\nblessings==1.6.1\r\nbpython==0.17\r\ncached-property==1.3.1\r\ncelery==4.1.0\r\ncertifi==2018.1.18\r\nchardet==3.0.4\r\ncoreapi==2.3.3\r\ncoreschema==0.0.4\r\ncoverage==4.4.2\r\ncurtsies==0.2.11\r\nDjango==1.11.9\r\ndjango-auth-ldap==1.3.0\r\ndjango-cachalot==1.5.0\r\ndjango-extensions==1.9.9\r\ndjango-model-utils==3.1.1\r\ndjango-prettyjson==0.3.0\r\ndjango-rest-swagger==2.1.2\r\ndjango-silk==1.1.0\r\ndjango-url-filter==0.3.4\r\ndjangorestframework==3.7.7\r\ndrf-extensions==0.3.1\r\ndrf-nested-routers==0.90.0\r\nenum-compat==0.0.2\r\ngevent==1.2.2\r\ngprof2dot==2016.10.13\r\ngreenlet==0.4.12\r\ngunicorn==19.7.1\r\nidna==2.6\r\nitypes==1.1.0\r\nJinja2==2.10\r\nkombu==4.1.0\r\nMarkdown==2.6.11\r\nMarkupSafe==1.0\r\nmercury==1.0.0\r\nmock==2.0.0\r\nopenapi-codec==1.3.2\r\npbr==3.1.1\r\npsycopg2==2.7.3.2\r\npycodestyle==2.3.1\r\nPygments==2.2.0\r\npyldap==2.4.45\r\npython-dateutil==2.6.1\r\npython-memcached==1.59\r\npytz==2017.3\r\nrequests==2.18.4\r\nsimplejson==3.13.2\r\nsix==1.11.0\r\nsqlparse==0.2.4\r\nstandardjson==0.3.1\r\nstevedore==1.28.0\r\ntyping==3.6.4\r\nuritemplate==3.0.0\r\nurllib3==1.22\r\nvine==1.1.4\r\nwcwidth==0.1.7\r\nWerkzeug==0.14.1\r\n```\r\n\r\nThanks\r\n-Selim\r\n"},"comment":{"url":"https://api.github.com/repos/jazzband/django-silk/issues/comments/446959225","html_url":"https://github.com/jazzband/django-silk/issues/265#issuecomment-446959225","issue_url":"https://api.github.com/repos/jazzband/django-silk/issues/265","id":446959225,"node_id":"MDEyOklzc3VlQ29tbWVudDQ0Njk1OTIyNQ==","user":{"login":"bj00rn","id":10961167,"node_id":"MDQ6VXNlcjEwOTYxMTY3","avatar_url":"https://avatars0.githubusercontent.com/u/10961167?v=4","gravatar_id":"","url":"https://api.github.com/users/bj00rn","html_url":"https://github.com/bj00rn","followers_url":"https://api.github.com/users/bj00rn/followers","following_url":"https://api.github.com/users/bj00rn/following{/other_user}","gists_url":"https://api.github.com/users/bj00rn/gists{/gist_id}","starred_url":"https://api.github.com/users/bj00rn/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/bj00rn/subscriptions","organizations_url":"https://api.github.com/users/bj00rn/orgs","repos_url":"https://api.github.com/users/bj00rn/repos","events_url":"https://api.github.com/users/bj00rn/events{/privacy}","received_events_url":"https://api.github.com/users/bj00rn/received_events","type":"User","site_admin":false},"created_at":"2018-12-13T12:57:15Z","updated_at":"2018-12-13T12:57:15Z","author_association":"NONE","body":"getting this as well. Serving django through 4x daphne workers\r\n\r\n```\r\n File \"/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py\", line 64, in execute\r\n     return self.cursor.execute(sql, params)\r\n psycopg2.extensions.TransactionRollbackError: deadlock detected\r\n DETAIL:  Process 4930 waits for AccessExclusiveLock on tuple (9,89) of relation 21806 of database 20072; blocked by process 4929.\r\n Process 4929 waits for ShareLock on transaction 94641; blocked by process 4928.\r\n Process 4928 waits for ShareLock on transaction 94582; blocked by process 4926.\r\n Process 4926 waits for AccessExclusiveLock on tuple (9,89) of relation 21806 of database 20072; blocked by process 4930.\r\n HINT:  See server log for query details.\r\n\r\n\r\n The above exception was the direct cause of the following exception:\r\n\r\n Traceback (most recent call last):\r\n   File \"/usr/local/lib/python3.6/site-packages/silk/middleware.py\", line 128, in process_response\r\n     self._process_response(request, response)\r\n   File \"/usr/local/lib/python3.6/contextlib.py\", line 52, in inner\r\n     return func(*args, **kwds)\r\n   File \"/usr/local/lib/python3.6/site-packages/silk/middleware.py\", line 110, in _process_response\r\n     collector.finalise()\r\n   File \"/usr/local/lib/python3.6/site-packages/silk/collector.py\", line 158, in finalise\r\n     query_model = models.SQLQuery.objects.create(**query)\r\n   File \"/usr/local/lib/python3.6/site-packages/django/db/models/manager.py\", line 85, in manager_method\r\n     return getattr(self.get_queryset(), name)(*args, **kwargs)\r\n   File \"/usr/local/lib/python3.6/site-packages/django/db/models/query.py\", line 394, in create\r\n     obj.save(force_insert=True, using=self.db)\r\n   File \"/usr/local/lib/python3.6/contextlib.py\", line 52, in inner\r\n     return func(*args, **kwds)\r\n   File \"/usr/local/lib/python3.6/site-packages/silk/models.py\", line 283, in save\r\n     self.request.save(update_fields=['num_sql_queries'])\r\n   File \"/usr/local/lib/python3.6/site-packages/silk/models.py\", line 167, in save\r\n     Request.garbage_collect(force=False)\r\n   File \"/usr/local/lib/python3.6/site-packages/silk/models.py\", line 152, in garbage_collect\r\n     cls.objects.filter(start_time__lte=time_cutoff).delete()\r\n   File \"/usr/local/lib/python3.6/site-packages/django/db/models/query.py\", line 620, in delete\r\n     deleted, _rows_count = collector.delete()\r\n   File \"/usr/local/lib/python3.6/site-packages/django/db/models/deletion.py\", line 284, in delete\r\n     count = qs._raw_delete(using=self.using)\r\n   File \"/usr/local/lib/python3.6/site-packages/django/db/models/query.py\", line 634, in _raw_delete\r\n     return sql.DeleteQuery(self.model).delete_qs(self, using)\r\n   File \"/usr/local/lib/python3.6/site-packages/django/db/models/sql/subqueries.py\", line 80, in delete_qs\r\n     cursor = self.get_compiler(using).execute_sql(CURSOR)\r\n   File \"/usr/local/lib/python3.6/site-packages/silk/sql.py\", line 45, in execute_sql\r\n     return self._execute_sql(*args, **kwargs)\r\n   File \"/usr/local/lib/python3.6/site-packages/django/db/models/sql/compiler.py\", line 899, in execute_sql\r\n     raise original_exception\r\n   File \"/usr/local/lib/python3.6/site-packages/django/db/models/sql/compiler.py\", line 889, in execute_sql\r\n     cursor.execute(sql, params)\r\n   File \"/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py\", line 64, in execute\r\n     return self.cursor.execute(sql, params)\r\n   File \"/usr/local/lib/python3.6/site-packages/django/db/utils.py\", line 94, in __exit__\r\n     six.reraise(dj_exc_type, dj_exc_value, traceback)\r\n   File \"/usr/local/lib/python3.6/site-packages/django/utils/six.py\", line 685, in reraise\r\n     raise value.with_traceback(tb)\r\n   File \"/usr/local/lib/python3.6/site-packages/django/db/backends/utils.py\", line 64, in execute\r\n     return self.cursor.execute(sql, params)\r\n django.db.utils.OperationalError: deadlock detected\r\n DETAIL:  Process 4930 waits for AccessExclusiveLock on tuple (9,89) of relation 21806 of database 20072; blocked by process 4929.\r\n Process 4929 waits for ShareLock on transaction 94641; blocked by process 4928.\r\n Process 4928 waits for ShareLock on transaction 94582; blocked by process 4926.\r\n Process 4926 waits for AccessExclusiveLock on tuple (9,89) of relation 21806 of database 20072; blocked by process 4930.\r\n HINT:  See server log for query details.\r\n\r\n\r\n During handling of the above exception, another exception occurred:\r\n\r\n Traceback (most recent call last):\r\n   File \"/usr/local/lib/python3.6/site-packages/silk/middleware.py\", line 110, in _process_response\r\n     collector.finalise()\r\n   File \"/usr/local/lib/python3.6/site-packages/silk/collector.py\", line 158, in finalise\r\n     query_model = models.SQLQuery.objects.create(**query)\r\n   File \"/usr/local/lib/python3.6/site-packages/django/db/models/manager.py\", line 85, in manager_method\r\n     return getattr(self.get_queryset(), name)(*args, **kwargs)\r\n   File \"/usr/local/lib/python3.6/site-packages/django/db/models/query.py\", line 392, in create\r\n     obj = self.model(**kwargs)\r\n   File \"/usr/local/lib/python3.6/site-packages/django/db/models/base.py\", line 573, in __init__\r\n     raise TypeError(\"'%s' is an invalid keyword argument for this function\" % list(kwargs)[0])\r\n TypeError: 'model' is an invalid keyword argument for this function\r\n```\r\n...\r\n ERROR:  deadlock detected\r\n DETAIL:  Process 4920 waits for ShareLock on transaction 94582; blocked by process 4926.\r\n      Process 4926 waits for AccessExclusiveLock on tuple (9,89) of relation 21806 of database 20072; blocked by process 4929.\r\n      Process 4929 waits for ShareLock on transaction 94641; blocked by process 4928.\r\n      Process 4928 waits for AccessExclusiveLock on tuple (44,75) of relation 21806 of database 20072; blocked by process 4920.\r\n      Process 4920: DELETE FROM \"silk_response\" WHERE \"silk_response\".\"request_id\" IN ('98a609f6-178c-4754-91d9-a500287f1a6e', '87277d11-24c6-481c-a51d-40adcad0891b', '0fce5919-c945-43ca-939f-915c6b89cb3a', '20eb66ac-23ee-466e-add7-888c98071a0f', 'f810b99c-446e-4272-88e0-4722ba133474', '92251ef1-feb4-4c72-a061-f04ebf8fb0a2', '6379e858-f6fc-44d3-aa5d-edc338c1683e', '436ff660-e456-42b3-ab88-018f18159d09')\r\n      Process 4926: DELETE FROM \"silk_response\" WHERE \"silk_response\".\"request_id\" IN ('80a46b34-8b35-455b-9b12-ab8ae03121d3', 'a46762ea-9262-4943-b95c-171fb98f2fcd', '2d9bbe1d-43e2-4a17-afe6-8ddcd8b2036c', '45a61cf6-45c2-4a38-824c-0d9774db54d2', 'ed59a924-4830-4b92-8e56-cea6cd2ea273')\r\n      Process 4929: DELETE FROM \"silk_response\" WHERE \"silk_response\".\"request_id\" IN ('a46762ea-9262-4943-b95c-171fb98f2fcd', '80a46b34-8b35-455b-9b12-ab8ae03121d3', '2d9bbe1d-43e2-4a17-afe6-8ddcd8b2036c', 'f810b99c-446e-4272-88e0-4722ba133474', '87277d11-24c6-481c-a51d-40adcad0891b', '45a61cf6-45c2-4a38-824c-0d9774db54d2')\r\n      Process 4928: DELETE FROM \"silk_response\" WHERE \"silk_response\".\"request_id\" IN ('80a46b34-8b35-455b-9b12-ab8ae03121d3', '45a61cf6-45c2-4a38-824c-0d9774db54d2', '87277d11-24c6-481c-a51d-40adcad0891b', '0fce5919-c945-43ca-939f-915c6b89cb3a', 'a46762ea-9262-4943-b95c-171fb98f2fcd', '20eb66ac-23ee-466e-add7-888c98071a0f', 'f810b99c-446e-4272-88e0-4722ba133474', '6379e858-f6fc-44d3-aa5d-edc338c1683e')\r\n HINT:  See server log for query details.\r\n CONTEXT:  while deleting tuple (44,75) in relation \"silk_response\"\r\n STATEMENT:  DELETE FROM \"silk_response\" WHERE \"silk_response\".\"request_id\" IN ('98a609f6-178c-4754-91d9-a500287f1a6e', '87277d11-24c6-481c-a51d-40adcad0891b', '0fce5919-c945-43ca-939f-915c6b89cb3a', '20eb66ac-23ee-466e-add7-888c98071a0f', 'f810b99c-446e-4272-88e0-4722ba133474', '92251ef1-feb4-4c72-a061-f04ebf8fb0a2', '6379e858-f6fc-44d3-aa5d-edc338c1683e', '436ff660-e456-42b3-ab88-018f18159d09')\r\n ERROR:  deadlock detected\r\n DETAIL:  Process 4930 waits for AccessExclusiveLock on tuple (9,89) of relation 21806 of database 20072; blocked by process 4929.\r\n      Process 4929 waits for ShareLock on transaction 94641; blocked by process 4928.\r\n      Process 4928 waits for ShareLock on transaction 94582; blocked by process 4926.\r\n      Process 4926 waits for AccessExclusiveLock on tuple (9,89) of relation 21806 of database 20072; blocked by process 4930.\r\n      Process 4930: DELETE FROM \"silk_response\" WHERE \"silk_response\".\"request_id\" IN ('80a46b34-8b35-455b-9b12-ab8ae03121d3', '87277d11-24c6-481c-a51d-40adcad0891b', 'a46762ea-9262-4943-b95c-171fb98f2fcd', '2d9bbe1d-43e2-4a17-afe6-8ddcd8b2036c', '45a61cf6-45c2-4a38-824c-0d9774db54d2', 'f810b99c-446e-4272-88e0-4722ba133474')\r\n      Process 4929: DELETE FROM \"silk_response\" WHERE \"silk_response\".\"request_id\" IN ('a46762ea-9262-4943-b95c-171fb98f2fcd', '80a46b34-8b35-455b-9b12-ab8ae03121d3', '2d9bbe1d-43e2-4a17-afe6-8ddcd8b2036c', 'f810b99c-446e-4272-88e0-4722ba133474', '87277d11-24c6-481c-a51d-40adcad0891b', '45a61cf6-45c2-4a38-824c-0d9774db54d2')\r\n      Process 4928: DELETE FROM \"silk_response\" WHERE \"silk_response\".\"request_id\" IN ('80a46b34-8b35-455b-9b12-ab8ae03121d3', '45a61cf6-45c2-4a38-824c-0d9774db54d2', '87277d11-24c6-481c-a51d-40adcad0891b', '0fce5919-c945-43ca-939f-915c6b89cb3a', 'a46762ea-9262-4943-b95c-171fb98f2fcd', '20eb66ac-23ee-466e-add7-888c98071a0f', 'f810b99c-446e-4272-88e0-4722ba133474', '6379e858-f6fc-44d3-aa5d-edc338c1683e')\r\n      Process 4926: DELETE FROM \"silk_response\" WHERE \"silk_response\".\"request_id\" IN ('80a46b34-8b35-455b-9b12-ab8ae03121d3', 'a46762ea-9262-4943-b95c-171fb98f2fcd', '2d9bbe1d-43e2-4a17-afe6-8ddcd8b2036c', '45a61cf6-45c2-4a38-824c-0d9774db54d2', 'ed59a924-4830-4b92-8e56-cea6cd2ea273')\r\n HINT:  See server log for query details.\r\n STATEMENT:  DELETE FROM \"silk_response\" WHERE \"silk_response\".\"request_id\" IN ('80a46b34-8b35-455b-9b12-ab8ae03121d3', '87277d11-24c6-481c-a51d-40adcad0891b', 'a46762ea-9262-4943-b95c-171fb98f2fcd', '2d9bbe1d-43e2-4a17-afe6-8ddcd8b2036c', '45a61cf6-45c2-4a38-824c-0d9774db54d2', 'f810b99c-446e-4272-88e0-4722ba133474')\r\n ERROR:  deadlock detected\r\n DETAIL:  Process 4926 waits for AccessExclusiveLock on tuple (9,89) of relation 21806 of database 20072; blocked by process 4929.\r\n      Process 4929 waits for ShareLock on transaction 94641; blocked by process 4928.\r\n      Process 4928 waits for ShareLock on transaction 94582; blocked by process 4926.\r\n      Process 4926: DELETE FROM \"silk_response\" WHERE \"silk_response\".\"request_id\" IN ('80a46b34-8b35-455b-9b12-ab8ae03121d3', 'a46762ea-9262-4943-b95c-171fb98f2fcd', '2d9bbe1d-43e2-4a17-afe6-8ddcd8b2036c', '45a61cf6-45c2-4a38-824c-0d9774db54d2', 'ed59a924-4830-4b92-8e56-cea6cd2ea273')\r\n      Process 4929: DELETE FROM \"silk_response\" WHERE \"silk_response\".\"request_id\" IN ('a46762ea-9262-4943-b95c-171fb98f2fcd', '80a46b34-8b35-455b-9b12-ab8ae03121d3', '2d9bbe1d-43e2-4a17-afe6-8ddcd8b2036c', 'f810b99c-446e-4272-88e0-4722ba133474', '87277d11-24c6-481c-a51d-40adcad0891b', '45a61cf6-45c2-4a38-824c-0d9774db54d2')\r\n      Process 4928: DELETE FROM \"silk_response\" WHERE \"silk_response\".\"request_id\" IN ('80a46b34-8b35-455b-9b12-ab8ae03121d3', '45a61cf6-45c2-4a38-824c-0d9774db54d2', '87277d11-24c6-481c-a51d-40adcad0891b', '0fce5919-c945-43ca-939f-915c6b89cb3a', 'a46762ea-9262-4943-b95c-171fb98f2fcd', '20eb66ac-23ee-466e-add7-888c98071a0f', 'f810b99c-446e-4272-88e0-4722ba133474', '6379e858-f6fc-44d3-aa5d-edc338c1683e')\r\n HINT:  See server log for query details.\r\n STATEMENT:  DELETE FROM \"silk_response\" WHERE \"silk_response\".\"request_id\" IN ('80a46b34-8b35-455b-9b12-ab8ae03121d3', 'a46762ea-9262-4943-b95c-171fb98f2fcd', '2d9bbe1d-43e2-4a17-afe6-8ddcd8b2036c', '45a61cf6-45c2-4a38-824c-0d9774db54d2', 'ed59a924-4830-4b92-8e56-cea6cd2ea273')\r\n...\r\n```\r\n"}}
{"actor":{"display_login":"bj00rn"}}