{"action":"created","issue":{"url":"https://api.github.com/repos/openvstorage/framework/issues/942","repository_url":"https://api.github.com/repos/openvstorage/framework","labels_url":"https://api.github.com/repos/openvstorage/framework/issues/942/labels{/name}","comments_url":"https://api.github.com/repos/openvstorage/framework/issues/942/comments","events_url":"https://api.github.com/repos/openvstorage/framework/issues/942/events","html_url":"https://github.com/openvstorage/framework/issues/942","id":178541945,"number":942,"title":"It seems pyrakoon also raises ArakoonSockRecvError these days","user":{"login":"khenderick","id":1453214,"avatar_url":"https://avatars.githubusercontent.com/u/1453214?v=3","gravatar_id":"","url":"https://api.github.com/users/khenderick","html_url":"https://github.com/khenderick","followers_url":"https://api.github.com/users/khenderick/followers","following_url":"https://api.github.com/users/khenderick/following{/other_user}","gists_url":"https://api.github.com/users/khenderick/gists{/gist_id}","starred_url":"https://api.github.com/users/khenderick/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/khenderick/subscriptions","organizations_url":"https://api.github.com/users/khenderick/orgs","repos_url":"https://api.github.com/users/khenderick/repos","events_url":"https://api.github.com/users/khenderick/events{/privacy}","received_events_url":"https://api.github.com/users/khenderick/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2016-09-22T07:35:50Z","updated_at":"2016-09-22T08:28:46Z","closed_at":null,"body":"It seems pyrakoon also raises ```ArakoonSockRecvError``` these days. We should also incorporate the retry for that error."},"comment":{"url":"https://api.github.com/repos/openvstorage/framework/issues/comments/248841775","html_url":"https://github.com/openvstorage/framework/issues/942#issuecomment-248841775","issue_url":"https://api.github.com/repos/openvstorage/framework/issues/942","id":248841775,"user":{"login":"khenderick","id":1453214,"avatar_url":"https://avatars.githubusercontent.com/u/1453214?v=3","gravatar_id":"","url":"https://api.github.com/users/khenderick","html_url":"https://github.com/khenderick","followers_url":"https://api.github.com/users/khenderick/followers","following_url":"https://api.github.com/users/khenderick/following{/other_user}","gists_url":"https://api.github.com/users/khenderick/gists{/gist_id}","starred_url":"https://api.github.com/users/khenderick/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/khenderick/subscriptions","organizations_url":"https://api.github.com/users/khenderick/orgs","repos_url":"https://api.github.com/users/khenderick/repos","events_url":"https://api.github.com/users/khenderick/events{/privacy}","received_events_url":"https://api.github.com/users/khenderick/received_events","type":"User","site_admin":false},"created_at":"2016-09-22T08:28:46Z","updated_at":"2016-09-22T08:28:46Z","body":"```\r\n2016-09-22 09:31:31 97400 +0200 - ovs100 - 26213/140079970371392 - extensions/pyrakoon_client - 274 - DEBUG - Error during arakoon call get, retry\r\n2016-09-22 09:31:31 97400 +0200 - ovs100 - 26209/140079970371392 - arakoon_client/pyrakoon - 269 - ERROR - timed out: Error while reading socket\r\nTraceback (most recent call last):\r\n  File \"/opt/OpenvStorage/ovs/extensions/db/arakoon/pyrakoon/pyrakoon/compat.py\", line 1342, in read\r\n    data = self._socket.recv(bytes_remaining)\r\ntimeout: timed out\r\n2016-09-22 09:31:31 97500 +0200 - ovs100 - 26209/140079970371392 - extensions/pyrakoon_client - 270 - ERROR - Error during get. Process 26209, thread 140079970371392, clientid 5155680\r\n2016-09-22 09:31:31 97600 +0200 - ovs100 - 17887/140079970371392 - arakoon_client/pyrakoon - 44 - ERROR - timed out: Error while reading socket\r\nTraceback (most recent call last):\r\n  File \"/opt/OpenvStorage/ovs/extensions/db/arakoon/pyrakoon/pyrakoon/compat.py\", line 1342, in read\r\n    data = self._socket.recv(bytes_remaining)\r\ntimeout: timed out\r\n2016-09-22 09:31:31 97600 +0200 - ovs100 - 17887/140079970371392 - extensions/pyrakoon_client - 45 - ERROR - Error during get. Process 17887, thread 140079970371392, clientid 5155680\r\n2016-09-22 09:31:31 97700 +0200 - ovs100 - 17887/140079970371392 - celery/celery.redirected - 47 - WARNING - 2016-09-22 09:31:31 97700 +0200 - ovs100 - 17887/140079970371392 - extensions/volatile mutex - 46 - WARNING - A lock on ovs_lock_voldrv_event_disk_f8b0edd5-5d09-4f3\r\n1-b92d-d6cbe46723ac was kept for 63.5493280888 sec\r\n2016-09-22 09:31:31 97700 +0200 - ovs100 - 26188/140079970371392 - extensions/pyrakoon_client - 250 - DEBUG - Error during arakoon call get, retry\r\n2016-09-22 09:31:31 98200 +0200 - ovs100 - 17866/140079970371392 - celery/celery.worker.job - 387 - ERROR - Task ovs.vdisk.resize_from_voldrv[24d4c2fc-4419-478c-a2b7-5589e9fc3c4e] raised unexpected: ArakoonSockRecvError('Error while receiving data from socket',)\r\nTraceback (most recent call last):\r\n  File \"/usr/lib/python2.7/dist-packages/celery/app/trace.py\", line 240, in trace_task\r\n    R = retval = fun(*args, **kwargs)\r\n  File \"/usr/lib/python2.7/dist-packages/celery/app/trace.py\", line 438, in __protected_call__\r\n    return self.run(*args, **kwargs)\r\n  File \"/opt/OpenvStorage/ovs/lib/helpers/decorators.py\", line 69, in new_function\r\n    return function(*args, **kwargs)\r\n  File \"/opt/OpenvStorage/ovs/lib/vdisk.py\", line 178, in resize_from_voldrv\r\n    VDiskController._set_vdisk_metadata_pagecache_size(vdisk)\r\n  File \"/opt/OpenvStorage/ovs/lib/vdisk.py\", line 1154, in _set_vdisk_metadata_pagecache_size\r\n    storagedriver_config.load()\r\n  File \"/opt/OpenvStorage/ovs/extensions/storageserver/storagedriver.py\", line 257, in load\r\n    self.configuration = json.loads(Configuration.get(self.key, raw=True))\r\n  File \"/opt/OpenvStorage/ovs/extensions/generic/configuration.py\", line 112, in get\r\n    data = Configuration._get(key_entries[0], raw)\r\n  File \"/opt/OpenvStorage/ovs/extensions/generic/configuration.py\", line 362, in _get\r\n    key=key)\r\n  File \"/opt/OpenvStorage/ovs/extensions/generic/configuration.py\", line 404, in _passthrough\r\n    return getattr(ArakoonConfiguration, method)(*args, **kwargs)\r\n  File \"/opt/OpenvStorage/ovs/extensions/db/arakoon/configuration.py\", line 119, in get\r\n    return client.get(key)\r\n  File \"/opt/OpenvStorage/ovs/extensions/db/arakoon/pyrakoon/client.py\", line 44, in new_function\r\n    return f(self, *args, **kw)\r\n  File \"/opt/OpenvStorage/ovs/extensions/db/arakoon/pyrakoon/client.py\", line 76, in get\r\n    return PyrakoonClient._try(self._identifier, self._client.get, key, consistency)\r\n  File \"/opt/OpenvStorage/ovs/extensions/db/arakoon/pyrakoon/client.py\", line 207, in _try\r\n    return_value = method(*args, **kwargs)\r\n  File \"<update_argspec>\", line 5, in get\r\n  File \"/opt/OpenvStorage/ovs/extensions/db/arakoon/pyrakoon/pyrakoon/compat.py\", line 145, in wrapped\r\n    return fun(*args, **kwargs)\r\n  File \"/opt/OpenvStorage/ovs/extensions/db/arakoon/pyrakoon/pyrakoon/compat.py\", line 128, in wrapped\r\n    return fun(*new_args)\r\n  File \"/opt/OpenvStorage/ovs/extensions/db/arakoon/pyrakoon/pyrakoon/compat.py\", line 246, in get\r\n    return self._client.get(key,consistency = consistency_)\r\n  File \"<update_argspec>\", line 5, in get\r\n  File \"/opt/OpenvStorage/ovs/extensions/db/arakoon/pyrakoon/pyrakoon/client/utils.py\", line 99, in wrapped\r\n    return self._process(message) #pylint: disable=W0212\r\n  File \"/opt/OpenvStorage/ovs/extensions/db/arakoon/pyrakoon/pyrakoon/compat.py\", line 1103, in _process\r\n    connection.read)\r\n  File \"/opt/OpenvStorage/ovs/extensions/db/arakoon/pyrakoon/pyrakoon/utils.py\", line 342, in read_blocking\r\n    value = read_fun(request.count)\r\n  File \"/opt/OpenvStorage/ovs/extensions/db/arakoon/pyrakoon/pyrakoon/compat.py\", line 1347, in read\r\n    raise ArakoonSockRecvError\r\nArakoonSockRecvError: Error while receiving data from socket\r\n2016-09-22 09:31:32 00800 +0200 - ovs100 - 17866/140079970371392 - celery/celery.worker.job - 388 - ERROR - Task ovs.vdisk.resize_from_voldrv[4bbf0ff3-f687-4072-8b50-11c4fd176f95] raised unexpected: ArakoonSockRecvError('Error while receiving data from socket',)\r\nTraceback (most recent call last):\r\n  File \"/usr/lib/python2.7/dist-packages/celery/app/trace.py\", line 240, in trace_task\r\n    R = retval = fun(*args, **kwargs)\r\n  File \"/usr/lib/python2.7/dist-packages/celery/app/trace.py\", line 438, in __protected_call__\r\n    return self.run(*args, **kwargs)\r\n  File \"/opt/OpenvStorage/ovs/lib/helpers/decorators.py\", line 56, in new_function\r\n    metadata = {'storagedriver': StorageDriverList.get_by_storagedriver_id(kwargs['storagedriver_id']).guid}\r\n  File \"/opt/OpenvStorage/ovs/dal/lists/storagedriverlist.py\", line 43, in get_by_storagedriver_id\r\n    'items': [('storagedriver_id', DataList.operator.EQUALS, storagedriver_id)]})\r\n  File \"/opt/OpenvStorage/ovs/dal/datalist.py\", line 82, in __init__\r\n    self._volatile = VolatileFactory.get_client()\r\n  File \"/opt/OpenvStorage/ovs/extensions/storage/volatilefactory.py\", line 38, in get_client\r\n    client_type = Configuration.get('/ovs/framework/stores|volatile')\r\n  File \"/opt/OpenvStorage/ovs/extensions/generic/configuration.py\", line 112, in get\r\n    data = Configuration._get(key_entries[0], raw)\r\n  File \"/opt/OpenvStorage/ovs/extensions/generic/configuration.py\", line 362, in _get\r\n    key=key)\r\n  File \"/opt/OpenvStorage/ovs/extensions/generic/configuration.py\", line 404, in _passthrough\r\n    return getattr(ArakoonConfiguration, method)(*args, **kwargs)\r\n  File \"/opt/OpenvStorage/ovs/extensions/db/arakoon/configuration.py\", line 119, in get\r\n    return client.get(key)\r\n  File \"/opt/OpenvStorage/ovs/extensions/db/arakoon/pyrakoon/client.py\", line 44, in new_function\r\n    return f(self, *args, **kw)\r\n  File \"/opt/OpenvStorage/ovs/extensions/db/arakoon/pyrakoon/client.py\", line 76, in get\r\n    return PyrakoonClient._try(self._identifier, self._client.get, key, consistency)\r\n  File \"/opt/OpenvStorage/ovs/extensions/db/arakoon/pyrakoon/client.py\", line 207, in _try\r\n    return_value = method(*args, **kwargs)\r\n  File \"<update_argspec>\", line 5, in get\r\n  File \"/opt/OpenvStorage/ovs/extensions/db/arakoon/pyrakoon/pyrakoon/compat.py\", line 145, in wrapped\r\n    return fun(*args, **kwargs)\r\n  File \"/opt/OpenvStorage/ovs/extensions/db/arakoon/pyrakoon/pyrakoon/compat.py\", line 128, in wrapped\r\n    return fun(*new_args)\r\n  File \"/opt/OpenvStorage/ovs/extensions/db/arakoon/pyrakoon/pyrakoon/compat.py\", line 246, in get\r\n    return self._client.get(key,consistency = consistency_)\r\n  File \"<update_argspec>\", line 5, in get\r\n  File \"/opt/OpenvStorage/ovs/extensions/db/arakoon/pyrakoon/pyrakoon/client/utils.py\", line 99, in wrapped\r\n    return self._process(message) #pylint: disable=W0212\r\n  File \"/opt/OpenvStorage/ovs/extensions/db/arakoon/pyrakoon/pyrakoon/compat.py\", line 1103, in _process\r\n    connection.read)\r\n  File \"/opt/OpenvStorage/ovs/extensions/db/arakoon/pyrakoon/pyrakoon/utils.py\", line 342, in read_blocking\r\n    value = read_fun(request.count)\r\n  File \"/opt/OpenvStorage/ovs/extensions/db/arakoon/pyrakoon/pyrakoon/compat.py\", line 1347, in read\r\n    raise ArakoonSockRecvError\r\nArakoonSockRecvError: Error while receiving data from socket\r\n2016-09-22 09:31:32 97600 +0200 - ovs100 - 26213/140079970371392 - extensions/pyrakoon_client - 275 - WARNING - Arakoon call get took 64.1s\r\n2016-09-22 09:31:32 97900 +0200 - ovs100 - 26188/140079970371392 - extensions/pyrakoon_client - 251 - WARNING - Arakoon call get took 61.06s\r\n```"}}
{"actor":{"display_login":"khenderick"}}