{"action":"created","issue":{"url":"https://api.github.com/repos/polaris-gslb/polaris-gslb/issues/46","repository_url":"https://api.github.com/repos/polaris-gslb/polaris-gslb","labels_url":"https://api.github.com/repos/polaris-gslb/polaris-gslb/issues/46/labels{/name}","comments_url":"https://api.github.com/repos/polaris-gslb/polaris-gslb/issues/46/comments","events_url":"https://api.github.com/repos/polaris-gslb/polaris-gslb/issues/46/events","html_url":"https://github.com/polaris-gslb/polaris-gslb/issues/46","id":306034077,"number":46,"title":"polaris_health.tracker: failed to write generic state to the shared memory","user":{"login":"mattackerman808","id":290456,"avatar_url":"https://avatars1.githubusercontent.com/u/290456?v=4","gravatar_id":"","url":"https://api.github.com/users/mattackerman808","html_url":"https://github.com/mattackerman808","followers_url":"https://api.github.com/users/mattackerman808/followers","following_url":"https://api.github.com/users/mattackerman808/following{/other_user}","gists_url":"https://api.github.com/users/mattackerman808/gists{/gist_id}","starred_url":"https://api.github.com/users/mattackerman808/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mattackerman808/subscriptions","organizations_url":"https://api.github.com/users/mattackerman808/orgs","repos_url":"https://api.github.com/users/mattackerman808/repos","events_url":"https://api.github.com/users/mattackerman808/events{/privacy}","received_events_url":"https://api.github.com/users/mattackerman808/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-03-16T18:31:24Z","updated_at":"2018-03-16T18:59:06Z","closed_at":null,"author_association":"NONE","body":"Under high usage of polaris-health I am seeing that warning log continually. A Polaris-health restart seems to fix it for awhile and then it starts again.\r\n\r\nI am not seeing any other symptom of this error but I am assuming it's bad as it has to do with the state tracker/change code in the tracker submodule.\r\n\r\nI have been attempting to debug this by adding logging but nothing useful is getting returned.\r\n\r\nI added the following:\r\n\r\n        # push generic form of the state\r\n        # add timestampt to the object\r\n        generic_form['timestamp'] = STATE_TIMESTAMP\r\n        val = self.sm.set(config.BASE['SHARED_MEM_GENERIC_STATE_KEY'],\r\n                           generic_form)\r\n        if val is True:\r\n            pushes_ok += 1\r\n        else:\r\n            log_msg = ('failed to write generic state to the shared memory')\r\n            LOG.warning(log_msg)\r\n            LOG.warning(val)\r\n            LOG.warning(config.BASE['SHARED_MEM_GENERIC_STATE_KEY'],generic_form)\r\n\r\nAnd the output is this when the issue is happening:\r\n\r\n2018-03-16T11:20:49.544714-07:00  2018-03-16 11:20:49,544 [WARNING] polaris_health.tracker: failed to write generic state to the shared memory\r\n2018-03-16T11:20:49.544788-07:00  2018-03-16 11:20:49,544 [WARNING] polaris_health.tracker: 0\r\n2018-03-16T11:20:49.544893-07:00  2018-03-16 11:20:49,544 [WARNING] polaris_health.tracker: polaris_health:generic_state\r\n2018-03-16T11:20:50.191790-07:00  2018-03-16 11:20:50,191 [WARNING] polaris_health.tracker: failed to write generic state to the shared memory\r\n2018-03-16T11:20:50.191881-07:00  2018-03-16 11:20:50,191 [WARNING] polaris_health.tracker: 0\r\n2018-03-16T11:20:50.191985-07:00  2018-03-16 11:20:50,191 [WARNING] polaris_health.tracker: polaris_health:generic_state\r\n2018-03-16T11:20:50.868508-07:00  2018-03-16 11:20:50,868 [WARNING] polaris_health.tracker: failed to write generic state to the shared memory\r\n2018-03-16T11:20:50.868582-07:00  2018-03-16 11:20:50,868 [WARNING] polaris_health.tracker: 0\r\n2018-03-16T11:20:50.868696-07:00  2018-03-16 11:20:50,868 [WARNING] polaris_health.tracker: polaris_health:generic_state\r\n\r\nPlease let me know how I can troubleshoot this further or why it gets into this state."},"comment":{"url":"https://api.github.com/repos/polaris-gslb/polaris-gslb/issues/comments/373813115","html_url":"https://github.com/polaris-gslb/polaris-gslb/issues/46#issuecomment-373813115","issue_url":"https://api.github.com/repos/polaris-gslb/polaris-gslb/issues/46","id":373813115,"user":{"login":"mattackerman808","id":290456,"avatar_url":"https://avatars1.githubusercontent.com/u/290456?v=4","gravatar_id":"","url":"https://api.github.com/users/mattackerman808","html_url":"https://github.com/mattackerman808","followers_url":"https://api.github.com/users/mattackerman808/followers","following_url":"https://api.github.com/users/mattackerman808/following{/other_user}","gists_url":"https://api.github.com/users/mattackerman808/gists{/gist_id}","starred_url":"https://api.github.com/users/mattackerman808/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mattackerman808/subscriptions","organizations_url":"https://api.github.com/users/mattackerman808/orgs","repos_url":"https://api.github.com/users/mattackerman808/repos","events_url":"https://api.github.com/users/mattackerman808/events{/privacy}","received_events_url":"https://api.github.com/users/mattackerman808/received_events","type":"User","site_admin":false},"created_at":"2018-03-16T18:59:06Z","updated_at":"2018-03-16T18:59:06Z","author_association":"NONE","body":"$ bin/polaris-memcache-control get-generic-state |wc -c\r\n2123921"}}
{"actor":{"display_login":"mattackerman808"}}