{"action":"created","issue":{"url":"https://api.github.com/repos/DataDog/dd-agent/issues/2930","repository_url":"https://api.github.com/repos/DataDog/dd-agent","labels_url":"https://api.github.com/repos/DataDog/dd-agent/issues/2930/labels{/name}","comments_url":"https://api.github.com/repos/DataDog/dd-agent/issues/2930/comments","events_url":"https://api.github.com/repos/DataDog/dd-agent/issues/2930/events","html_url":"https://github.com/DataDog/dd-agent/issues/2930","id":183833764,"number":2930,"title":"5.9.1: docker_daemon _pid error","user":{"login":"markine","id":9396824,"avatar_url":"https://avatars.githubusercontent.com/u/9396824?v=3","gravatar_id":"","url":"https://api.github.com/users/markine","html_url":"https://github.com/markine","followers_url":"https://api.github.com/users/markine/followers","following_url":"https://api.github.com/users/markine/following{/other_user}","gists_url":"https://api.github.com/users/markine/gists{/gist_id}","starred_url":"https://api.github.com/users/markine/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/markine/subscriptions","organizations_url":"https://api.github.com/users/markine/orgs","repos_url":"https://api.github.com/users/markine/repos","events_url":"https://api.github.com/users/markine/events{/privacy}","received_events_url":"https://api.github.com/users/markine/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":2,"created_at":"2016-10-19T00:03:21Z","updated_at":"2016-10-27T12:52:24Z","closed_at":null,"body":"Deploying dd-agent 5.9.1 using the docker-dd-agent wrapper as a Daemonset in Kubernetes 1.2.4 generates an error in docker_daemon:\r\n\r\n```\r\nroot@dd-agent-vvvec:/# /etc/init.d/datadog-agent info -v\r\n2016-10-19 00:00:33,103 | WARNING | dd.collector | utils.service_discovery.config(config.py:32) | No configuration backend provided for service discovery. Only auto config templates will be used.\r\n===================\r\nCollector (v 5.9.1)\r\n===================\r\n\r\n  Status date: 2016-10-19 00:00:16 (16s ago)\r\n  Pid: 16\r\n  Platform: Linux-3.16.0-4-amd64-x86_64-with-debian-8.6\r\n  Python Version: 2.7.12, 64bit\r\n  Logs: <stderr>, /var/log/datadog/collector.log\r\n\r\n  Clocks\r\n  ======\r\n  \r\n    NTP offset: 0.0045 s\r\n    System UTC time: 2016-10-19 00:00:33.348994\r\n  \r\n  Paths\r\n  =====\r\n  \r\n    conf.d: /etc/dd-agent/conf.d\r\n    checks.d: /opt/datadog-agent/agent/checks.d\r\n  \r\n  Hostnames\r\n  =========\r\n  \r\n    ec2-hostname: ip-172-20-0-5.ec2.internal\r\n    local-ipv4: 172.20.0.5\r\n    local-hostname: ip-172-20-0-5.ec2.internal\r\n    socket-hostname: dd-agent-vvvec\r\n    public-hostname: XXXXXXXXXXXX.compute-1.amazonaws.com\r\n    hostname: ip-172-20-0-5.ec2.internal\r\n    instance-id: i-XXXXXXX\r\n    public-ipv4: XXXXXXXXXX\r\n    socket-fqdn: dd-agent-vvvec\r\n  \r\n  Checks\r\n  ======\r\n  \r\n    ntp\r\n    ---\r\n      - Collected 0 metrics, 0 events & 1 service check\r\n  \r\n    disk\r\n    ----\r\n      - instance #0 [OK]\r\n      - Collected 32 metrics, 0 events & 1 service check\r\n  \r\n    docker_daemon\r\n    -------------\r\n      - instance #0 [ERROR]: \"'_pid'\"\r\n        Traceback (most recent call last):\r\n          File \"/opt/datadog-agent/agent/checks/__init__.py\", line 771, in run\r\n            self.check(copy.deepcopy(instance))\r\n          File \"/opt/datadog-agent/agent/checks.d/docker_daemon.py\", line 254, in check\r\n            self._report_performance_metrics(containers_by_id)\r\n          File \"/opt/datadog-agent/agent/checks.d/docker_daemon.py\", line 525, in _report_performance_metrics\r\n            self._report_cgroup_metrics(container, tags)\r\n          File \"/opt/datadog-agent/agent/checks.d/docker_daemon.py\", line 543, in _report_cgroup_metrics\r\n            stat_file = self._get_cgroup_from_proc(cgroup[\"cgroup\"], container['_pid'], cgroup['file'])\r\n        KeyError: '_pid'\r\n        \r\n      - Collected 51 metrics, 0 events & 2 service checks\r\n  \r\n    kubernetes\r\n    ----------\r\n      - instance #0 [OK]\r\n      - Collected 206 metrics, 0 events & 5 service checks\r\n  \r\n  \r\n  Emitters\r\n  ========\r\n  \r\n    - http_emitter [OK]\r\n\r\n===================\r\nDogstatsd (v 5.9.1)\r\n===================\r\n\r\n  Status date: 2016-10-19 00:00:31 (1s ago)\r\n  Pid: 13\r\n  Platform: Linux-3.16.0-4-amd64-x86_64-with-debian-8.6\r\n  Python Version: 2.7.12, 64bit\r\n  Logs: <stderr>, /var/log/datadog/dogstatsd.log\r\n\r\n  Flush count: 4\r\n  Packet Count: 0\r\n  Packets per second: 0.0\r\n  Metric count: 1\r\n  Event count: 0\r\n  Service check count: 0\r\n\r\n===================\r\nForwarder (v 5.9.1)\r\n===================\r\n\r\n  Status date: 2016-10-19 00:00:31 (2s ago)\r\n  Pid: 15\r\n  Platform: Linux-3.16.0-4-amd64-x86_64-with-debian-8.6\r\n  Python Version: 2.7.12, 64bit\r\n  Logs: <stderr>, /var/log/datadog/forwarder.log\r\n\r\n  Queue Size: 447 bytes\r\n  Queue Length: 1\r\n  Flush Count: 13\r\n  Transactions received: 5\r\n  Transactions flushed: 4\r\n  Transactions rejected: 0\r\n \r\n```"},"comment":{"url":"https://api.github.com/repos/DataDog/dd-agent/issues/comments/256632094","html_url":"https://github.com/DataDog/dd-agent/issues/2930#issuecomment-256632094","issue_url":"https://api.github.com/repos/DataDog/dd-agent/issues/2930","id":256632094,"user":{"login":"arobson","id":115508,"avatar_url":"https://avatars.githubusercontent.com/u/115508?v=3","gravatar_id":"","url":"https://api.github.com/users/arobson","html_url":"https://github.com/arobson","followers_url":"https://api.github.com/users/arobson/followers","following_url":"https://api.github.com/users/arobson/following{/other_user}","gists_url":"https://api.github.com/users/arobson/gists{/gist_id}","starred_url":"https://api.github.com/users/arobson/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/arobson/subscriptions","organizations_url":"https://api.github.com/users/arobson/orgs","repos_url":"https://api.github.com/users/arobson/repos","events_url":"https://api.github.com/users/arobson/events{/privacy}","received_events_url":"https://api.github.com/users/arobson/received_events","type":"User","site_admin":false},"created_at":"2016-10-27T12:52:24Z","updated_at":"2016-10-27T12:52:24Z","body":"Getting this consistently on about half of our servers running kubernetes 1.4.0, in AWS using the `datadog/docker-dd-agent:latest` (agent reports version 5.9.1) image with the recommended daemonset manifest. This only recently started happening so it makes me wonder if it's the result of a recent change in the agent that we got as a result of deploying updates to our configuration that also pulled new versions of that image/tag."}}
{"actor":{"display_login":"arobson"}}