{"action":"created","issue":{"url":"https://api.github.com/repos/ncabatoff/process-exporter/issues/51","repository_url":"https://api.github.com/repos/ncabatoff/process-exporter","labels_url":"https://api.github.com/repos/ncabatoff/process-exporter/issues/51/labels{/name}","comments_url":"https://api.github.com/repos/ncabatoff/process-exporter/issues/51/comments","events_url":"https://api.github.com/repos/ncabatoff/process-exporter/issues/51/events","html_url":"https://github.com/ncabatoff/process-exporter/issues/51","id":348295660,"node_id":"MDU6SXNzdWUzNDgyOTU2NjA=","number":51,"title":"Possible regression in v0.3.1 amd64 release on Debian testing","user":{"login":"mike42","id":2080552,"node_id":"MDQ6VXNlcjIwODA1NTI=","avatar_url":"https://avatars2.githubusercontent.com/u/2080552?v=4","gravatar_id":"","url":"https://api.github.com/users/mike42","html_url":"https://github.com/mike42","followers_url":"https://api.github.com/users/mike42/followers","following_url":"https://api.github.com/users/mike42/following{/other_user}","gists_url":"https://api.github.com/users/mike42/gists{/gist_id}","starred_url":"https://api.github.com/users/mike42/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mike42/subscriptions","organizations_url":"https://api.github.com/users/mike42/orgs","repos_url":"https://api.github.com/users/mike42/repos","events_url":"https://api.github.com/users/mike42/events{/privacy}","received_events_url":"https://api.github.com/users/mike42/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2018-08-07T12:17:28Z","updated_at":"2018-08-08T10:09:36Z","closed_at":"2018-08-08T10:09:36Z","author_association":"NONE","body":"I currently use this package on a fairly up-to-date Debian testing `amd64` system, with the older `process-exporter-0.2.12.linux-amd64` release.\r\n\r\nKernel version is:\r\n\r\n```\r\n$ uname -a\r\nLinux [hostname] 4.17.0-1-amd64 #1 SMP Debian 4.17.8-1 (2018-07-20) x86_64 GNU/Linux\r\n```\r\n\r\nVersion 0.2.12 works on this setup with a simple configuration:\r\n\r\n```yaml\r\nprocess_names:\r\n  - name: \"{{.Comm}}\"\r\n    cmdline: \r\n    - '.+'\r\n```\r\n\r\nAnd this command line:\r\n\r\n```\r\n./process-exporter --config.path foo.yml\r\n```\r\n\r\nMany, many process metrics are then produced, which is fantastic.\r\n\r\n```\r\n$ curl http://localhost:9256/metrics 2> /dev/null | wc -l \r\n2084\r\n```\r\n\r\nNow if I shut that down and repeat the same process with `process-exporter-0.3.1.linux-amd64`, I get no metrics.\r\n\r\n```\r\n$ curl http://localhost:9256/metrics 2> /dev/null | wc -l \r\n129\r\n```\r\n\r\nThe output indicates some issues were encountered, but I can't see any debug options which might explain what those issues were:\r\n\r\n```\r\nnamedprocess_scrape_errors 0\r\n# HELP namedprocess_scrape_partial_errors incremented each time a tracked proc's metrics collection fails partially, e.g. unreadable I/O stats\r\n# TYPE namedprocess_scrape_partial_errors counter\r\nnamedprocess_scrape_partial_errors 0\r\n# HELP namedprocess_scrape_procread_errors incremented each time a proc's metrics collection fails\r\n# TYPE namedprocess_scrape_procread_errors counter\r\nnamedprocess_scrape_procread_errors 271\r\n```\r\n\r\nI next attempted to use `strace` to see if invalid files were being opened.\r\n\r\n0.2.12 starts like this:\r\n\r\n```\r\n$ strace ./process-exporter --config.path foo.yml 2>&1 | grep open\r\nopenat(AT_FDCWD, \"/proc/sys/net/core/somaxconn\", O_RDONLY|O_CLOEXEC) = 3\r\nopenat(AT_FDCWD, \"/proc/stat\", O_RDONLY|O_CLOEXEC) = 3\r\nopenat(AT_FDCWD, \"foo.yml\", O_RDONLY|O_CLOEXEC) = 3\r\nopenat(AT_FDCWD, \"/etc//localtime\", O_RDONLY) = 3\r\nopenat(AT_FDCWD, \"/proc/stat\", O_RDONLY|O_CLOEXEC) = 3\r\nopenat(AT_FDCWD, \"/proc\", O_RDONLY|O_CLOEXEC) = 3\r\nopenat(AT_FDCWD, \"/proc/1/stat\", O_RDONLY|O_CLOEXEC) = 3\r\nopenat(AT_FDCWD, \"/proc/1/io\", O_RDONLY|O_CLOEXEC) = -1 EACCES (Permission denied)\r\nopenat(AT_FDCWD, \"/proc/1/fd\", O_RDONLY|O_CLOEXEC) = -1 EACCES (Permission denied)\r\nopenat(AT_FDCWD, \"/proc/1/limits\", O_RDONLY|O_CLOEXEC) = 3\r\nopenat(AT_FDCWD, \"/proc/1/cmdline\", O_RDONLY|O_CLOEXEC) = 3\r\nopenat(AT_FDCWD, \"/proc/2/stat\", O_RDONLY|O_CLOEXEC) = 3\r\nopenat(AT_FDCWD, \"/proc/2/io\", O_RDONLY|O_CLOEXEC) = -1 EACCES (Permission denied)\r\nopenat(AT_FDCWD, \"/proc/2/fd\", O_RDONLY|O_CLOEXEC) = -1 EACCES (Permission denied)\r\nopenat(AT_FDCWD, \"/proc/2/limits\", O_RDONLY|O_CLOEXEC) = 3\r\nopenat(AT_FDCWD, \"/proc/2/cmdline\", O_RDONLY|O_CLOEXEC) = 3\r\n```\r\n\r\nWhile 0.3.1 starts like this:\r\n\r\n```\r\n$ strace ./process-exporter --config.path foo.yml 2>&1 | grep open\r\nopenat(AT_FDCWD, \"/etc/ld.so.cache\", O_RDONLY|O_CLOEXEC) = 3\r\nopenat(AT_FDCWD, \"/lib/x86_64-linux-gnu/libpthread.so.0\", O_RDONLY|O_CLOEXEC) = 3\r\nopenat(AT_FDCWD, \"/lib/x86_64-linux-gnu/libc.so.6\", O_RDONLY|O_CLOEXEC) = 3\r\nopenat(AT_FDCWD, \"/proc/sys/net/core/somaxconn\", O_RDONLY|O_CLOEXEC) = 3\r\nopenat(AT_FDCWD, \"/proc/stat\", O_RDONLY|O_CLOEXEC) = 3\r\nopenat(AT_FDCWD, \"foo.yml\", O_RDONLY|O_CLOEXEC) = 3\r\nopenat(AT_FDCWD, \"/etc//localtime\", O_RDONLY) = 3\r\nopenat(AT_FDCWD, \"/proc/stat\", O_RDONLY|O_CLOEXEC) = 3\r\nopenat(AT_FDCWD, \"/proc\", O_RDONLY|O_CLOEXEC) = 3\r\nopenat(AT_FDCWD, \"/proc/1/stat\", O_RDONLY|O_CLOEXEC) = 3\r\nopenat(AT_FDCWD, \"/proc/1/status\", O_RDONLY|O_CLOEXEC) = 3\r\nopenat(AT_FDCWD, \"/proc/2/stat\", O_RDONLY|O_CLOEXEC) = 3\r\nopenat(AT_FDCWD, \"/proc/2/status\", O_RDONLY|O_CLOEXEC) = 3\r\n```\r\n\r\nWith no new files feeding into the erroneous output, I'm not really sure how to debug further, so I'm just sticking to the older version for now. I can post examples of `/proc/stat`, `/proc/1/stat` and `/proc/1/status` or more complete output if you think it would be relevant.\r\n\r\nAside, #50 is reported to be affected by OS version, not `process-exporter` version, I'm assuming that these are different issues."},"comment":{"url":"https://api.github.com/repos/ncabatoff/process-exporter/issues/comments/411356046","html_url":"https://github.com/ncabatoff/process-exporter/issues/51#issuecomment-411356046","issue_url":"https://api.github.com/repos/ncabatoff/process-exporter/issues/51","id":411356046,"node_id":"MDEyOklzc3VlQ29tbWVudDQxMTM1NjA0Ng==","user":{"login":"mike42","id":2080552,"node_id":"MDQ6VXNlcjIwODA1NTI=","avatar_url":"https://avatars2.githubusercontent.com/u/2080552?v=4","gravatar_id":"","url":"https://api.github.com/users/mike42","html_url":"https://github.com/mike42","followers_url":"https://api.github.com/users/mike42/followers","following_url":"https://api.github.com/users/mike42/following{/other_user}","gists_url":"https://api.github.com/users/mike42/gists{/gist_id}","starred_url":"https://api.github.com/users/mike42/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/mike42/subscriptions","organizations_url":"https://api.github.com/users/mike42/orgs","repos_url":"https://api.github.com/users/mike42/repos","events_url":"https://api.github.com/users/mike42/events{/privacy}","received_events_url":"https://api.github.com/users/mike42/received_events","type":"User","site_admin":false},"created_at":"2018-08-08T10:09:36Z","updated_at":"2018-08-08T10:09:36Z","author_association":"NONE","body":"I've just tested the 0.3.2 release, and it gives a successful scrape on my system, so I'll close this off. Thanks for your work on this exporter!\r\n\r\nI've attached the scrape output ([metrics.txt](https://github.com/ncabatoff/process-exporter/files/2269830/metrics.txt)) and included some `/proc` snippets below, in case it can be used to add a unit test.\r\n\r\n--\r\n\r\n```\r\n# cat /proc/188/stat\r\n188 (bash) S 0 188 188 34818 256 4194560 1190 10638 9 93 4 2 165 41 20 0 1 0 87615 20434944 930 18446744073709551615 4194304 5242124 140720909478560 0 0 0 134283264 3670020 1266777851 1 0 0 17 2 0 0 0 0 0 7341384 7388228 34840576 140720909482127 140720909482137 140720909482137 140720909484014 0\r\n# cat /proc/188/status\r\nName:\tbash\r\nUmask:\t0022\r\nState:\tS (sleeping)\r\nTgid:\t188\r\nNgid:\t0\r\nPid:\t188\r\nPPid:\t0\r\nTracerPid:\t0\r\nUid:\t0\t0\t0\t0\r\nGid:\t0\t0\t0\t0\r\nFDSize:\t256\r\nGroups:\t0 \r\nNStgid:\t188\r\nNSpid:\t188\r\nNSpgid:\t188\r\nNSsid:\t188\r\nVmPeak:\t   19956 kB\r\nVmSize:\t   19956 kB\r\nVmLck:\t       0 kB\r\nVmPin:\t       0 kB\r\nVmHWM:\t    3720 kB\r\nVmRSS:\t    3720 kB\r\nRssAnon:\t     608 kB\r\nRssFile:\t    3112 kB\r\nRssShmem:\t       0 kB\r\nVmData:\t     604 kB\r\nVmStk:\t     132 kB\r\nVmExe:\t    1024 kB\r\nVmLib:\t    2124 kB\r\nVmPTE:\t      80 kB\r\nVmSwap:\t       0 kB\r\nHugetlbPages:\t       0 kB\r\nCoreDumping:\t0\r\nThreads:\t1\r\nSigQ:\t0/127087\r\nSigPnd:\t0000000000000000\r\nShdPnd:\t0000000000000000\r\nSigBlk:\t0000000008010000\r\nSigIgn:\t0000000000380004\r\nSigCgt:\t000000004b817efb\r\nCapInh:\t0000000000000000\r\nCapPrm:\t0000003cfdfcffff\r\nCapEff:\t0000003cfdfcffff\r\nCapBnd:\t0000003cfdfcffff\r\nCapAmb:\t0000000000000000\r\nNoNewPrivs:\t0\r\nSeccomp:\t2\r\nSpeculation_Store_Bypass:\tthread force mitigated\r\nCpus_allowed:\tff\r\nCpus_allowed_list:\t0-7\r\nMems_allowed:\t00000000,00000001\r\nMems_allowed_list:\t0\r\nvoluntary_ctxt_switches:\t349\r\nnonvoluntary_ctxt_switches:\t41\r\n```\r\n"}}
{"actor":{"display_login":"mike42"}}