{"action":"created","issue":{"url":"https://api.github.com/repos/openshift/origin/issues/11795","repository_url":"https://api.github.com/repos/openshift/origin","labels_url":"https://api.github.com/repos/openshift/origin/issues/11795/labels{/name}","comments_url":"https://api.github.com/repos/openshift/origin/issues/11795/comments","events_url":"https://api.github.com/repos/openshift/origin/issues/11795/events","html_url":"https://github.com/openshift/origin/pull/11795","id":187443221,"number":11795,"title":"Memory monitoring for jenkins during extended tests","user":{"login":"jupierce","id":19783215,"avatar_url":"https://avatars.githubusercontent.com/u/19783215?v=3","gravatar_id":"","url":"https://api.github.com/users/jupierce","html_url":"https://github.com/jupierce","followers_url":"https://api.github.com/users/jupierce/followers","following_url":"https://api.github.com/users/jupierce/following{/other_user}","gists_url":"https://api.github.com/users/jupierce/gists{/gist_id}","starred_url":"https://api.github.com/users/jupierce/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jupierce/subscriptions","organizations_url":"https://api.github.com/users/jupierce/orgs","repos_url":"https://api.github.com/users/jupierce/repos","events_url":"https://api.github.com/users/jupierce/events{/privacy}","received_events_url":"https://api.github.com/users/jupierce/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":17,"created_at":"2016-11-04T21:33:45Z","updated_at":"2016-11-09T17:58:54Z","closed_at":null,"pull_request":{"url":"https://api.github.com/repos/openshift/origin/pulls/11795","html_url":"https://github.com/openshift/origin/pull/11795","diff_url":"https://github.com/openshift/origin/pull/11795.diff","patch_url":"https://github.com/openshift/origin/pull/11795.patch"},"body":"Example entry:\r\n```\r\nJenkins memory statistics at 2016-11-04 17:28:23.024556251 -0400 EDT\r\nUSER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND\r\n1000090+   476  0.0  0.0  47432  3212 ?        Rs   21:28   0:00 ps faux\r\n1000090+     1 30.5  2.7 6063144 438908 ?      Ssl  21:25   0:52 java -Xmx307m -Duser.home=/var/lib/jenkins -Dfile.encoding=UTF8 -jar /usr/lib/jenkins/jenkins.war\r\ncache 90386432\r\nrss 442425344\r\nrss_huge 0\r\nmapped_file 14123008\r\ndirty 20480\r\nwriteback 0\r\nswap 180346880\r\npgpgin 310560\r\npgpgout 180479\r\npgfault 194011\r\npgmajfault 4551\r\ninactive_anon 258682880\r\nactive_anon 255033344\r\ninactive_file 4714496\r\nactive_file 14323712\r\nunevictable 0\r\nhierarchical_memory_limit 536870912\r\nhierarchical_memsw_limit 9223372036854771712\r\ntotal_cache 90386432\r\ntotal_rss 442425344\r\ntotal_rss_huge 0\r\ntotal_mapped_file 14123008\r\ntotal_dirty 20480\r\ntotal_writeback 0\r\ntotal_swap 180346880\r\ntotal_pgpgin 310560\r\ntotal_pgpgout 180479\r\ntotal_pgfault 194011\r\ntotal_pgmajfault 4551\r\ntotal_inactive_anon 258682880\r\ntotal_active_anon 255033344\r\ntotal_inactive_file 4714496\r\ntotal_active_file 14323712\r\ntotal_unevictable 0\r\nrecent_rotated_anon 15485\r\nrecent_rotated_file 520\r\nrecent_scanned_anon 26309\r\nrecent_scanned_file 719\r\n```"},"comment":{"url":"https://api.github.com/repos/openshift/origin/issues/comments/259480409","html_url":"https://github.com/openshift/origin/pull/11795#issuecomment-259480409","issue_url":"https://api.github.com/repos/openshift/origin/issues/11795","id":259480409,"user":{"login":"jupierce","id":19783215,"avatar_url":"https://avatars.githubusercontent.com/u/19783215?v=3","gravatar_id":"","url":"https://api.github.com/users/jupierce","html_url":"https://github.com/jupierce","followers_url":"https://api.github.com/users/jupierce/followers","following_url":"https://api.github.com/users/jupierce/following{/other_user}","gists_url":"https://api.github.com/users/jupierce/gists{/gist_id}","starred_url":"https://api.github.com/users/jupierce/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jupierce/subscriptions","organizations_url":"https://api.github.com/users/jupierce/orgs","repos_url":"https://api.github.com/users/jupierce/repos","events_url":"https://api.github.com/users/jupierce/events{/privacy}","received_events_url":"https://api.github.com/users/jupierce/received_events","type":"User","site_admin":false},"created_at":"2016-11-09T17:58:54Z","updated_at":"2016-11-09T17:58:54Z","body":"https://ci.openshift.redhat.com/jenkins/job/test_pr_origin_extended/776/consoleFull\r\nShows memory growth and an OOM kill, but new logging statements were a moment too late to the party. Reduced the RSS threshold in response -- running tests again.\r\n```\r\nUnable to acquire Jenkins histogram: exit status 137\r\nNov  9 12:07:03.676: INFO: Running 'oc exec --config=/tmp/extended-test-jenkins-plugin-6kis9-r6b9i-user.kubeconfig --namespace=extended-test-jenkins-plugin-6kis9-r6b9i --namespace extended-test-jenkins-plugin-6kis9-r6b9i-jenkins jenkins-1-0sv14 -- jstack 1'\r\nNov  9 12:07:03.947: INFO: Error running &{/data/src/github.com/openshift/origin/_output/local/bin/linux/amd64/oc [oc exec --config=/tmp/extended-test-jenkins-plugin-6kis9-r6b9i-user.kubeconfig --namespace=extended-test-jenkins-plugin-6kis9-r6b9i --namespace extended-test-jenkins-plugin-6kis9-r6b9i-jenkins jenkins-1-0sv14 -- jstack 1] []   error: Internal error occurred: error executing command in container: container not found (\"jenkins\")\r\n error: Internal error occurred: error executing command in container: container not found (\"jenkins\")\r\n [] <nil> 0xc820a61660 exit status 1 <nil> true [0xc82002c9c8 0xc82002ca08 0xc82002ca08] [0xc82002c9c8 0xc82002ca08] [0xc82002c9d8 0xc82002ca00] [0xb187a0 0xb18900] 0xc820fdf920}:\r\nerror: Internal error occurred: error executing command in container: container not found (\"jenkins\")\r\nUnable to acquire Jenkins thread dump: exit status 1\r\nNov  9 12:07:03.947: INFO: Running 'oc exec --config=/tmp/extended-test-jenkins-plugin-6kis9-r6b9i-user.kubeconfig --namespace=extended-test-jenkins-plugin-6kis9-r6b9i --namespace extended-test-jenkins-plugin-6kis9-r6b9i-jenkins jenkins-1-0sv14 -- cat /sys/fs/cgroup/memory/memory.stat'\r\nRetrieving Jekins resource: \"http://172.30.249.192:80/job/test-exec-dsl-job/lastBuild/buildNumber\"\r\nError accessing resource: Unable to GET uri \"http://172.30.249.192:80/job/test-exec-dsl-job/lastBuild/buildNumber\": Get http://172.30.249.192:80/job/test-exec-dsl-job/lastBuild/buildNumber: dial tcp 172.30.249.192:80: getsockopt: connection refused\r\nNov  9 12:07:04.209: INFO: Error running &{/data/src/github.com/openshift/origin/_output/local/bin/linux/amd64/oc [oc exec --config=/tmp/extended-test-jenkins-plugin-6kis9-r6b9i-user.kubeconfig --namespace=extended-test-jenkins-plugin-6kis9-r6b9i --namespace extended-test-jenkins-plugin-6kis9-r6b9i-jenkins jenkins-1-0sv14 -- cat /sys/fs/cgroup/memory/memory.stat] []   error: Internal error occurred: error executing command in container: container not found (\"jenkins\")\r\n error: Internal error occurred: error executing command in container: container not found (\"jenkins\")\r\n [] <nil> 0xc821248000 exit status 1 <nil> true [0xc82002ca18 0xc82002ca40 0xc82002ca40] [0xc82002ca18 0xc82002ca40] [0xc82002ca20 0xc82002ca38] [0xb187a0 0xb18900] 0xc820fdfbc0}:\r\nerror: Internal error occurred: error executing command in container: container not found (\"jenkins\")\r\n\r\n```"}}
{"actor":{"display_login":"jupierce"}}