{"action":"created","issue":{"url":"https://api.github.com/repos/scylladb/scylla/issues/3646","repository_url":"https://api.github.com/repos/scylladb/scylla","labels_url":"https://api.github.com/repos/scylladb/scylla/issues/3646/labels{/name}","comments_url":"https://api.github.com/repos/scylladb/scylla/issues/3646/comments","events_url":"https://api.github.com/repos/scylladb/scylla/issues/3646/events","html_url":"https://github.com/scylladb/scylla/issues/3646","id":346984155,"node_id":"MDU6SXNzdWUzNDY5ODQxNTU=","number":3646,"title":"jmx crash","user":{"login":"avnerbarr","id":2404795,"node_id":"MDQ6VXNlcjI0MDQ3OTU=","avatar_url":"https://avatars0.githubusercontent.com/u/2404795?v=4","gravatar_id":"","url":"https://api.github.com/users/avnerbarr","html_url":"https://github.com/avnerbarr","followers_url":"https://api.github.com/users/avnerbarr/followers","following_url":"https://api.github.com/users/avnerbarr/following{/other_user}","gists_url":"https://api.github.com/users/avnerbarr/gists{/gist_id}","starred_url":"https://api.github.com/users/avnerbarr/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/avnerbarr/subscriptions","organizations_url":"https://api.github.com/users/avnerbarr/orgs","repos_url":"https://api.github.com/users/avnerbarr/repos","events_url":"https://api.github.com/users/avnerbarr/events{/privacy}","received_events_url":"https://api.github.com/users/avnerbarr/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":5,"created_at":"2018-08-02T12:11:00Z","updated_at":"2018-08-03T14:18:47Z","closed_at":null,"author_association":"NONE","body":"```\r\n[root@scylladb-cluster-1-work-99f7e05.use avner]# rpm -q scylla-jmx\r\nscylla-jmx-2.3.rc1-20180730.1dc4338.el7.noarch\r\n```\r\n\r\n\r\n```\r\nStarting the JMX server\r\nAug 02 12:00:41 scylladb-cluster-1-work-99f7e05.use.dynamicyield.com scylla-jmx[32166]: Exception in thread \"main\" java.lang.OutOfMemoryError: Java heap space\r\nAug 02 12:00:41 scylladb-cluster-1-work-99f7e05.use.dynamicyield.com scylla-jmx[32166]: at java.util.HashMap.resize(HashMap.java:704)\r\nAug 02 12:00:41 scylladb-cluster-1-work-99f7e05.use.dynamicyield.com scylla-jmx[32166]: at java.util.HashMap.putVal(HashMap.java:663)\r\nAug 02 12:00:41 scylladb-cluster-1-work-99f7e05.use.dynamicyield.com scylla-jmx[32166]: at java.util.HashMap.put(HashMap.java:612)\r\nAug 02 12:00:41 scylladb-cluster-1-work-99f7e05.use.dynamicyield.com scylla-jmx[32166]: at java.util.HashSet.add(HashSet.java:220)\r\nAug 02 12:00:41 scylladb-cluster-1-work-99f7e05.use.dynamicyield.com scylla-jmx[32166]: at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.objectNamesFromFilteredNamedObject\r\nAug 02 12:00:41 scylladb-cluster-1-work-99f7e05.use.dynamicyield.com scylla-jmx[32166]: at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.queryNamesImpl(DefaultMBeanServerI\r\nAug 02 12:00:41 scylladb-cluster-1-work-99f7e05.use.dynamicyield.com scylla-jmx[32166]: at com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.queryNames(DefaultMBeanServerInter\r\nAug 02 12:00:41 scylladb-cluster-1-work-99f7e05.use.dynamicyield.com scylla-jmx[32166]: at com.sun.jmx.mbeanserver.JmxMBeanServer.queryNames(JmxMBeanServer.java:619)\r\nAug 02 12:00:41 scylladb-cluster-1-work-99f7e05.use.dynamicyield.com scylla-jmx[32166]: at com.scylladb.jmx.utils.APIMBeanServer.queryNames(APIMBeanServer.java:106)\r\nAug 02 12:00:41 scylladb-cluster-1-work-99f7e05.use.dynamicyield.com scylla-jmx[32166]: at com.scylladb.jmx.main.Main.main(Main.java:49)\r\nAug 02 12:00:41 scylladb-cluster-1-work-99f7e05.use.dynamicyield.com systemd[1]: scylla-jmx.service: main process exited, code=exited, status=1/FAILURE\r\nAug 02 12:00:41 scylladb-cluster-1-work-99f7e05.use.dynamicyield.com systemd[1]: Unit scylla-jmx.service entered failed state.\r\nAug 02 12:00:41 scylladb-cluster-1-work-99f7e05.use.dynamicyield.com systemd[1]: scylla-jmx.service failed.\r\n```"},"comment":{"url":"https://api.github.com/repos/scylladb/scylla/issues/comments/410268025","html_url":"https://github.com/scylladb/scylla/issues/3646#issuecomment-410268025","issue_url":"https://api.github.com/repos/scylladb/scylla/issues/3646","id":410268025,"node_id":"MDEyOklzc3VlQ29tbWVudDQxMDI2ODAyNQ==","user":{"login":"haaawk","id":1311149,"node_id":"MDQ6VXNlcjEzMTExNDk=","avatar_url":"https://avatars2.githubusercontent.com/u/1311149?v=4","gravatar_id":"","url":"https://api.github.com/users/haaawk","html_url":"https://github.com/haaawk","followers_url":"https://api.github.com/users/haaawk/followers","following_url":"https://api.github.com/users/haaawk/following{/other_user}","gists_url":"https://api.github.com/users/haaawk/gists{/gist_id}","starred_url":"https://api.github.com/users/haaawk/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/haaawk/subscriptions","organizations_url":"https://api.github.com/users/haaawk/orgs","repos_url":"https://api.github.com/users/haaawk/repos","events_url":"https://api.github.com/users/haaawk/events{/privacy}","received_events_url":"https://api.github.com/users/haaawk/received_events","type":"User","site_admin":false},"created_at":"2018-08-03T14:18:47Z","updated_at":"2018-08-03T14:18:47Z","author_association":"MEMBER","body":"Thanks @avnerbarr. \r\n\r\nWould it be possible for you to run scylla-jmx with the script I'm attaching?\r\n[scylla-jmx.txt](https://github.com/scylladb/scylla/files/2257627/scylla-jmx.txt)\r\nThe only difference is that I added -XX:+HeapDumpOnOutOfMemoryError to dump a heap dump on OOM error.\r\n\r\nAnother option would be to find a way for me to reproduce your problem locally.\r\n\r\nThe easiest but also least helpful option is to run jcmd <scylla-jmx process id> GC.class_histogram to create a class histogram.\r\n"}}
{"actor":{"display_login":"haaawk"}}