{"action":"closed","issue":{"url":"https://api.github.com/repos/dpkp/kafka-python/issues/852","repository_url":"https://api.github.com/repos/dpkp/kafka-python","labels_url":"https://api.github.com/repos/dpkp/kafka-python/issues/852/labels{/name}","comments_url":"https://api.github.com/repos/dpkp/kafka-python/issues/852/comments","events_url":"https://api.github.com/repos/dpkp/kafka-python/issues/852/events","html_url":"https://github.com/dpkp/kafka-python/issues/852","id":184665740,"number":852,"title":"kafka server  wedged at 100% CPU with no traffic!","user":{"login":"ZiCog","id":2519277,"avatar_url":"https://avatars.githubusercontent.com/u/2519277?v=3","gravatar_id":"","url":"https://api.github.com/users/ZiCog","html_url":"https://github.com/ZiCog","followers_url":"https://api.github.com/users/ZiCog/followers","following_url":"https://api.github.com/users/ZiCog/following{/other_user}","gists_url":"https://api.github.com/users/ZiCog/gists{/gist_id}","starred_url":"https://api.github.com/users/ZiCog/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/ZiCog/subscriptions","organizations_url":"https://api.github.com/users/ZiCog/orgs","repos_url":"https://api.github.com/users/ZiCog/repos","events_url":"https://api.github.com/users/ZiCog/events{/privacy}","received_events_url":"https://api.github.com/users/ZiCog/received_events","type":"User","site_admin":false},"labels":[],"state":"closed","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2016-10-23T01:35:47Z","updated_at":"2016-10-23T17:43:09Z","closed_at":"2016-10-23T17:43:09Z","body":"I was just trying my first tentative experiments with kafka. Which ended up with it wedged at 100% CPU load for ten minutes while no traffic was flowing.\r\n\r\nI have kafka_2.11-0.10.0.0 running on a headless Debian Jessie PC and set it running as per the starter instructions:\r\n\r\n```\r\n$ nohup  bin/zookeeper-server-start.sh config/zookeeper.properties &\r\n$ nohup bin/kafka-server-start.sh config/server.properties &\r\n\r\n```\r\nI create a couple of topics:\r\n\r\n```\r\n$bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --zookeeper localhost:2181   --topic topic1 --from-beginning\r\n$ bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --zookeeper localhost:2181   --topic topic2 --from-beginning\r\n\r\n```\r\nThen on another machine I start experimenting with node.js and kafka-node.\r\n\r\nFirstly a producer that puts a few hundred thousand messages into the topics. Then a consumer to get them back again. Which both worked.\r\n\r\nThe code which triggered the server blow up was only a little test of getting latest offsets:\r\n\r\n```\r\n// Fetch first or last offsets or from a given time.\r\nconst offset = new kafka.Offset(client);\r\noffset.fetch([\r\n    { topic: 'topic1', partition: 0, time: -2, maxNum: 10 },   // First available offset\r\n    { topic: 'topic2', partition: 0, time: -1, maxNum: 10 }    // Latest offset\r\n], function(err, data) {\r\n    if (err) {\r\n        return console.log(\"Error: offset.fetch:\", err);\r\n    }\r\n    console.log(data);\r\n    var firstOffsetTopic1 = data['topic1']['0'][0];\r\n    var latestOffsetTopic2 = data['topic2']['0'][0];\r\n\r\n    console.log('First offset topic 1', firstOffsetTopic1);\r\n    console.log('Latest offset topic2', latestOffsetTopic2);\r\n});\r\n\r\nlet partition = 0;\r\nlet topic = 'topic1';\r\noffset.fetchLatestOffsets([topic], function (error, offsets) {\r\n    if (error) {\r\n        return console.log('Error: error: offset.fetchLatestOffsets', err);\r\n    }\r\n    console.log(offsets[topic][partition]);\r\n});\r\n\r\n```\r\nWhich I notice sometimes fails with: \r\n\r\n```\r\n$ node kafka-node-test.js\r\n\r\nevents.js:160\r\n      throw er; // Unhandled 'error' event\r\n      ^\r\nBrokerNotAvailableError: Could not find a broker\r\n    at new BrokerNotAvailableError (C:\\cygwin64\\home\\michael\\fits-view-webgl\\node_modules\\kafka-node\\lib\\errors\\BrokerNotAvailableError.js:11:9)\r\n    at Client.brokerForLeader (C:\\cygwin64\\home\\michael\\fits-view-webgl\\node_modules\\kafka-node\\lib\\client.js:609:26)\r\n    at Client.loadMetadataForTopics (C:\\cygwin64\\home\\michael\\fits-view-webgl\\node_modules\\kafka-node\\lib\\client.js:341:21)\r\n    at Client.send (C:\\cygwin64\\home\\michael\\fits-view-webgl\\node_modules\\kafka-node\\lib\\client.js:498:10)\r\n    at Client.sendOffsetRequest (C:\\cygwin64\\home\\michael\\fits-view-webgl\\node_modules\\kafka-node\\lib\\client.js:270:8)\r\n    at Offset.fetch (C:\\cygwin64\\home\\michael\\fits-view-webgl\\node_modules\\kafka-node\\lib\\offset.js:31:15)\r\n    at .<anonymous> (C:\\cygwin64\\home\\michael\\fits-view-webgl\\node_modules\\kafka-node\\lib\\offset.js:27:12)\r\n    at tryOnTimeout (timers.js:224:11)\r\n    at Timer.listOnTimeout (timers.js:198:5)\r\n\r\n```\r\nSorry if this is not a very helpful issue report. I the are any logs or such that I can dig out that might give more info I will do so.\r\n\r\nMeanwhile I notice some exceptions in my nohup file:\r\n\r\n```\r\nessed session termination for sessionid: 0x157ec4b8b500078 (org.apache.zookeeper.server.PrepRequestProcessor)\r\n[2016-10-23 03:28:48,005] INFO Closed socket connection for client /192.168.1.244:50752 which had sessionid 0x157ec4b8b500078 (org.apache.zookeeper.server.NIOServerCnxn)\r\n[2016-10-23 03:29:06,000] INFO Expiring session 0x157ec4b8b500079, timeout of 30000ms exceeded (org.apache.zookeeper.server.ZooKeeperServer)\r\n[2016-10-23 03:29:06,000] INFO Processed session termination for sessionid: 0x157ec4b8b500079 (org.apache.zookeeper.server.PrepRequestProcessor)\r\n[2016-10-23 03:29:06,048] INFO Got user-level KeeperException when processing sessionid:0x157ec4b8b50007a type:create cxid:0x19 zxid:0x2ab txntype:-1 reqpath:n/a Error Path:/consumers/kafka-node-group Error:KeeperErrorCode = NodeExists for /consumers/kafka-node-group (org.apache.zookeeper.server.PrepRequestProcessor)\r\n[2016-10-23 03:29:06,059] INFO Got user-level KeeperException when processing sessionid:0x157ec4b8b50007a type:create cxid:0x1a zxid:0x2ac txntype:-1 reqpath:n/a Error Path:/consumers/kafka-node-group/owners Error:KeeperErrorCode = NodeExists for /consumers/kafka-node-group/owners (org.apache.zookeeper.server.PrepRequestProcessor)\r\n[2016-10-23 03:29:06,078] INFO Got user-level KeeperException when processing sessionid:0x157ec4b8b50007a type:create cxid:0x1b zxid:0x2ad txntype:-1 reqpath:n/a Error Path:/consumers/kafka-node-group/owners/topic2 Error:KeeperErrorCode = NodeExists for /consumers/kafka-node-group/owners/topic2 (org.apache.zookeeper.server.PrepRequestProcessor)\r\n[2016-10-23 03:29:08,002] WARN Exception causing close of session 0x157ec4b8b50007a due to java.io.IOException: Connection reset by peer (org.apache.zookeeper.server.NIOServerCnxn)\r\n[2016-10-23 03:29:08,003] INFO Closed socket connection for client /192.168.1.244:50757 which had sessionid 0x157ec4b8b50007a (org.apache.zookeeper.server.NIOServerCnxn)\r\n[2016-10-23 03:29:09,477] INFO Accepted socket connection from /192.168.1.244:50763 (org.apache.zookeeper.server.NIOServerCnxnFactory)\r\n[2016-10-23 03:29:09,480] WARN Connection request from old client /192.168.1.244:50763; will be dropped if server is in r-o mode (org.apache.zookeeper.server.ZooKeeperServer)\r\n[2016-10-23 03:29:09,480] INFO Client attempting to establish new session at /192.168.1.244:50763 (org.apache.zookeeper.server.ZooKeeperServer)\r\n[2016-10-23 03:29:09,484] INFO Established session 0x157ec4b8b50007b with negotiated timeout 30000 for client /192.168.1.244:50763 (org.apache.zookeeper.server.ZooKeeperServer)\r\n[2016-10-23 03:29:09,507] INFO Got user-level KeeperException when processing sessionid:0x157ec4b8b50007b type:create cxid:0x2 zxid:0x2b0 txntype:-1 reqpath:n/a Error Path:/consumers/kafka-node-group Error:KeeperErrorCode = NodeExists for /consumers/kafka-node-group (org.apache.zookeeper.server.PrepRequestProcessor)\r\n[2016-10-23 03:29:09,513] INFO Got user-level KeeperException when processing sessionid:0x157ec4b8b50007b type:create cxid:0x4 zxid:0x2b1 txntype:-1 reqpath:n/a Error Path:/consumers/kafka-node-group/ids Error:KeeperErrorCode = NodeExists for /consumers/kafka-node-group/ids (org.apache.zookeeper.server.PrepRequestProcessor)\r\n[2016-10-23 03:29:19,497] WARN Exception causing close of session 0x157ec4b8b50007b due to java.io.IOException: Connection reset by peer (org.apache.zookeeper.server.NIOServerCnxn)\r\n[2016-10-23 03:29:19,498] INFO Closed socket connection for client /192.168.1.244:50763 which had sessionid 0x157ec4b8b50007b (org.apache.zookeeper.server.NIOServerCnxn)\r\n[2016-10-23 03:29:39,000] INFO Expiring session 0x157ec4b8b50007a, timeout of 30000ms exceeded (org.apache.zookeeper.server.ZooKeeperServer)\r\n[2016-10-23 03:29:39,000] INFO Processed session termination for sessionid: 0x157ec4b8b50007a (org.apache.zookeeper.server.PrepRequestProcessor)\r\n[2016-10-23 03:29:42,000] INFO Expiring session 0x157ec4b8b50007b, timeout of 30000ms exceeded (org.apache.zookeeper.server.ZooKeeperServer)\r\n[2016-10-23 03:29:42,001] INFO Processed session termination for sessionid: 0x157ec4b8b50007b (org.apache.zookeeper.server.PrepRequestProcessor)\r\n[2016-10-23 03:30:00,451] INFO Accepted socket connection from /192.168.1.244:50768 (org.apache.zookeeper.server.NIOServerCnxnFactory)\r\n[2016-10-23 03:30:00,456] WARN Connection request from old client /192.168.1.244:50768; will be dropped if server is in r-o mode (org.apache.zookeeper.server.ZooKeeperServer)\r\n[2016-10-23 03:30:00,456] INFO Client attempting to establish new session at /192.168.1.244:50768 (org.apache.zookeeper.server.ZooKeeperServer)\r\n[2016-10-23 03:30:00,460] INFO Established session 0x157ec4b8b50007c with negotiated timeout 30000 for client /192.168.1.244:50768 (org.apache.zookeeper.server.ZooKeeperServer)\r\n[2016-10-23 03:30:00,605] WARN Exception causing close of session 0x157ec4b8b50007c due to java.io.IOException: Connection reset by peer (org.apache.zookeeper.server.NIOServerCnxn)\r\n[2016-10-23 03:30:00,606] INFO Closed socket connection for client /192.168.1.244:50768 which had sessionid 0x157ec4b8b50007c (org.apache.zookeeper.server.NIOServerCnxn)\r\n[2016-10-23 03:30:08,655] INFO [Group Metadata Manager on Broker 0]: Removed 0 expired offsets in 0 milliseconds. (kafka.coordinator.GroupMetadataManager)\r\n[2016-10-23 03:30:21,105] INFO Accepted socket connection from /192.168.1.244:50772 (org.apache.zookeeper.server.NIOServerCnxnFactory)\r\n[2016-10-23 03:30:21,110] WARN Connection request from old client /192.168.1.244:50772; will be dropped if server is in r-o mode (org.apache.zookeeper.server.ZooKeeperServer)\r\n[2016-10-23 03:30:21,111] INFO Client attempting to establish new session at /192.168.1.244:50772 (org.apache.zookeeper.server.ZooKeeperServer)\r\n[2016-10-23 03:30:21,114] INFO Established session 0x157ec4b8b50007d with negotiated timeout 30000 for client /192.168.1.244:50772 (org.apache.zookeeper.server.ZooKeeperServer)\r\n[2016-10-23 03:30:33,000] INFO Expiring session 0x157ec4b8b50007c, timeout of 30000ms exceeded (org.apache.zookeeper.server.ZooKeeperServer)\r\n[2016-10-23 03:30:33,000] INFO Processed session termination for sessionid: 0x157ec4b8b50007c (org.apache.zookeeper.server.PrepRequestProcessor)\r\n[2016-10-23 03:30:55,518] WARN Exception causing close of session 0x157ec4b8b50007d due to java.io.IOException: Connection reset by peer (org.apache.zookeeper.server.NIOServerCnxn)\r\n[2016-10-23 03:30:55,519] INFO Closed socket connection for client /192.168.1.244:50772 which had sessionid 0x157ec4b8b50007d (org.apache.zookeeper.server.NIOServerCnxn)\r\n[2016-10-23 03:31:24,000] INFO Expiring session 0x157ec4b8b50007d, timeout of 30000ms exceeded (org.apache.zookeeper.server.ZooKeeperServer)\r\n[2016-10-23 03:31:24,000] INFO Processed session termination for sessionid: 0x157ec4b8b50007d (org.apache.zookeeper.server.PrepRequestProcessor)\r\n[2016-10-23 03:37:57,657] INFO [Kafka Server 0], shutting down (kafka.server.KafkaServer)\r\n[2016-10-23 03:37:57,660] INFO [Kafka Server 0], Starting controlled shutdown (kafka.server.KafkaServer)\r\n[2016-10-23 03:37:57,705] INFO [Kafka Server 0], Controlled shutdown succeeded (kafka.server.KafkaServer)\r\n[2016-10-23 03:37:57,708] INFO [Socket Server on Broker 0], Shutting down (kafka.network.SocketServer)\r\n[2016-10-23 03:38:36,538] WARN caught end of stream exception (org.apache.zookeeper.server.NIOServerCnxn)\r\nEndOfStreamException: Unable to read additional data from client sessionid 0x157ec4b8b500000, likely client has closed socket\r\n        at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)\r\n        at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)\r\n        at java.lang.Thread.run(Thread.java:745)\r\n[2016-10-23 03:38:36,539] INFO Closed socket connection for client /127.0.0.1:47189 which had sessionid 0x157ec4b8b500000 (org.apache.zookeeper.server.NIOServerCnxn)\r\n[2016-10-23 03:38:42,000] INFO Expiring session 0x157ec4b8b500000, timeout of 6000ms exceeded (org.apache.zookeeper.server.ZooKeeperServer)\r\n[2016-10-23 03:38:42,000] INFO Processed session termination for sessionid: 0x157ec4b8b500000 (org.apache.zookeeper.server.PrepRequestProcessor)\r\n\r\n\r\n```"}}
{"actor":{"display_login":"dpkp"}}