{"action":"created","issue":{"url":"https://api.github.com/repos/netty/netty/issues/5104","repository_url":"https://api.github.com/repos/netty/netty","labels_url":"https://api.github.com/repos/netty/netty/issues/5104/labels{/name}","comments_url":"https://api.github.com/repos/netty/netty/issues/5104/comments","events_url":"https://api.github.com/repos/netty/netty/issues/5104/events","html_url":"https://github.com/netty/netty/issues/5104","id":146917047,"number":5104,"title":"All worker threads waiting for addBefore task to finish","user":{"login":"flozano","id":1159291,"avatar_url":"https://avatars.githubusercontent.com/u/1159291?v=3","gravatar_id":"","url":"https://api.github.com/users/flozano","html_url":"https://github.com/flozano","followers_url":"https://api.github.com/users/flozano/followers","following_url":"https://api.github.com/users/flozano/following{/other_user}","gists_url":"https://api.github.com/users/flozano/gists{/gist_id}","starred_url":"https://api.github.com/users/flozano/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/flozano/subscriptions","organizations_url":"https://api.github.com/users/flozano/orgs","repos_url":"https://api.github.com/users/flozano/repos","events_url":"https://api.github.com/users/flozano/events{/privacy}","received_events_url":"https://api.github.com/users/flozano/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":7,"created_at":"2016-04-08T12:29:19Z","updated_at":"2016-04-08T13:00:09Z","closed_at":null,"body":"Netty 4.1.0.CR6\r\nJDK 1.8.0_20\r\nLinux 2.6.32-431.el6.x86_64\r\n\r\nThe EventLoopGroup size is 2.\r\nIn our code we try to add a Idle\r\n\r\n```java\r\nchannel.pipeline().addBefore(ChannelHandlerConstant.MQTT_DECODER_HANDLER_NAME,\r\n\t\t\t\t\t\tChannelHandlerConstant.KEEP_ALIVE_TIMER_HANDLER_NAME, idleStateHandler);\r\n```\r\n\r\nAnd, since upgraded to CR6, quickly all the threads are locked like this:\r\n\r\n```\r\n\"mqtt-netty-worker-3-2\" #37 prio=5 os_prio=0 tid=0x00007fb085424800 nid=0x14fa in Object.wait() [0x00007fb0517b7000]\r\n   java.lang.Thread.State: WAITING (on object monitor)\r\n    at java.lang.Object.wait(Native Method)\r\n    at java.lang.Object.wait(Object.java:502)\r\n    at io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:255)\r\n    - locked <0x00000000e6dd0a98> (a io.netty.util.concurrent.PromiseTask)\r\n    at io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:33)\r\n    at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:32)\r\n    at io.netty.channel.DefaultChannelPipeline.waitForFuture(DefaultChannelPipeline.java:783)\r\n    at io.netty.channel.DefaultChannelPipeline.addBefore(DefaultChannelPipeline.java:300)\r\n    at io.netty.channel.DefaultChannelPipeline.addBefore(DefaultChannelPipeline.java:248)\r\n\r\n\"mqtt-netty-worker-3-1\" #36 prio=5 os_prio=0 tid=0x00007fb085422800 nid=0x14f9 in Object.wait() [0x00007fb0518b8000]\r\n   java.lang.Thread.State: WAITING (on object monitor)\r\n    at java.lang.Object.wait(Native Method)\r\n    at java.lang.Object.wait(Object.java:502)\r\n    at io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:255)\r\n    - locked <0x00000000e6dd0d38> (a io.netty.util.concurrent.PromiseTask)\r\n    at io.netty.util.concurrent.DefaultPromise.await(DefaultPromise.java:33)\r\n    at io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:32)\r\n    at io.netty.channel.DefaultChannelPipeline.waitForFuture(DefaultChannelPipeline.java:783)\r\n    at io.netty.channel.DefaultChannelPipeline.addBefore(DefaultChannelPipeline.java:300)\r\n    at io.netty.channel.DefaultChannelPipeline.addBefore(DefaultChannelPipeline.java:248)\r\n(...)\r\n```\r\n\r\nIt doesn't reproduce in the previous netty version we used (4.1.0.Beta8).\r\n"},"comment":{"url":"https://api.github.com/repos/netty/netty/issues/comments/207422850","html_url":"https://github.com/netty/netty/issues/5104#issuecomment-207422850","issue_url":"https://api.github.com/repos/netty/netty/issues/5104","id":207422850,"user":{"login":"flozano","id":1159291,"avatar_url":"https://avatars.githubusercontent.com/u/1159291?v=3","gravatar_id":"","url":"https://api.github.com/users/flozano","html_url":"https://github.com/flozano","followers_url":"https://api.github.com/users/flozano/followers","following_url":"https://api.github.com/users/flozano/following{/other_user}","gists_url":"https://api.github.com/users/flozano/gists{/gist_id}","starred_url":"https://api.github.com/users/flozano/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/flozano/subscriptions","organizations_url":"https://api.github.com/users/flozano/orgs","repos_url":"https://api.github.com/users/flozano/repos","events_url":"https://api.github.com/users/flozano/events{/privacy}","received_events_url":"https://api.github.com/users/flozano/received_events","type":"User","site_admin":false},"created_at":"2016-04-08T13:00:09Z","updated_at":"2016-04-08T13:00:09Z","body":"> @flozano also when you say these are locked are these never recover ?\r\n\r\nYeah, no recovery in sight, the server gets fully locked."}}