{"action":"created","issue":{"url":"https://api.github.com/repos/ninjaframework/ninja/issues/502","repository_url":"https://api.github.com/repos/ninjaframework/ninja","labels_url":"https://api.github.com/repos/ninjaframework/ninja/issues/502/labels{/name}","comments_url":"https://api.github.com/repos/ninjaframework/ninja/issues/502/comments","events_url":"https://api.github.com/repos/ninjaframework/ninja/issues/502/events","html_url":"https://github.com/ninjaframework/ninja/issues/502","id":147133014,"number":502,"title":"Ninja app drastically slow / stop responding after some time.","user":{"login":"danielsawan","id":838554,"avatar_url":"https://avatars.githubusercontent.com/u/838554?v=3","gravatar_id":"","url":"https://api.github.com/users/danielsawan","html_url":"https://github.com/danielsawan","followers_url":"https://api.github.com/users/danielsawan/followers","following_url":"https://api.github.com/users/danielsawan/following{/other_user}","gists_url":"https://api.github.com/users/danielsawan/gists{/gist_id}","starred_url":"https://api.github.com/users/danielsawan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/danielsawan/subscriptions","organizations_url":"https://api.github.com/users/danielsawan/orgs","repos_url":"https://api.github.com/users/danielsawan/repos","events_url":"https://api.github.com/users/danielsawan/events{/privacy}","received_events_url":"https://api.github.com/users/danielsawan/received_events","type":"User","site_admin":false},"labels":[],"state":"open","locked":false,"assignee":null,"milestone":null,"comments":13,"created_at":"2016-04-09T14:29:04Z","updated_at":"2016-04-11T00:01:41Z","closed_at":null,"body":"Hi there\r\n\r\nI have a probleme that occure after 1 / 2 days of exploitation in my production server.\r\nThe symptoms are : \r\n\r\n- all the server 8 core CPU at 100% usage.\r\n- the ninja app stop responding to request\r\n- ninja app memory usage rise up to 5/6 Go of Ram\r\n- my server dump is full of (arround 150) : \r\n\r\n```\r\n\"qtp55909012-2270\" \r\n   java.lang.Thread.State: TIMED_WAITING\r\n        at sun.misc.Unsafe.park(Native Method)\r\n        at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)\r\n        at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)\r\n        at org.eclipse.jetty.util.BlockingArrayQueue.poll(BlockingArrayQueue.java:389)\r\n        at org.eclipse.jetty.util.thread.QueuedThreadPool.idleJobPoll(QueuedThreadPool.java:531)\r\n        at org.eclipse.jetty.util.thread.QueuedThreadPool.access$700(QueuedThreadPool.java:47)\r\n        at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:590)\r\n        at java.lang.Thread.run(Thread.java:745)\r\n```\r\n\r\n\r\nAfter searching on the google I think that the problem is that the Ninja JVM reach the XMX (which I have not set at the startup command on my 32Go of ram server). And then the JVM use all the CPU to try to free some memory without success. So the CPU is fully used and the Ninja app stop responding.\r\n\r\nAnd after looking at all dump I have like 6 threads running some clients code and all the rest is that sun.misc.Unsafe.park(Native Method) : about 150 occurrences of this.\r\nI don’t know if this is another symptom or the root cause of the problem. I am still waiting that the problem occure one more time in order to check my memory usage and log files.\r\n\r\nIt is the first time I am facing this sort of problem. And the website has not so many user yet : around 8/12 at the same time at maximum. I found on Google that it can be a lock problem. I am a little bit lost. Any help would be appreciated.\r\n\r\nThanks.\r\n\r\n"},"comment":{"url":"https://api.github.com/repos/ninjaframework/ninja/issues/comments/208097622","html_url":"https://github.com/ninjaframework/ninja/issues/502#issuecomment-208097622","issue_url":"https://api.github.com/repos/ninjaframework/ninja/issues/502","id":208097622,"user":{"login":"danielsawan","id":838554,"avatar_url":"https://avatars.githubusercontent.com/u/838554?v=3","gravatar_id":"","url":"https://api.github.com/users/danielsawan","html_url":"https://github.com/danielsawan","followers_url":"https://api.github.com/users/danielsawan/followers","following_url":"https://api.github.com/users/danielsawan/following{/other_user}","gists_url":"https://api.github.com/users/danielsawan/gists{/gist_id}","starred_url":"https://api.github.com/users/danielsawan/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/danielsawan/subscriptions","organizations_url":"https://api.github.com/users/danielsawan/orgs","repos_url":"https://api.github.com/users/danielsawan/repos","events_url":"https://api.github.com/users/danielsawan/events{/privacy}","received_events_url":"https://api.github.com/users/danielsawan/received_events","type":"User","site_admin":false},"created_at":"2016-04-11T00:01:41Z","updated_at":"2016-04-11T00:01:41Z","body":"Nice tips.\r\nI tried this and what happen is really strange.\r\nhere is the stacktrace : http://puu.sh/odJ5L/5a2e8a3a97.png\r\n\r\nso it seems to be called by Ninja.java onRouteRequest. \r\nhere is my Ninja.java now : http://puu.sh/odJbk/0da803d427.png\r\n\r\nthe more i dig the less i understand."}}