title: Gradual increase in Resident Memory usage by Jboss(Java) process
body: <p>We are facing an issue where the Resident memory of the Java process grows gradually. We have Xmx defined at 4096 MB and XX:MaxPermSize=1536m. The number of active threads ~1500 with an Xss of 256K defined. </p>
  
  <p>When the application server(JBoss 6.1) starts the resident memory used is ~5.6GB(have been using top command to monitor it); it gradually grows(around 0.3 to 0.5 Gb per day) till it grows to ~7.4 Gb, when the kernel's OOM killer kills the process due to shortage of RAM space(The server has 9GB of RAM).</p>
  
  <p>We have been regularly monitoring the thread dump- no suspect of a thread leak. We are still unable to figure out from where this extra memory is coming from.</p>
  
  <p>Pmap output shows a number of Anon blocks(apart from the regular blocks for stack and heap), mostly in arenas of 64 Mb, which are un-accounted for in terms of memory usage by heap, perm gen &amp; stacks.</p>
  
  <p>In the heap dump we have also tried looking for DirectByteBuffers and sun.misc.Unsafe objects, which are generally used for non-heap memory allocation, but the number of objects as well as the memory capacity seem nominal. Is it possible that there can still be un- freed native memory even after these objects are GCed? Any other classes that may result in using up non-heap memory?</p>
  
  <p>Our application does have native calls on its own, but it's possible that some third party libs have them.</p>
  
  <p>Any ideas on what could be causing this? Anything other detail/tool that could further help debugging such an increase? Any known issue that we should look out for? Platform : Jboss 6.1 running on Centos 5.6.   </p>