title: CPU usage going 100% on making any transaction
body: <p>Recently I came up across strange scenario where I could see the application slows down after some point of time due  to which I am unable to load application.</p>
  
  <p>So I started monitoring the server and found that for some of the scenario's the GC activity is going more than 60% and it is impacting directly on application perfomance so in order to fix it I tried to increase the heap memory to 2.5G from 2GB but still I could find same issue can anyone help me is  increasing Heap size further is an ideal approach or something else I can do.</p>
  
  <p>System Ram Capability : 6 GB</p>
  
  <p>Following is VisualVm monitor screenshot:
  <a href="https://i.stack.imgur.com/P4Z8i.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/P4Z8i.png" alt="VisualVmScreenshot"></a></p>
  
  <p>Kindly let me know if further info is required for analysis.</p>
  
  <p>Thread Dump:</p>
  
  <pre><code>http-apr-8080-exec-15" - Thread t@110
     java.lang.Thread.State: RUNNABLE
          at java.util.regex.Pattern$BnM.match(Pattern.java:5464)
          at java.util.regex.Matcher.search(Matcher.java:1248)
          at java.util.regex.Matcher.find(Matcher.java:637)
          at java.util.regex.Matcher.replaceAll(Matcher.java:951)
          at java.lang.String.replace(String.java:2240)
          at com.srik.security.SecureTrace.maskOther(SecureTrace.java:135)
          at com.srik.security.SecureTrace.maskJsonData(SecureTrace.java:102)
          at com.srik.mbweb.manager.MUSR.printResponse(MUSR.java:261)
          at com.srik.mbweb.manager.MUSR.invokeService(MUSR.java:212)
          at com.srik.mbweb.manager.MU.invokeCacheableService(MU.java:30)
          at com.srik.mbweb.manager.MU$$FastClassBySpringCGLIB$$45933c60.invoke(&lt;generated&gt;)
          at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
          at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:738)
          at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
          at org.springframework.cache.interceptor.CacheInterceptor$1.invoke(CacheInterceptor.java:52)
          at org.springframework.cache.interceptor.CacheAspectSupport.invokeOperation(CacheAspectSupport.java:345)
          at org.springframework.cache.interceptor.CacheAspectSupport.execute(CacheAspectSupport.java:408)
          at org.springframework.cache.interceptor.CacheAspectSupport.execute(CacheAspectSupport.java:327)
          at org.springframework.cache.interceptor.CacheInterceptor.invoke(CacheInterceptor.java:61)
          at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
          at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:673)
          at com.srik.mbweb.manager.MU$$EnhancerBySpringCGLIB$$36289a8f.invokeCacheableService(&lt;generated&gt;)
          at com.srik.mbweb.manager.ChannelUnitedServiceManagerProxy.invokeService(ChannelUnitedServiceManagerProxy.java:35)
          at com.srik.mbweb.controller.MobileUnitedController.processRequest(MobileUnitedController.java:32)
          at com.srik.mbweb.controller.MobileUnitedControllerBase.invokeService(MobileUnitedControllerBase.java:58)
          at sun.reflect.GeneratedMethodAccessor216.invoke(Unknown Source)
          at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
          at java.lang.reflect.Method.invoke(Method.java:498)
          at org.springframework.web.method.support.InvocableHandlerMethod.doInvoke(InvocableHandlerMethod.java:205)
          at org.springframework.web.method.support.InvocableHandlerMethod.invokeForRequest(InvocableHandlerMethod.java:133)
          at org.springframework.web.servlet.mvc.method.annotation.ServletInvocableHandlerMethod.invokeAndHandle(ServletInvocableHandlerMethod.java:97)
          at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.invokeHandlerMethod(RequestMappingHandlerAdapter.java:827)
          at org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerAdapter.handleInternal(RequestMappingHandlerAdapter.java:738)
          at org.springframework.web.servlet.mvc.method.AbstractHandlerMethodAdapter.handle(AbstractHandlerMethodAdapter.java:85)
          at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:967)
          at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:901)
          at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:970)
          at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:872)
          at javax.servlet.http.HttpServlet.service(HttpServlet.java:648)
          at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:846)
          at javax.servlet.http.HttpServlet.service(HttpServlet.java:729)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:292)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
          at com.srik.statemachine.web.AbstractStateMachineAwareFilter.doFilterInternal(AbstractStateMachineAwareFilter.java:63)
          at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
          at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
          at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:197)
          at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
          at com.srik.mbweb.common.PreAuthenticationFilter.doFilterInternal(PreAuthenticationFilter.java:137)
          at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
          at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
          at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
          at com.srik.mbweb.common.StatelessCSRFFilter.doFilterInternal(StatelessCSRFFilter.java:51)
          at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
          at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
          at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
          at com.srik.mbweb.common.LoggerFilter.doFilterInternal(LoggerFilter.java:30)
          at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
          at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
          at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
          at com.srik.mbweb.multitenant.filter.FITenantFilter.doFilterInternal(FITenantFilter.java:110)
          at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
          at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346)
          at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:262)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
          at com.srik.mbweb.multitenant.filter.FITenantFilter.doFilterInternal(FITenantFilter.java:110)
          at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
          at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:240)
          at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:207)
          at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:212)
          at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:106)
          at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:502)
          at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:141)
          at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:79)
          at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:616)
          at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:88)
          at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:528)
          at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1100)
          at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:687)
          at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.doRun(AprEndpoint.java:2521)
          at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:2510)
          - locked &lt;1e41e77c&gt; (a org.apache.tomcat.util.net.AprEndpoint$AprSocketWrapper)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
          at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
          at java.lang.Thread.run(Thread.java:748)
  
     Locked ownable synchronizers:
          - locked &lt;5896885f&gt; (a java.util.concurrent.ThreadPoolExecutor$Worker)
  </code></pre>
  
  <p>SecureTrace method:</p>
  
  <pre><code>private static Pattern PWD_XML_PATTERN = Pattern.compile("^(&lt;.*?(?:(?i)Aomn|Amn|amn|aamn)&gt;)(.*?)(&lt;\\/.*?(?:(?i)Aomn|Amn|amn|aamn)&gt;)");
   public static String maskData(String data) {
         try {
           for(Matcher m = PWD_XML_PATTERN.matcher(data); m.find(); data = m.replaceAll("AA****BB")) {
  
               ;
           }
         } catch (Exception var2) {
         var2.printStackTrace();
         }
  
        return maskOther(data);
     }
  
    private static String maskOther(String data) {
         String mask = null;
         int groupLen = false;
        Matcher matcher = null;
         try {
          matcher = MASK_PATTERN.matcher(data);
          while(matcher.find()) {
             mask = "****";
             String group = matcher.group();
              int groupLen = group.length();
              if (groupLen &gt; 5) {
                mask = mask + group.substring(groupLen - 4);
                  data = data.replace(group, mask);
               }
            }
        } catch (Exception var5) {
           ;      }
  
        return data;
     }
  </code></pre>