<p>I am calling the multiple rest API and for this i am using the <code>ExecutorService</code> for parallel processing. Consumer hit my app more than 10 times every seconds and i observed  <code>ExecutorService</code> very slow to response. I have deployed my application on <code>kubernetes</code> using tomcat web server.
below is my code , not sure what is causing this to slow.</p>
<pre><code>   ExecutorService WORKER_THREAD_POOL = Executors.newFixedThreadPool(100);

            Collection tasks = new ArrayList();
  for( String str: datalist){
 tasks.add(new MyThreadPool(str)); // MyThreadPool internally calls the REST API using Google HttpRequest.
}
 List&lt;Future&lt;BasePolicy&gt;&gt; futures = null;
            try {
                long startProcessingTime = System.currentTimeMillis();
                futures = WORKER_THREAD_POOL.invokeAll(tasks);
                WORKER_THREAD_POOL.shutdown();

                if (!WORKER_THREAD_POOL.awaitTermination(60000, TimeUnit.SECONDS)) {
                    WORKER_THREAD_POOL.shutdownNow();
                }

                long totalProcessingTime = System.currentTimeMillis() - startProcessingTime;
                log.info(&quot;total time to complete the thread pool- &quot; + totalProcessingTime);
            } catch (InterruptedException e) {
                log.error(&quot;error occured during ASYNC process &quot;);
                e.printStackTrace();
            }
            log.info(&quot;Finished Waiting All threads completed &quot;);
            for (Future&lt;Data&gt; mFuture : futures) {
               // my logic 
            }
</code></pre>

user1591156
<blockquote>
<p>and i observed ExecutorService very slow to response. <br></p>
</blockquote>
<pre><code>ExecutorService WORKER_THREAD_POOL = Executors.newFixedThreadPool(100);
</code></pre>
<p>Let's start with the question: How many cores does your machine have? <br>
For, <code>newFixedThreadPool</code>, it is suggested to start with as many threads as the number of cores in your machine (if the tasks are long running ones). With a value of <code>100</code>, your CPU is going to be busy with <strong>scheduling and context-switching</strong>. On top of that, since it's a <code>newFixedThreadPool</code>, so many threads are going to remain in pool even when there is less load - not helpin the CPU.</p>
<p>As suggested in <em>Effective Java</em>, <code>newCachedThreadPool</code> <em>usually does the good thing</em>.
<a href="https://stackoverflow.com/questions/17957382/fixedthreadpool-vs-cachedthreadpool-the-lesser-of-two-evils">Here</a> is more on the Thread Pools.</p>
<p>If you are expecting a really heavy load, I think deployment of apps on multiple servers would be the wise thing to do (this depends on the capacity of your single server). Threads are not going to help and can eventually make your app slow.</p>
