title: Setting Ideal size of Thread Pool
body: <p>What is the difference between-</p>
  
  <p><code>newSingleThreadExecutor vs newFixedThreadPool(20)</code></p>
  
  <p>in terms of Operating System and Programming point of view.</p>
  
  <p>Whenever I am running my program using <code>newSingleThreadExecutor</code> my program works very well and end to end latency(95th percentile) comes around <code>5ms</code>.</p>
  
  <p>But as soon as I start running my program using-</p>
  
  <p><code>newFixedThreadPool(20)</code></p>
  
  <p>my program performance degrades and I start seeing end to end latency as <code>37ms</code>.</p>
  
  <p>So now I am trying to understand from architecture point of view what does number of threads  means here? And how to decide what is the optimal number of threads I should choose?</p>
  
  <p>And if I am using more number of threads then what will happen?</p>
  
  <p>If anyone can explain me these simple things in a layman language then that will be very useful to me. Thanks for the help.</p>
  
  <p>My machine config spec- I am running my program from Linux machine-</p>
  
  <pre><code>processor       : 0
  vendor_id       : GenuineIntel
  cpu family      : 6
  model           : 45
  model name      : Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
  stepping        : 7
  cpu MHz         : 2599.999
  cache size      : 20480 KB
  fpu             : yes
  fpu_exception   : yes
  cpuid level     : 13
  wp              : yes
  flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology tsc_reliable nonstop_tsc aperfmperf pni pclmulqdq ssse3 cx16 sse4_1 sse4_2 popcnt aes hypervisor lahf_lm arat pln pts
  bogomips        : 5199.99
  clflush size    : 64
  cache_alignment : 64
  address sizes   : 40 bits physical, 48 bits virtual
  power management:
  
  processor       : 1
  vendor_id       : GenuineIntel
  cpu family      : 6
  model           : 45
  model name      : Intel(R) Xeon(R) CPU E5-2670 0 @ 2.60GHz
  stepping        : 7
  cpu MHz         : 2599.999
  cache size      : 20480 KB
  fpu             : yes
  fpu_exception   : yes
  cpuid level     : 13
  wp              : yes
  flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good xtopology tsc_reliable nonstop_tsc aperfmperf pni pclmulqdq ssse3 cx16 sse4_1 sse4_2 popcnt aes hypervisor lahf_lm arat pln pts
  bogomips        : 5199.99
  clflush size    : 64
  cache_alignment : 64
  address sizes   : 40 bits physical, 48 bits virtual
  power management:
  </code></pre>