 <p>There are different possibilities of implementing a threading system for a virtual machine. The two extreme forms are:</p> <ol> <li><strong>Green threads</strong>: All Java <code>Thread</code> instances are managed within one native OS thread. This can cause problems if a method blocks within a <code>native</code> invocation what makes this implementation complex. In the end, implementers need to introduce <em>renegade threads</em> for holding native locks to overcome such limitations. </li> <li><strong>Native threads</strong>: Each Java <code>Thread</code> instance is backed by a native OS thread.</li> </ol> <p>For the named limitations of green threads, all modern JVM implementations, including HotSpot, choose the later implementation. This implies that the OS needs to reserve some memory for each created thread. Also, there is some runtime overhead for creating such a thread as it needs direct interaction with the underlying OS. At some point, these costs accumulate and the OS refuses the creation of new threads to prevent the stability of your overall system.</p> <p>Threads should therefore be pooled for resue. Object pooling is normally considered bad practice as many programers used it to ease the JVM's garbage collector. This is not longer useful as modern garbage collectors are optimized for handling short-living objects. Today, by pooling objects you might in contrary slow down your system. However, if an object is backed by costly native resources (as a <code>Thread</code>), pooling is still a recommended practice. Look into the <a href="http://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ExecutorService.html" rel="nofollow"><code>ExecutorService</code></a> for a canonical way of pooling threads in Java.</p> <p>In general, consider that context switches of threads are expensive. You should not create a new thread for small tasks, this will slow your application down. Rather make your application less concurrent. You only have a number of cores which can work concurrently in the first place, creating more threads than your (non-virtual) cores will not improve runtime performance. Are you implementing some sort of divide-and-conquer algorithm? Look into Java's <a href="http://docs.oracle.com/javase/8/docs/api/java/util/concurrent/ForkJoinPool.html" rel="nofollow"><code>ForkJoinPool</code></a>.</p>