title: Thread safe memory pool
body: <p>My application currently is highly performance critical and is requests 3-5 million objects per frame. Initially, to get the ball rolling, I <code>new'd</code> everything and got the application to work and test my algorithms. The application is multi-threaded. </p>
  
  <p>Once I was happy with the performance, I started to create a memory manager for my objects. The obvious reason is memory fragmentation and wastage. The application could not continue for more than a few frames before crashing due to memory fragmentation. I have checked for memory leaks and know the application is leak free. </p>
  
  <p>So I started creating a simple memory manager using TBB's <code>concurrent_queue</code>. The queue stores a maximum set of elements the application is allowed to use. The class requiring new elements pops elements from the queue. The <code>try_pop</code> method is, according to Intel's documentation, lock-free. This worked quite well as far as memory consumption goes (although there is still memory fragmentation, but not nearly as much as before). The problem I am facing now is that the application's performance has slowed down approximately 4 times according to my own simple profiler (I do not have access to commercial profilers or know of any that will work on a real-time application... any recommendation would be appreciated). </p>
  
  <p>My question is, is there a thread-safe memory pool that is scalable. A <code>must-have</code> feature of the pool is fast recycling of elements and making them available. If there is none, any tips/tricks performance wise?</p>
  
  <p><strong>EDIT:</strong> I thought I would explain the problem a bit more. I could easily initialize <em>n</em> number of arrays where <em>n</em> is the number of threads and start using the objects from the arrays per thread. This will work perfectly for some cases. In my case, I am recycling the elements as well (potentially every frame) and they could be recycled at any point in the array; i.e. it may be from <code>elementArray[0]</code> or <code>elementArray[10]</code> or <code>elementArray[1000]</code> part of the array. Now I will have a fragmented array of elements consisting of elements that are ready to be used and elements that are in-use :(</p>