title: outlier slowness due to thread scheduling?
body: <p>I'm running a multi-threaded C++ application on a Solaris 11. I guess it could happen on Linux as well, so I added 'Linux' tag. It's a quite powerful box, but also runs many applications. Occasionally it's really slow for my application to process a request dealt by a thread from a thread pool. Pseudocode is like below:</p>
  
  <pre><code>start = get_high_resolution_timer();
  indexes = mymap.find(key);
  for i in indexes:
      return_vector.push_back(cached_data[i]); // cached_data is vector&lt;shared_ptr&lt;Something&gt; &gt;
  end = get_high_resolution_timer();
  
  print (end-start)/1000000; // only care about milliseconds number
  </code></pre>
  
  <p>The whole dataset cached_data vector is not huge, and most of times indexes has either 1 or less than 10 indexes. </p>
  
  <p>Normally the print is 0 because it takes less than 1ms. Occasionally it prints 10s or 100s milliseconds. I'm trying to figure out why it takes so long. It should not take couple of milliseconds to iterate the whole dataset and copy everything, I would think. The code might not be cpu cache friendly, but it should not have such a drastic impact, I guess. Is it more likely that OS decided to schedule another thread when current thread executes something between 'start' and 'end'? 
  Thanks,
  Derek</p>