title: How to properly Multithread in OpenCV in 2019?
body: <h3>Background:</h3>
  
  <p>I read some articles and posts regarding Multithreading in OpenCV:</p>
  
  <ul>
  <li>On the one hand you can build OpenCV with TBB or OpenMP support which parallelize OpenCV's functions internally.</li>
  <li>On the other hand you can create multiple threads yourself and call the functions parallel to realize multithreading on application level.</li>
  </ul>
  
  <p>But I couldn't get consistent answers which method of multithreading is the right way to go.</p>
  
  <p>Regarding TBB, an <a href="http://answers.opencv.org/question/696/how-to-enable-vectorization-in-opencv/?answer=699#post-id-699" rel="noreferrer">answer</a> from 2012 with 5 upvotes:</p>
  
  <blockquote>
    <p>With WITH_TBB=ON OpenCV tries to use several threads for some functions. The problem is that just a handsome of function are threaded with TBB at the moment (may be a dozen). So, it is hard to see any speedup. OpenCV philosophy here is that application should be multi-threaded, not OpenCV functions.[...]</p>
  </blockquote>
  
  <p>Regarding multithreading on application level, an <a href="http://answers.opencv.org/question/195828/why-is-threading-slowing-down-over-time/" rel="noreferrer">comment</a> from an moderator on <a href="http://answers.opencv.org" rel="noreferrer">answers.opencv.org</a>:</p>
  
  <blockquote>
    <p>please avoid using your own multithreading with opencv. a lot of functions are explicitly not thread-safe.
    rather rebuild the opencv libs with TBB or openmp support.</p>
  </blockquote>
  
  <p>But another <a href="http://answers.opencv.org/question/32415/thread-safe/?answer=32452#post-id-32452" rel="noreferrer">answer</a> with 3 upvotes is stating:</p>
  
  <blockquote>
    <p>The library itself is thread safe in that you can have multiple calls into the library at the same time, however the data is not always thread safe.</p>
  </blockquote>
  
  <h3>Problem Description:</h3>
  
  <p>So I thought it was at least okay to use (multi)threading on application level. But I encountered strange performance problems when running my program for longer time periods.</p>
  
  <p>After investigating these performance problems I created this minimal, complete, and verifiable example code:</p>
  
  <pre><code>#include "opencv2\opencv.hpp"
  #include &lt;vector&gt;
  #include &lt;chrono&gt;
  #include &lt;thread&gt;
  
  using namespace cv;
  using namespace std;
  using namespace std::chrono;
  
  void blurSlowdown(void*) {
      Mat m1(360, 640, CV_8UC3);
      Mat m2(360, 640, CV_8UC3);
      medianBlur(m1, m2, 3);
  }
  
  int main()
  {
      for (;;) {
          high_resolution_clock::time_point start = high_resolution_clock::now();
  
          for (int k = 0; k &lt; 100; k++) {
              thread t(blurSlowdown, nullptr);
              t.join(); //INTENTIONALLY PUT HERE READ PROBLEM DESCRIPTION
          }
  
          high_resolution_clock::time_point end = high_resolution_clock::now();
          cout &lt;&lt; duration_cast&lt;microseconds&gt;(end - start).count() &lt;&lt; endl;
      }
  }
  </code></pre>
  
  <h3>Actual Behavior:</h3>
  
  <p>If the program is running for an extended period of time the time spans printed by</p>
  
  <pre><code>cout &lt;&lt; duration_cast&lt;microseconds&gt;(end - start).count() &lt;&lt; endl;
  </code></pre>
  
  <p>are getting larger and larger.   </p>
  
  <p>After running the program for around 10 minutes the printed timespans have doubled, which is not explainable with normal fluctuations.</p>
  
  <h3>Expected Behavior:</h3>
  
  <p>The behavior of the program I would expect is that the time spans are staying pretty much constant, even tho they might be longer than calling the function directly.</p>
  
  <h3>Notes:</h3>
  
  <p>When calling the function directly:</p>
  
  <pre><code>[...]
  for (int k = 0; k &lt; 100; k++) {
      blurSlowdown(nullptr);
  }
  [...]
  </code></pre>
  
  <p>The printed time spans are staying constant.</p>
  
  <p>When not calling the cv function:</p>
  
  <pre><code>void blurSlowdown(void*) {
      Mat m1(360, 640, CV_8UC3);
      Mat m2(360, 640, CV_8UC3);
      //medianBlur(m1, m2, 3);
  }
  </code></pre>
  
  <p>The printed time spans are staying constant too. So there must be something wrong when using threading in combination with OpenCV functions.</p>
  
  <ul>
  <li>I know that the code above does NOT achieve actual multithreading there will only be one thread active at the same time that is calling the <code>blurSlowdown()</code> function.</li>
  <li>I know that creating threads and and cleaning them up afterwards is not coming free and will be slower than calling the function directly.</li>
  <li>It is <strong><em>NOT</em></strong> about that the code is slow in general. <strong>The problem is that the printed time spans are getting <em>longer and longer over time</em></strong>.</li>
  <li>The problem is not related to the <code>medianBlur()</code> function since it happens on other with other functions like <code>erode()</code> or <code>blur()</code> too.</li>
  <li>The problem was reproduced under Mac under clang++ see comment by @Mark Setchell</li>
  <li>The problem is amplified when using the debug library instead of the release</li>
  </ul>
  
  <h3>My testing environment:</h3>
  
  <ul>
  <li>Windows 10 64bit</li>
  <li>MSVC compiler</li>
  <li>Official OpenCV 3.4.2 binaries</li>
  </ul>
  
  <h3>My Questions:</h3>
  
  <ul>
  <li>Is it okay to use (multi)threading on application level with OpenCV?</li>
  <li>If yes, why are the time spans printed by my program above <strong><em>GROWING</em></strong> over time?</li>
  <li>If no, why is OpenCV then <a href="http://answers.opencv.org/question/32415/thread-safe/?answer=32452#post-id-32452" rel="noreferrer">considered thread safe</a> and please explain how to interpret the <a href="http://answers.opencv.org/question/696/how-to-enable-vectorization-in-opencv/?answer=699#post-id-699" rel="noreferrer">statement from Kirill Kornyakov</a> instead</li>
  <li>Is TBB / OpenMP in 2019 now widely supported?  </li>
  <li>If yes, what offers better performance, multithreading on application level(if allowed) or TBB / OpenMP?</li>
  </ul>