title: Multithreaded C++ Program Not Running In Parallel Using vector<thread> and .join()
body: <p><strong>Note:</strong> This is the first post I have made on this site, but I have searched extensively and was not able to find a solution to my problem.
  I have written a program which essentially tests all permutations of a vector of numbers to find an optimal sequence as defined by me. Of course, computing permutations of numbers is very time consuming even for small inputs, so I am trying to speed things up by using multithreading.</p>
  
  <hr>
  
  <p>Here is a small sample which replicates the problem:</p>
  
  <pre><code>class TaskObject {
  public:
      void operator()() {
          recursiveFunc();
      }
  private:
      Solution *bestSolution; //Shared by every TaskObject, but can only be accessed by one at a time
      void recursiveFunc() {
          if (base_case) { 
              //Only part where shared object is accessed
              //base_case is rarely reached
              return;
          }
          recursiveFunc();
      }
  };
  
  void runSolutionWithThreads() {
      vector&lt;thread&gt; threads(std::thread::hardware_concurrency());
      vector&lt;TaskObject&gt; tasks_vector(std::thread::hardware_concurrency());
      updateTasks(); //Sets parameters that intialize the first call to recursiveFunc
      for (int q = 0; q &lt; (int)tasks_vector.size(); ++q) {
          threads[q] = std::thread(tasks_vector[q]);
      }
      for (int i = 0; i &lt; (int)threads.size(); ++i) {
          threads[i].join();
      }
  }
  </code></pre>
  
  <p>I imagined that this would enable all threads to run in parallel, but I can see using the performance profiler in visual studio and in the advanced settings of windows task manager that only 1 thread is running at a time. On a system with access to 4 threads, the CPU gets bounded at 25%. <em>I get correct output every time I run, so there are no issues with the algorithm logic. Work is spread out as evenly as possible among all task objects. Collisions with shared data rarely occur. Program implementation with thread pool always ran at nearly 100%.</em></p>
  
  <p>The objects submitted to the threads don't print to cout and all have their own copies of the data required to perform their work except for one shared object they all reference by pointer.</p>
  
  <pre><code>private:
      Solution* bestSolution;
  </code></pre>
  
  <p>This shared data is not susceptible to a data race condition since I used lock_guard from mutex to make it so only one thread can update bestSolution at a time.</p>
  
  <p><strong>In other words, why isn't my CPU running at nearly 100% for my multithreaded program which uses as many threads as there are available in the system?</strong></p>
  
  <p>I can readily update this post with more information if needed.</p>