title: Java: Wait in a loop until tasks of ThreadPoolExecutor are done before continuing
body: <p>I'm working on making the Dijkstra algorithm parallel. Per node threads are made to look at all the edges of the current node. This was made parallel with threads but there is too much overhead. This resulted in a longer time than the sequential version of the algorithm.</p>
  
  <p>ThreadPool was added to solve this problem but i'm having trouble with waiting until the tasks are done before I can move on to the next iteration. Only after all tasks for one node is done we should move on. We need the results of all tasks before I can search for the next closest by node.</p>
  
  <p>I tried doing executor.shutdown() but with this aproach it won't accept new tasks. How can we wait in the loop until every task is finished without having to declare the ThreadPoolExecutor every time. Doing this will defeat the purpose of the less overhead by using this instead of regular threads.</p>
  
  <p>One thing I thought about was an BlockingQueue that add the tasks(edges). But also for this solution i'm stuck on waiting for tasks to finish without shudown().</p>
  
  <pre><code>public void apply(int numberOfThreads) {
          ThreadPoolExecutor executor = (ThreadPoolExecutor) Executors.newFixedThreadPool(numberOfThreads);
  
          class DijkstraTask implements Runnable {
  
              private String name;
  
              public DijkstraTask(String name) {
                  this.name = name;
              }
  
              public String getName() {
                  return name;
              }
  
              @Override
              public void run() {
                  calculateShortestDistances(numberOfThreads);
              }
          }
  
          // Visit every node, in order of stored distance
          for (int i = 0; i &lt; this.nodes.length; i++) {
  
              //Add task for each node
              for (int t = 0; t &lt; numberOfThreads; t++) {
                  executor.execute(new DijkstraTask("Task " + t));
              }
  
              //Wait until finished?
              while (executor.getActiveCount() &gt; 0) {
                  System.out.println("Active count: " + executor.getActiveCount());
              }
  
              //Look through the results of the tasks and get the next node that is closest by
              currentNode = getNodeShortestDistanced();
  
              //Reset the threadCounter for next iteration
              this.setCount(0);
          }
      }
  </code></pre>
  
  <p>The amount of edges is divided by the number of threads. So 8 edges and 2 threads means each thread will deal with 4 edges in parallel. </p>
  
  <pre><code>public void calculateShortestDistances(int numberOfThreads) {
  
          int threadCounter = this.getCount();
          this.setCount(count + 1);
  
          // Loop round the edges that are joined to the current node
          currentNodeEdges = this.nodes[currentNode].getEdges();
  
          int edgesPerThread = currentNodeEdges.size() / numberOfThreads;
          int modulo = currentNodeEdges.size() % numberOfThreads;
          this.nodes[0].setDistanceFromSource(0);
          //Process the edges per thread
          for (int joinedEdge = (edgesPerThread * threadCounter); joinedEdge &lt; (edgesPerThread * (threadCounter + 1)); joinedEdge++) {
  
              System.out.println("Start: " + (edgesPerThread * threadCounter) + ". End: " + (edgesPerThread * (threadCounter + 1) + ".JoinedEdge: " + joinedEdge) + ". Total: " + currentNodeEdges.size());
              // Determine the joined edge neighbour of the current node
              int neighbourIndex = currentNodeEdges.get(joinedEdge).getNeighbourIndex(currentNode);
  
              // Only interested in an unvisited neighbour
              if (!this.nodes[neighbourIndex].isVisited()) {
                  // Calculate the tentative distance for the neighbour
                  int tentative = this.nodes[currentNode].getDistanceFromSource() + currentNodeEdges.get(joinedEdge).getLength();
                  // Overwrite if the tentative distance is less than what's currently stored
                  if (tentative &lt; nodes[neighbourIndex].getDistanceFromSource()) {
                      nodes[neighbourIndex].setDistanceFromSource(tentative);
                  }
              }
          }
  
          //if we have a modulo above 0, the last thread will process the remaining edges
          if (modulo &gt; 0 &amp;&amp; numberOfThreads == (threadCounter + 1)) {
              for (int joinedEdge = (edgesPerThread * threadCounter); joinedEdge &lt; (edgesPerThread * (threadCounter) + modulo); joinedEdge++) {
                  // Determine the joined edge neighbour of the current node
                  int neighbourIndex = currentNodeEdges.get(joinedEdge).getNeighbourIndex(currentNode);
  
                  // Only interested in an unvisited neighbour
                  if (!this.nodes[neighbourIndex].isVisited()) {
                      // Calculate the tentative distance for the neighbour
                      int tentative = this.nodes[currentNode].getDistanceFromSource() + currentNodeEdges.get(joinedEdge).getLength();
                      // Overwrite if the tentative distance is less than what's currently stored
                      if (tentative &lt; nodes[neighbourIndex].getDistanceFromSource()) {
                          nodes[neighbourIndex].setDistanceFromSource(tentative);
                      }
                  }
              }
          }
          // All neighbours are checked so this node is now visited
          nodes[currentNode].setVisited(true);
      }
  </code></pre>
  
  <p>Thanks for helping me!</p>