title: Multithreaded job queue manager
body: <p>I need to manage CPU-heavy multitaskable jobs in an interactive application. Just as background, my specific application is an engineering design interface. As a user tweaks different parameters and options to a model,  multiple simulations are run in the background and results displayed as they complete, likely even as the user is still editing values. Since the multiple simulations take variable time (some are milliseconds, some take 5 seconds, some take 10 minutes), it's basically a matter of getting feedback displayed as fast as possible, but often aborting jobs that started previously but are now no longer needed because of the user's changes have already invalidated them. Different user changes may invalidate different computations so at any time I may have 10 different simulations running. Somesimulations have multiple parts which have dependencies (simulations A and B can be seperately computed, but I need their results to seed simulation C so I need to wait for both A and B to finish first before starting C.)</p>
  
  <p>I feel pretty confident that the code-level method to handle this kind of application is some kind of multithreaded job queue. This would include features of submitting jobs for execution, setting task priorities, waiting for jobs to finish, specifying dependencies (do this job, but only after job X and job Y have finished), canceling subsets of jobs that fit some criteria, querying what jobs remain, setting worker thread counts and priorities, and so on. And multiplatform support is very useful too. </p>
  
  <p>These are not new ideas or desires in software, but I'm at the early design phase of my application where I need to make a choice about what library to use for managing such tasks. I've written my own crude thread managers in the past in C (I think it's a rite of passage) but I want to use modern tools to base my work on, not my own previous hacks.</p>
  
  <p>The first thought is to run to <a href="http://openmp.org/wp/" rel="noreferrer">OpenMP</a> but I'm not sure it's what I want. OpenMP is great for parallelizing at a fine level, automatically unrolling loops and such. While multiplatform, it also invades your code with #pragmas. But mostly it's not designed for managing large tasks.. especially cancelling pending jobs or specifying dependencies. Possible, yes, but it's not elegant.</p>
  
  <p>I noticed that <a href="http://www.switchonthecode.com/tutorials/how-google-chrome-stores-passwords" rel="noreferrer">Google Chrome uses such a job manager for even the most trivial tasks.</a> The design goal seems to be to keep the user interaction thread as light and nimble as possible, so anything that can get spawned off asynchronously, should be. From looking at the Chrome source this doesn't seem to be a generic library, but it still is interesting to see how the design uses asynchronous launches to keep interaction fast. This is getting to be similar to what I'm doing.</p>
  
  <p>There are a still other  options:</p>
  
  <p><a href="http://members.gamedev.net/Rivorus/surge/html/index.html" rel="noreferrer">Surge.Act:</a> a Boost-like library for defining jobs. It builds on OpenMP, but does allow chaining of dependencies which is nice. It doesn't seem to feel like it's got a manager that can be queried, jobs cancelled, etc. It's a stale project so it's scary to depend on it.</p>
  
  <p><a href="http://www.codeproject.com/KB/threads/multi_threaded_job_queue.aspx" rel="noreferrer">Job Queue</a> is quite close to what I'm thinking of, but it's a 5 year old article, not a supported library.</p>
  
  <p><a href="http://www.boost.org/doc/libs/1_38_0/doc/html/thread.html" rel="noreferrer">Boost.threads</a> does have nice platform independent synchronization but that's not a job manager. <a href="http://pocoproject.org/" rel="noreferrer">POCO</a> has very clean designs for task launching, but again not a full manager for chaining tasks. (Maybe I'm underestimating POCO though). </p>
  
  <p>So while there are options available, I'm not satisfied and I feel the urge to roll my own library again. But I'd rather use something that's already in existence. Even after searching (here on SO and on the net) I haven't found anything that feels right, though I imagine this must be a kind of tool that is often needed, so surely there's some community library or at least common design.
  On SO there's been some <a href="https://stackoverflow.com/questions/38501/thread-pool-for-executing-arbitrary-tasks-with-different-priorities">posts</a> about <a href="https://stackoverflow.com/questions/125812/managing-threadpool-starvation-within-a-multithreaded-work-queue-processor">job queues</a>, but nothing that seems to fit. </p>
  
  <p>My post here is to ask you all what existing tools I've missed, and/or how you've rolled your own such multithreaded job queue.</p>