title: Queuing long running tasks in a web application
body: <p>A user can perform an action on our web app which takes anywhere from 100ms to 10 seconds, I wish to return a result to the browser immediately and then show the results to the user once the task has finished processing. The action is syncing data from a third party and is implemented as a class library (DLL).</p>
  
  <p>Normally it's suggested to use a queue like RabbitMQ or MSMQ and have a worker which writes the results to a database which is polled by an AJAX request from the browser to check for updates.</p>
  
  <p>However the aim is to reduce the latency so it's <em>as close to running the task synchronously as possible</em> while being able to handle spikes in processing the long running task without affecting the rest of the website.</p>
  
  <p><strong>How should the backend be architected?</strong> In my mind, the process would be: starting the task, running the task with minimal latency, notifying the end user the task is finished (ASAP) and finally displaying results in the browser.</p>
  
  <p><img src="https://i.stack.imgur.com/EgBPT.png" alt="Long Running Task. Credits: Haishi. Source: &lt;code&gt;http://haishibai.blogspot.co.uk/2012/12/dealing-with-long-running-jobs.html&lt;/code&gt;"></p>
  
  <p><strong>Examples</strong></p>
  
  <p>Generating sitemaps with <a href="http://www.xml-sitemaps.com/" rel="noreferrer">http://www.xml-sitemaps.com/</a> uses chunked transfer encoding to send a <code>&lt;script&gt;</code> tag every second to call a Javascript function to update the page with the latest status.</p>
  
  <p>Checking SSL certificates with <a href="https://www.ssllabs.com/ssltest/" rel="noreferrer">https://www.ssllabs.com/ssltest/</a> seems to refresh the whole page with an updated status.</p>