title: C# BlockingCollection producer consumer without blocking consumer thread
body: <p>I have a situation where I need to have a large number (hundreds) of queues, where the items should be processed in order (need single threaded consumer). My first implementation, <a href="http://msdn.microsoft.com/en-us/library/dd267312.aspx" rel="nofollow">based on the samples,</a> I used a single long-running Task per BlockingCollection to consume the queue items. However, I ended up having an applications with hundreds of threads mostly sitting idle doing nothing but consuming memory, since the queues are empty most of the time.</p>
  
  <p>I thought it would be better to only have a consumer Task running only if there's something in the queue to process, however, I haven't been able to find samples that provide what the best practices should be.</p>
  
  <p>I came up with a solution similar to the one below. But the problem is, every item results in a new Task (maybe this is inefficient? Waste of resources?). But if I don't create a new task for every item, I can't guarantee that an item won't be sitting in the queue unprocessed.</p>
  
  <pre><code>    private object _processSyncObj = new object();
      private volatile bool _isProcessing;
      private BlockingCollection&lt;string&gt; _queue = new BlockingCollection&lt;string&gt;();
  
      private void EnqueueItem(string item)
      {
          _queue.Add(item);
          Task.Factory.StartNew(ProcessQueue);
      }
  
      private void ProcessQueue()
      {
          if (_isProcessing)
              return;
  
          lock (_processSyncObj)
          {
               string item;
               while (_isProcessing = _queue.TryTake(out item))
               {
                   // process item
               }
          }
      }
  </code></pre>
  
  <p>What are the best practices/best solution for this situation with a guarantee that no situation exists where an item is in the queue, but no consumer is running?</p>