title: How to prevent duplicate values in a shared queue
body: <p>A producer thread queries a data store and puts objects into a queue.  Each consumer thread(s) will then pull an object off of the shared queue and do a very long call to an external service.  When the call returns, the consumer marks the object as having been completed.</p>
  
  <p>My problem is that I basically have to wait until the queue is empty before the producer can add to it again, or else I risk getting duplicates being sent through.  </p>
  
  <p>[edit]  Someone asked a good question over IRC and I figured I would add the answer here.  The question was, "Why do your producers produce duplicates?"  The answer is basically that the producer produces duplicates because we don't track a "sending" state of each object, only "sent" or "unsent".</p>
  
  <p>Is there a way that I can check for duplicates in the queue?  </p>