title: MSMQ messages bound for clustered MSMQ instance get stuck in outgoing queues
body: <p>We have clustered MSMQ for a set of NServiceBus services, and everything runs great until it doesn't.  Outgoing queues on one server start filling up, and pretty soon the whole system is hung.</p>
  
  <p>More details:</p>
  
  <p>We have a clustered MSMQ between servers N1 and N2.  Other clustered resources are only services that operate directly on the clustered queues as local, i.e. NServiceBus distributors.</p>
  
  <p>All of the worker processes live on separate servers, Services3 and Services4.</p>
  
  <p>For those unfamiliar with NServiceBus, work goes into a clustered work queue managed by the distributor.  Worker apps on Service3 and Services4 send "I'm Ready for Work" messages to a clustered control queue managed by the same distributor, and the distributor responds by sending a unit of work to the worker process's input queue.</p>
  
  <p>At some point, this process can get completely hung.  Here is a picture of the outgoing queues on the clustered MSMQ instance when the system is hung:</p>
  
  <p><img src="https://i.stack.imgur.com/7iiIz.png" alt="Clustered MSMQ Outgoing Queues in Hung State"></p>
  
  <p>If I fail over the cluster to the other node, it's like the whole system gets a kick in the pants.  Here is a picture of the same clustered MSMQ instance shortly after a failover:</p>
  
  <p><img src="https://i.stack.imgur.com/CwYA8.png" alt="Clustered MSMQ Outgoing Queues After Failover"></p>
  
  <p>Can anyone explain this behavior, and what I can do to avoid it, to keep the system running smoothly?</p>