title: Named pipes efficient asynchronous design
body: <p><strong>The problem</strong>: </p>
  
  <p>To design an efficient and very fast named-pipes client server framework.</p>
  
  <p><strong>Current state</strong>:</p>
  
  <p>I already have battle proven production tested framework. It is fast, however it uses one thread per one pipe connection and if there are many clients the number of threads could fast be to high. I already use smart thread pool (task pool in fact) that can scale with need.</p>
  
  <p>I already use OVERLAPED mode for pipes, but then I block with WaitForSingleObject or WaitForMultipleObjects so that is why I need one thread per connection on the server side</p>
  
  <p><strong>Desired solution:</strong></p>
  
  <p>Client is fine as it is, but on the server side I would like to use one thread only per client request and not per connection. So instead of using one thread for the whole lifecycle of client (connect / disconnect) I would use one thread per task. So only when client requests data and no more.</p>
  
  <p>I saw an example on MSDN that uses array of OVERLAPED structures and then uses WaitForMultipleObjects to wait on them all. I find this a bad design. Two problems I see here. First you have to maintain an array that can grow quite large and deletions will be costly. Second, you have a lot of events, one for each array member.</p>
  
  <p>I also saw completion ports, like <a href="http://msdn.microsoft.com/en-us/library/windows/desktop/aa363862%28v=vs.85%29.aspx" rel="nofollow">CreateIoCompletionPort</a> and <a href="http://msdn.microsoft.com/en-us/library/windows/desktop/aa364986%28v=vs.85%29.aspx" rel="nofollow">GetQueuedCompletionStatus</a>, but I don't see how they are any better.</p>
  
  <p>What I would like is something <a href="http://msdn.microsoft.com/en-us/library/windows/desktop/aa365468%28v=vs.85%29.aspx" rel="nofollow">ReadFileEx</a> and <a href="http://msdn.microsoft.com/en-us/library/windows/desktop/aa365748%28v=vs.85%29.aspx" rel="nofollow">WriteFileEx</a> do, they call a callback routine 
  when the operation is completed. This is a true async style of programming. But the problem is that ConnectNamedPipe does not support that and furthermore I saw that the thread needs to be in alertable state and you need to call some of the *Ex functions to have that.</p>
  
  <p><strong>So how is such a problem best solved?</strong></p>
  
  <p>Here is how MSDN does it: <a href="http://msdn.microsoft.com/en-us/library/windows/desktop/aa365603%28v=vs.85%29.aspx" rel="nofollow">http://msdn.microsoft.com/en-us/library/windows/desktop/aa365603(v=vs.85).aspx</a></p>
  
  <p>The problem I see with this approach is that I can't see how you could have 100 clients connected at once if the limit to <a href="http://msdn.microsoft.com/en-us/library/windows/desktop/ms687025%28v=vs.85%29.aspx" rel="nofollow">WaitForMultipleObjects</a> is 64 handles. Sure I can disconnect the pipe after each request, but the idea is to have a permanent client connection just like in TCP server and to track the client through whole life-cycle with each client having unique ID and client specific data.</p>
  
  <p>The ideal pseudo code should be like this:</p>
  
  <pre><code>repeat
    // wait for the connection or for one client to send data
    Result = ConnectNamedPipe or ReadFile or Disconnect; 
  
    case Result of
      CONNECTED: CreateNewClient; // we create a new client
      DATA: AssignWorkerThread; // here we process client request in a thread
      DISCONNECT: CleanupAndDeleteClient // release the client object and data
    end;
  until Aborted;
  </code></pre>
  
  <p>This way we have only one listener thread that accepts connect / disconnect / onData events. Thread pool (worker thread) only process the actual request. This way 5 worker threads can serve a lot of clients that are connected.</p>
  
  <p>P.S. 
  My current code should not be important. I code this in Delphi but its pure WinAPI so the language does not matter.</p>
  
  <p><strong>EDIT:</strong></p>
  
  <p>For now IOCP look like the solution:</p>
  
  <blockquote>
    <p>I/O completion ports provide an efficient threading model for
    processing multiple asynchronous I/O requests on a multiprocessor
    system. When a process creates an I/O completion port, the system
    creates an associated queue object for requests whose sole purpose is
    to service these requests. Processes that handle many concurrent
    asynchronous I/O requests can do so more quickly and efficiently by
    using I/O completion ports in conjunction with a pre-allocated thread
    pool than by creating threads at the time they receive an I/O request.</p>
  </blockquote>