title: Java High-load NIO TCP server
body: <p>As a part of my research I'm writing an high-load TCP/IP echo server in Java. I want to serve about 3-4k of clients and see the maximum possible messages per second that I can squeeze out of it. Message size is quite small - up to 100 bytes. This work doesn't have any practical purpose - only a research. </p>
  
  <p>According to numerous presentations that I've seen (HornetQ benchmarks, LMAX Disruptor talks, etc), real-world high-load systems tend to serve millions of transactions per second (I believe Disruptor mentioned about 6 mils and and Hornet - 8.5). For example, <a href="https://stackoverflow.com/questions/8316891/how-to-read-efficiently-from-socket-using-java-nio">this post</a> states that it possible to achieve up to 40M MPS. So I took it as a rough estimate of what should modern hardware be capable of. </p>
  
  <p>I wrote simplest single-threaded NIO server and launched a load test. I was little surprised that I can get only about 100k MPS on localhost and 25k with actual networking. Numbers look quite small. I was testing on Win7 x64, core i7. Looking at CPU load - only one core is busy (which is expected on a single-threaded app), while the rest sit idle. However even if I load all 8 cores (including virtual) I will have no more than 800k MPS - not even close to 40 millions :)</p>
  
  <p>My question is: what is a typical pattern for serving massive amounts of messages to clients? Should I distribute networking load over several different sockets inside a single JVM and use some sort of load balancer like HAProxy to distribute load to multiple cores? Or I should look towards using multiple Selectors in my NIO code? Or maybe even distribute the load between multiple JVMs and use Chronicle to build an inter-process communication between them? Will testing on a proper serverside OS like CentOS make a big difference (maybe it is Windows that slows things down)? </p>
  
  <p>Below is the sample code of my server. It always answers with "ok" to any incoming data. I know that in real world I'd need to track the size of the message and be prepared that one message might be split between multiple reads however I'd like to keep things super-simple for now.</p>
  
  <pre><code>public class EchoServer {
  
  private static final int BUFFER_SIZE = 1024;
  private final static int DEFAULT_PORT = 9090;
  
  // The buffer into which we'll read data when it's available
  private ByteBuffer readBuffer = ByteBuffer.allocate(BUFFER_SIZE);
  
  private InetAddress hostAddress = null;
  
  private int port;
  private Selector selector;
  
  private long loopTime;
  private long numMessages = 0;
  
  public EchoServer() throws IOException {
      this(DEFAULT_PORT);
  }
  
  public EchoServer(int port) throws IOException {
      this.port = port;
      selector = initSelector();
      loop();
  }
  
  private void loop() {
      while (true) {
          try{
              selector.select();
              Iterator&lt;SelectionKey&gt; selectedKeys = selector.selectedKeys().iterator();
              while (selectedKeys.hasNext()) {
                  SelectionKey key = selectedKeys.next();
                  selectedKeys.remove();
  
                  if (!key.isValid()) {
                      continue;
                  }
  
                  // Check what event is available and deal with it
                  if (key.isAcceptable()) {
                      accept(key);
                  } else if (key.isReadable()) {
                      read(key);
                  } else if (key.isWritable()) {
                      write(key);
                  }
              }
  
          } catch (Exception e) {
              e.printStackTrace();
              System.exit(1);
          }
      }
  }
  
  private void accept(SelectionKey key) throws IOException {
      ServerSocketChannel serverSocketChannel = (ServerSocketChannel) key.channel();
  
      SocketChannel socketChannel = serverSocketChannel.accept();
      socketChannel.configureBlocking(false);
      socketChannel.setOption(StandardSocketOptions.SO_KEEPALIVE, true);
      socketChannel.setOption(StandardSocketOptions.TCP_NODELAY, true);
      socketChannel.register(selector, SelectionKey.OP_READ);
  
      System.out.println("Client is connected");
  }
  
  private void read(SelectionKey key) throws IOException {
      SocketChannel socketChannel = (SocketChannel) key.channel();
  
      // Clear out our read buffer so it's ready for new data
      readBuffer.clear();
  
      // Attempt to read off the channel
      int numRead;
      try {
          numRead = socketChannel.read(readBuffer);
      } catch (IOException e) {
          key.cancel();
          socketChannel.close();
  
          System.out.println("Forceful shutdown");
          return;
      }
  
      if (numRead == -1) {
          System.out.println("Graceful shutdown");
          key.channel().close();
          key.cancel();
  
          return;
      }
  
      socketChannel.register(selector, SelectionKey.OP_WRITE);
  
      numMessages++;
      if (numMessages%100000 == 0) {
          long elapsed = System.currentTimeMillis() - loopTime;
          loopTime = System.currentTimeMillis();
          System.out.println(elapsed);
      }
  }
  
  private void write(SelectionKey key) throws IOException {
      SocketChannel socketChannel = (SocketChannel) key.channel();
      ByteBuffer dummyResponse = ByteBuffer.wrap("ok".getBytes("UTF-8"));
  
      socketChannel.write(dummyResponse);
      if (dummyResponse.remaining() &gt; 0) {
          System.err.print("Filled UP");
      }
  
      key.interestOps(SelectionKey.OP_READ);
  }
  
  private Selector initSelector() throws IOException {
      Selector socketSelector = SelectorProvider.provider().openSelector();
  
      ServerSocketChannel serverChannel = ServerSocketChannel.open();
      serverChannel.configureBlocking(false);
  
      InetSocketAddress isa = new InetSocketAddress(hostAddress, port);
      serverChannel.socket().bind(isa);
      serverChannel.register(socketSelector, SelectionKey.OP_ACCEPT);
      return socketSelector;
  }
  
  public static void main(String[] args) throws IOException {
      System.out.println("Starting echo server");
      new EchoServer();
  }
  }
  </code></pre>