title: Linux - Too many closed connections
body: <p>I'm coding an application opening on a single linux machine 1800 connections/minute using Netty (async nio). A connection lives for a few seconds and then it is closed or it is timeouted after 20 secs if no answer is received. More, the read/write timeout is 30 secs and the request header contains <code>connection=close</code>. 
  After a while (2-3 hours) I get a lot of exceptions in the logs because Netty is unable to create new connections due to a lack of resources. 
  I increased the max number of open files in limits.conf as:</p>
  
  <pre><code>root            hard    nofile           200000
  root            soft    nofile           200000
  </code></pre>
  
  <p>Here is the output of <code>netstat -nat | awk '{print $6}' | sort | uniq -c | sort -n</code>:</p>
  
  <pre><code>   1 established)
     1 FIN_WAIT2
     1 Foreign
     2 TIME_WAIT
     6 LISTEN
    739 SYN_SENT
   6439 LAST_ACK
   6705 CLOSE_WAIT
  12484 ESTABLISHED
  </code></pre>
  
  <p>This is the output of the <code>ss -s</code> command:</p>
  
  <pre><code>Total: 194738 (kernel 194975)
  TCP:   201128 (estab 13052, closed 174321, orphaned 6477, synrecv 0, timewait 3/0), ports 0
  
  Transport Total     IP        IPv6
  *         194975    -         -
  RAW       0         0         0
  UDP       17        12        5
  TCP       26807     8         26799
  INET      26824     20        26804
  FRAG      0         0         0
  </code></pre>
  
  <p>Also <code>ls -l /proc/2448/fd | wc -l</code> gives about 199K.</p>
  
  <p>That said, the questions are about the closed connections reported in the <code>ss -s</code> command output: </p>
  
  <p>1)what are they exactly?</p>
  
  <p>2)why do they keep dangling without being destroyed?</p>
  
  <p>3)Is there any setting (timeout or whatever) which can help to keep them under a reasonable limit?</p>