title: Process threads stuck with __pthread_enable_asynccancel ()
body: <p>I am debugging an issue with a multi-threaded TCP server application on CentOS platform. The application suddenly stopped processing connections. Even the event logging with <code>syslog</code> was not seen in the log files. It was as if the application become a black hole. </p>
  
  <p>I killed the process with signal 11 to get the core dump. In the core dump I observed that all threads are stuck with similar patterns.</p>
  
  <p>Ex:</p>
  
  <pre><code>Thread 19 (Thread 0xb2de8b70 (LWP 3722)):
  #0  0x00c9e424 in __kernel_vsyscall ()
  #1  0x00c17189 in __pthread_enable_asynccancel () from /lib/libpthread.so.0
  #2  0x080b367d in server_debug_stats_timer_handler (tmp=0x0) at server_debug_utils.c:75 ==&gt; this line as a event print with syslog(...)
  </code></pre>
  
  <p>All most all threads are attempting a <code>syslog(...)</code> print but gets stuck with <code>__pthread_enable_asynccancel ()</code>.</p>
  
  <p>What is being done with <code>__pthread_enable_asynccancel ()</code> and why it isn't returning?</p>
  
  <p>Here is info reg from the thread mentioned:</p>
  
  <pre><code>(gdb) info reg
  eax            0xfffffe00       -512
  ecx            0x80     128
  edx            0x2      2
  ebx            0x154ea3c4       357475268
  esp            0xb2de8174       0xb2de8174
  ebp            0xb2de81a8       0xb2de81a8
  esi            0x0      0
  edi            0x0      0
  eip            0xc9e424 0xc9e424 &lt;__kernel_vsyscall+16&gt;
  eflags         0x200246 [ PF ZF IF ID ]
  cs             0x73     115
  ss             0x7b     123
  ds             0x7b     123
  es             0x7b     123
  fs             0x0      0
  gs             0x33     51
  (gdb)
  
  (gdb) print $orig_eax
  $1 = 240
  </code></pre>
  
  <p>(<code>$orig_eax = 240</code> is <code>SYS_futex</code>)</p>
  
  <p>One of the thread state is as shown below:</p>
  
  <pre><code>    Thread 27 (Thread 0xa97d9b70 (LWP 3737)):
  #0  0x00c9e424 in __kernel_vsyscall ()
  #1  0x00faabb3 in inet_ntop () from /lib/libc.so.6
  #2  0x00f20e76 in freopen64 () from /lib/libc.so.6
  #3  0x00f96a55 in fcvt_r () from /lib/libc.so.6
  #4  0x00f96fd7 in qfcvt_r () from /lib/libc.so.6
  #5  0x00a19932 in app_signal_handler (signum=11) at appBaseClass.cpp:920
  #6  &lt;signal handler called&gt;
  #7  0x00f2aec5 in sYSMALLOc () from /lib/libc.so.6
  #8  0x0043431a in CRYPTO_free () from /usr/local/lr/packages/stg_app/5.3.8/lib/ssl/libcrypto.so.10
  #9  0x00000000 in ?? ()
  
  (gdb) print $orig_eax $5 = 240`
  </code></pre>