{"action":"created","issue":{"url":"https://api.github.com/repos/Terracotta-OSS/terracotta-core/issues/237","repository_url":"https://api.github.com/repos/Terracotta-OSS/terracotta-core","labels_url":"https://api.github.com/repos/Terracotta-OSS/terracotta-core/issues/237/labels{/name}","comments_url":"https://api.github.com/repos/Terracotta-OSS/terracotta-core/issues/237/comments","events_url":"https://api.github.com/repos/Terracotta-OSS/terracotta-core/issues/237/events","html_url":"https://github.com/Terracotta-OSS/terracotta-core/issues/237","id":162541695,"number":237,"title":"start-tc-server.sh does not accurately indicate server termination","user":{"login":"cljohnso","id":8737388,"avatar_url":"https://avatars.githubusercontent.com/u/8737388?v=3","gravatar_id":"","url":"https://api.github.com/users/cljohnso","html_url":"https://github.com/cljohnso","followers_url":"https://api.github.com/users/cljohnso/followers","following_url":"https://api.github.com/users/cljohnso/following{/other_user}","gists_url":"https://api.github.com/users/cljohnso/gists{/gist_id}","starred_url":"https://api.github.com/users/cljohnso/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/cljohnso/subscriptions","organizations_url":"https://api.github.com/users/cljohnso/orgs","repos_url":"https://api.github.com/users/cljohnso/repos","events_url":"https://api.github.com/users/cljohnso/events{/privacy}","received_events_url":"https://api.github.com/users/cljohnso/received_events","type":"User","site_admin":false},"labels":[{"url":"https://api.github.com/repos/Terracotta-OSS/terracotta-core/labels/bug","name":"bug","color":"fc2929"}],"state":"open","locked":false,"assignee":null,"assignees":[],"milestone":null,"comments":1,"created_at":"2016-06-27T20:32:17Z","updated_at":"2016-06-27T22:13:41Z","closed_at":null,"body":"The code in `start-tc-server.sh` relies on the shell `wait` command to coordinate exit of the script with server termination.  Unfortunately, a `TERM` signal to the `start-tc-server.sh` process causes the `wait` to terminate early (while properly propagating `TERM` to the background server).  (From the Bash documentation ... \"_When Bash is waiting for an asynchronous command via the wait builtin, the reception of a signal for which a trap has been set will cause the wait builtin to return immediately with an exit status greater than 128, immediately after which the trap is executed._\")  So, although server process termination is initiated, it is not complete when the `wait` completes and, in addition, the `wait` exit code does not reflect the exit code of the server process.\r\n\r\nThe sample Bash code below can be used to demonstrate both the interrupted `wait` and one possible way of correcting the issue.  The sample solution depends on proper operation of `kill -0 $PID` -- while this is part of the POSIX standard *NIX, there may be some systems for which this does not operation properly (see \"The Zero Signal\" in http://www.linux.org/threads/kill-signals-and-commands-revised.8096/).  A solution relying on `kill -0` will need to be tested on each platform we support.  It may be possible to avoid the use of `kill -0` by relying on retrying `wait` until a 127 (\"not an active child\") is returned.\r\n\r\n----\r\n\r\nThe following is the test script output from a sample run with the TERM handler enabled in the server (background) process -- as it would be in our Java server case:\r\n\r\n```text\r\n$ HANDLE_TERM=true ./tryWaitDriver\r\n0 ./tryWaitDriver starting ./tryWait\r\n0 ./tryWaitDriver pausing 5 seconds before sending TERM to ./tryWait PID=58584\r\n0 ./tryWait starting ... PID=58584\r\n0 ./tryWait Started 58586; rc=0\r\n0 ./tryWait Entering wait 58586\r\n0 ./tryWaitBackground started; PID=58586, PPID=58584\r\n0 ./tryWaitBackground entering sleep 30\r\n5 ./tryWaitDriver sending TERM to ./tryWait PID=58584\r\n5 ./tryWaitDriver waiting for ./tryWait PID=58584 to complete\r\n5 ./tryWait Processing TERM signal; sending TERM to 58586\r\n5 ./tryWait Exited wait 58586; exitValue=143            <== wait interrupted by SIGTERM\r\n5 ./tryWait Entering wait 58586                         <== re-driving wait after interruption (process not done)\r\n30 ./tryWaitBackground received TERM\r\n30 ./tryWaitBackground exiting with rc=7\r\n30 ./tryWait Exited wait 58586; exitValue=7             <== wait completed by background process exit\r\n30 ./tryWait Terminated server; rc=7\r\n30 ./tryWait Exiting; rc=7                              <== Exiting with *real* exit code\r\n30 ./tryWaitDriver wait complete; rc=7\r\n```\r\n\r\nNote the `exitValue` from the following TERM signal processing in `tryWait` -- 143.  This is 128 + `SIGTERM` (15).  The real background process exit code isn't observed for several more seconds and after a second `wait` from `tryWait` -- 7.\r\n\r\n----\r\n\r\nThe following sample output is from the scripts with the TERM handler in the server (background) process _disabled_.  This is included to demonstrate that the `wait` exit code is the same when indicating a SIGTERM-interrupted `wait` or `wait` picking up the exit code of a process without a TERM handler.\r\n\r\n```text\r\n$ HANDLE_TERM=false ./tryWaitDriver\r\n0 ./tryWaitDriver starting ./tryWait\r\n0 ./tryWaitDriver pausing 5 seconds before sending TERM to ./tryWait PID=58648\r\n0 ./tryWait starting ... PID=58648\r\n0 ./tryWait Started 58650; rc=0\r\n0 ./tryWait Entering wait 58650\r\n0 ./tryWaitBackground started; PID=58650, PPID=58648\r\n0 ./tryWaitBackground entering sleep 30\r\n5 ./tryWaitDriver sending TERM to ./tryWait PID=58648\r\n5 ./tryWaitDriver waiting for ./tryWait PID=58648 to complete\r\n5 ./tryWait Processing TERM signal; sending TERM to 58650\r\n5 ./tryWait Exited wait 58650; exitValue=143            <== wait interrupted by SIGTERM\r\n5 ./tryWait Entering wait 58650                         <== re-driving wait after interruption (process not done)\r\n5 ./tryWait Exited wait 58650; exitValue=143            <== wait completed by background process exit\r\n5 ./tryWait Terminated server; rc=143\r\n5 ./tryWait Exiting; rc=143\r\n5 ./tryWaitDriver wait complete; rc=143\r\n```\r\n\r\n----\r\n\r\n####TryWaitDriver####\r\n```bash\r\n#!/bin/bash\r\n# Tests signal handling in BASH.  This script is the driver sscript for the\r\n# ./tryWait and ./tryWaitBackground scripts used to demonstrate the effect of\r\n# SIGTERM on a bash wait command.\r\n#\r\n# Run as \r\n#   ./tryWaitDriver\r\n# or\r\n#   HANDLE_TERM=true ./tryWaitDriver\r\n# enables the TERM signal handler in ./tryWaitBackground.\r\n# Run as\r\n#   HANDLE_TERM=false ./tryWaitDriver\r\n# omits the TERM single handler in ./tryWaitBackground.\r\n#\r\n# When run with the TERM handler in tryWaitBackground enabled, the script shows that wait \r\n# can be terminated by SIGTERM and the wait exit code reflects interruption of the wait \r\n# command and *not* termination of waited-upon (tryWaitBackground) process.  When the TERM \r\n# handler in tryWaitBackground is disabled, the script shows that wait is again still \r\n# terminated by SIGTERM with the wait exit code reflecting the wait interruption.  The code \r\n# in tryWait differs from the code in start-tc-server.sh in that the tryWait version repeats\r\n# the wait (in a loop) until a \"kill -0\" indicates the background (server) process no longer\r\n# exists.  This causes the wait command to be re-driven and the *second* wait provides the \r\n# background process exit code.\r\n#\r\n\r\necho \"$SECONDS $0 starting ./tryWait\"\r\n./tryWait $SECONDS &\r\nPID=$!\r\n\r\necho \"$SECONDS $0 pausing 5 seconds before sending TERM to ./tryWait PID=$PID\"\r\nsleep 5\r\necho \"$SECONDS $0 sending TERM to ./tryWait PID=$PID\"\r\nkill -TERM $PID\r\necho \"$SECONDS $0 waiting for ./tryWait PID=$PID to complete\"\r\nwait $PID\r\necho \"$SECONDS $0 wait complete; rc=$?\"\r\n```\r\n\r\n####TryWait####\r\n```bash\r\n#!/bin/bash\r\n\r\nif [ $1 ]; then\r\n  SECONDS=$1\r\nfi\r\n\r\necho \"$SECONDS $0 starting ... PID=$$\"\r\n\r\nPID=0\r\nfunction cleanup_TERM {\r\n  echo \"$SECONDS $0 Processing TERM signal; sending TERM to $PID\"\r\n  kill -TERM $PID\r\n}\r\ntrap cleanup_TERM SIGTERM\r\n\r\nstart=true\r\nwhile $start; do\r\n  ./tryWaitBackground $SECONDS &\r\n  PID=$!\r\n  rc=$?\r\n  echo \"$SECONDS $0 Started $PID; rc=$rc\"\r\n\r\n  # Loop on wait until the background process is no longer alive.\r\n  while kill -0 $PID 2>/dev/null ; do\r\n    echo \"$SECONDS $0 Entering wait $PID\"\r\n    wait $PID\r\n    exitValue=$?\r\n    echo \"$SECONDS $0 Exited wait $PID; exitValue=$exitValue\"\r\n  done\r\n\r\n  start=false\r\n\r\n  echo \"$SECONDS $0 Terminated server; rc=$exitValue\"\r\ndone\r\n\r\necho \"$SECONDS $0 Exiting; rc=$exitValue\"\r\nexit $exitValue\r\n```\r\n\r\n####tryWaitBackground####\r\n```bash\r\n#!/bin/bash\r\ndelay=30\r\n\r\nif [ $1 ]; then\r\n  SECONDS=$1\r\nfi\r\n\r\necho \"$SECONDS $0 started; PID=$$, PPID=$PPID\"\r\n\r\nif [ \"$HANDLE_TERM\" == \"\" -o \"$HANDLE_TERM\" == \"true\" ]; then\r\n  function handle_TERM {\r\n    echo \"$SECONDS $0 received TERM\"\r\n    echo \"$SECONDS $0 exiting with rc=7\"\r\n    exit 7\r\n  }\r\n  trap handle_TERM SIGTERM\r\nfi\r\n\r\necho \"$SECONDS $0 entering sleep $delay\"\r\nsleep $delay\r\necho \"$SECONDS $0 exiting sleep $delay; rc=$?\"\r\n\r\necho \"$SECONDS $0 exiting with rc=2\"\r\nexit 2\r\n```"},"comment":{"url":"https://api.github.com/repos/Terracotta-OSS/terracotta-core/issues/comments/228892076","html_url":"https://github.com/Terracotta-OSS/terracotta-core/issues/237#issuecomment-228892076","issue_url":"https://api.github.com/repos/Terracotta-OSS/terracotta-core/issues/237","id":228892076,"user":{"login":"jd0-sag","id":16048428,"avatar_url":"https://avatars.githubusercontent.com/u/16048428?v=3","gravatar_id":"","url":"https://api.github.com/users/jd0-sag","html_url":"https://github.com/jd0-sag","followers_url":"https://api.github.com/users/jd0-sag/followers","following_url":"https://api.github.com/users/jd0-sag/following{/other_user}","gists_url":"https://api.github.com/users/jd0-sag/gists{/gist_id}","starred_url":"https://api.github.com/users/jd0-sag/starred{/owner}{/repo}","subscriptions_url":"https://api.github.com/users/jd0-sag/subscriptions","organizations_url":"https://api.github.com/users/jd0-sag/orgs","repos_url":"https://api.github.com/users/jd0-sag/repos","events_url":"https://api.github.com/users/jd0-sag/events{/privacy}","received_events_url":"https://api.github.com/users/jd0-sag/received_events","type":"User","site_admin":false},"created_at":"2016-06-27T22:13:41Z","updated_at":"2016-06-27T22:13:41Z","body":"Given that Windows already needs this (which is why we also need the ability to look up the inferior process PID), in galvan, the corresponding Unix approach would be to invoke kill, directly:  `kill -- -<PID>` (note the extra \"-\" when compared to `kill PID`).\r\n\r\nThis would then allow us to revert the shell script to its original shape."}}
{"actor":{"display_login":"jd0-sag"}}