Take the 2-minute tour ×

I use shell scripts to react to system events and update status displays in my window manager. For example, one script determines the current wifi status by listenining to multiple sources:

  1. associate/dissociate events from wpa_supplicant
  2. address changes from ip (so i know when dhcpcd has assigned an address)
  3. a timer process (so the signal strength updates from time to time)

To achieve the multiplexing, I end up spawning background processes:

{ wpa_cli -p /var/run/wpa_supplicant -i wlan0 -a echo &
ip monitor address &
while sleep 30; do echo; done } |
while read line; do update_wifi_status; done &

ie, the setup is that whenever any of the event sources output a line, my wifi status updates. The entire pipeline is run in the background (the final '&') because I also watch another event source that causes my script to terminate:

kill $!

The kill is supposed to clean up the background processes, but in this form it doesn't quite do the job. The 'wpa_cli' and 'ip' processes always survive, at least, and nor do they die on their next event (in theory they should get a SIGPIPE; I guess the reading process must still be alive too).

The question is, how to reliably [and elegantly!] clean up all the background processes spawned?

share|improve this question
The whole cgroups stuff is precisely for handling such situations. Perhaps serverwatch.com/tutorials/article.php/3920051/… and serverwatch.com/tutorials/article.php/3921001/… are of help. –  vonbrand Jan 30 '13 at 11:48
Thanks! The links focus on desktop performance but with a bit more investigation it became obvious how to use cgroups for this case –  sqweek Feb 3 '13 at 2:03
add comment

3 Answers

You can get the PID of each one of them by putting something like


after the wpa_cli,


after the ip, and at the end of the script you kill them both:

kill $PID1
kill $PID2

However, that won't work if the processes fork. Then cgroups would be the best solution.

share|improve this answer
Right. The complication in this case the processes in question are launched in a subshell, which is (a) in the background and (b) part of a pipeline. So, it's not trivial to propagate the saved PID variables back to the parent shell (which is the only shell that knows when to terminate). I'm thinking a construct based on "trap EXIT" would allow me to elegantly clean up the background processes launched within the subshell, and then I just need a reliable way to signal termination to the subshell (which may be as simple as kill $!, not sure). Good point about forking processes. –  sqweek Jan 31 '13 at 11:56
add comment

OK, I've come up with a pretty decent solution that doesn't use cgroups. It won't work in the face of forking processes, as Leonardo Dagnino pointed out.

One of the problems with manually keeping track of process IDs via $! to kill them later is the inherent race condition - if the process finishes before you kill it, the script will send a signal to a non-existant, or possibly incorrect, process.

We can check for process termination within the shell via the wait builtin, but we can only wait for the termination of either all background processes, or a single pid. In both cases wait blocks, which makes it unsuitable for the task of checking whether a given PID is still running.

In searching for a solution to the above I stumbled upon the jobs command, which I previously thought was only available to interactive shells. Turns out it works fine it scripts, and automatically keeps track of the background processes we've launched - if a process has terminated, it won't show up in the jobs list anymore.

Therefore, the command:

trap 'kill $(jobs -p)' EXIT

is enough to ensure -- in simple cases -- the termination of background processes when the current shell exits.

In my case one is not enough, because I'm launching background process from a subshell aswell, and traps are cleared for each new subshell. So, I need to do the same trap within the subshell:

{ trap 'kill $(jobs -p)' EXIT
ip monitor address &
while echo; do sleep 30; done } |
while read line; do update_wifi_status; done &

Finally, jobs -p only gives the pid of the last process in the pipeline (just like $!). As you can see, I'm spawning background processes in the first process of the background pipeline, so I want to signal that pid aswell.

The first process' pid can be obtained from jobs, but I'm not sure how portably this can be achieved. Using bash, I get output in this style:

$ sleep 20 | sleep 20 &
$ jobs -l
[1]+ 25180 Running                 sleep 20
     25181                       | sleep 20 &

So, by using a slightly modified kill command from the parent script, I can signal all processes in the pipeline:

kill $(jobs -l |awk '$2 == "|" {print $1; next} {print $2}')
share|improve this answer
First, jobs -p gives the PID of the first process in a pipe, the process group leader, not the last process. Second, jobs -l lists PIDs of killed or exited processes too - so your potential race condition is there again. Why not put the pipe into a true subshell, opened with (...)? jobs -p returns the PID of this subshell. Or use ps which, in the PPID column, lists the parent PIDs (ie. your $$). Note also the $PPID Bash variable. –  Andreas Spindler Mar 3 '13 at 12:03
Thanks for the corrections! I'm not sure what I would gain by using a true subshell - it doesn't get me any closer to the background processes started on the left of the pipe. To be honest my main gripe with ps is the grotesquely bloated util that is GNU ps, which I've never learned to use well for scripting in favour of saner tools. It sounds like a good approach though... For my case, it looks like pgrep -g $$ gives me exactly what I want (PIDs of all processes belonging to the same process group as $$)... I'm just not 100$ sure where the "process group" boundary is. –  sqweek Mar 8 '13 at 6:06
Actually, I think it's even simpler... the shell calls setpgid() when it starts up to create a new process group, and you can use regular old kill to signal a process group by using a negative number. So... kill -$$ should kill every background task - as long as none of them started their own process group :) –  sqweek Mar 8 '13 at 6:43
add comment
up vote 0 down vote accepted

The super simple solution is to add this at the end of the script:

kill -- -$$


$$ gives us the PID of the running shell. So, kill $$ would send a SIGTERM to the shell process. However, if we negate the PID, kill sends a SIGTERM to every process in the process group. We need the -- beforehand so kill knows that -$$ is a process group ID and not a flag.

When the shell starts, it creates a new process group[1]. Every forked process becomes a member of that process group, unless they explicitly change their process group via a syscall (setpgid).

Thanks to everyone who contributed and made me dig a little deeper, I learnt stuff! :)

[1] is there other circumstances where the shell creates a process group? eg. on starting a subshell? i don't know...

share|improve this answer
add comment

Your Answer


