Hello, and welcome back to CS631 "Advanced Program in the UNIX Environment". As you should remember, we ended our last segment looking at process groups and sessions and how these interact with a controlling terminal. In this video, we'll pick up where we left off and illustrate the concept of job control in the shell, first introduced in the C shell, then adopted by the Korn shell and eventually incorporated in the standard Bourne shell. The idea here was based on the very limiting circumstances of standard terminal interactions in the 80s: a user only had a single terminal, and could thus not run multiple tasks at the same time. The C shell, developed at Berkeley by Bill Joy -- who would later go on to co-found Sun Microsystems -- introduced a way for the shell to suspend the current process and switch another process to the foreground. To this day, this remains one of my all-time favorite productivity hacks: you avoid interrupting whatever it is you're doing, suspend the process or process group mid-execution, do whatever you were going to do, then continue where you left off. Honestly, Control-Z is one of my most frequently used key combinations. --- So what's "job control" really, and how does it work? This is where we left off last time: a login session comprising several process groups, with a foreground process group interacting with the controlling terminal and a session leader being informed of changes from the controlling terminal on hangup, for example. Let's see some examples of what job control and interactions with the controlling terminal look like: --- If we run a command in the shell, we know that this command will be placed into its own process group, as shown here: The shell, with process ID 41, a process leader for the process group 41, and session leader for the session ID 41 fork-exec'd the ps(1) command, then called - setpgid(2) to place the newly created process into its own process group, PID 753 / process group ID 753. --- After ps(1) completes, it exits. As we know, its exit status will linger until its parent process, the shell, waits for it. Now the shell may simply sit there and wait, blocking any other input, so that when the child process terminates, it will immediately reap it. Having done so, the shell then is able to report the exit status, providing for one illustration of how a completed foreground process group can report a change in status to its parent. --- But let's see what happens when we start a process in the background, using the common shell syntax of an ampersand after the command. Not surprisingly, we see the background command being placed in its own process group within this session. [pause] Now if this command completes, nothing much happens, it appears. Since the program is in the background, the shell did not wait(2) for it and block -- instead, it returned immediately and allowed us to enter other commands. But if we hit return... ...then the shell will report that the background process has completed. That is, it must have called wait(2) on the background process to get the exit status information, but clearly it didn't block waiting on the process. So how did the shell know to call wait(2) on its child process? The answer is that when the background process terminated, it generated a signal to notifiy its parent -- SIGCHLD. The parent, our shell, is set up to call wait(2) whenever it receives this signal. - So we see that just as with the foreground process group, the login shell sets the process group of the background process and then can be notified of a change in status. But so we have been talking about the controlling terminal, which allows us to control the foreground process group. So at some point, the login shell - must have allocated such a terminal. That is, it called tcsetpgrp(2) for the controlling terminal, allowing the user to interact with the login shell and the jobs the shell creates. --- How does the interaction with the controlling terminal work? Well, input from the terminal goes to the foreground process - cat(1) in this case. Output from the commands goes to the terminal, too - unless we redirect it to a file. But when we call 'cat file', the output goes to the terminal, as expected. When we provide input via the controlling terminal, cat(1) will call read(2), which now happily blocks until we provide more data. We can stop by using the Ctrl+D keyboard sequence on our controlling terminal, which, by convention, sends a literal EOF character, or we can somehow try to interrupt the command. As I'm sure you know, hitting Ctrl+C on your keyboard causes the program to be interrupted - this is done by, again, the terminal driver sending a signal -- SIGINT, in this case -- to the foreground process group. - So I/O with the controlling terminal interacts with the foreground process group, but certain keyboard sequences may also trigger the generation of certain signals to be delivered to this foreground group. --- Alright, so foreground processes interacting with the controlling terminal aren't very surprising. But what if you have a background process group that wants to perform I/O? If we run 'cat file' in the background, [pause] then the shell will happily put the 'cat' command into the background, and that process will then write data to stdout, which is still connected to the controlling terminal, even though it's in the background. The shell will not wait for the command to terminate before it prints the next command prompt, so we get the result we saw here: the prompt is printed first, and the output kinda garbles the terminal screen. If we then hit return, we get the status confirmation that the command completed. [pause] But that's a bit annoying now, isn't it? If we were doing something here in the foreground, we wouldn't want sme background process to randomly barf whatever it wants to write to the terminal that we're using right now! Fortunately, we can instruct our terminal to not allow that. By using the stty(1) command, we can ask the terminal driver to suspend the background process whenver it wants to generate output, and instead send a signal -- SIGTTOU -- to let us know. [pause] Note that now when we run 'cat file', nothing happens. Our background process group has been suspended, and when we hit return the next time, the shell will report this change in status to us. We can then ask the shell to bring this process to the foreground so it can complete the I/O... - The interactions of the background process group with the controlling terminal then may also trigger notification via a signal -- SIGTTOU in the case of output to the terminal, and SIGTTIN in the case of a background process trying to read data from the terminal. Being able to handle these signals and move process groups from the background to the foreground is what is known as "job control" in the shell. Most shells implement a 'jobs' builtin command that allows you to inspect the status of any current jobs. --- For example, if we run our pipeline from the previous video in the background, then we can run 'jobs' to have it show this process group as two processes, both currently running. With those jobs running in the background, we can now do something else in the foreground, such as edit a source file. If we wish to do something else now, we don't have to close the editor and lose our place in the code: we can instead suspend the current foreground process -- vim, in this case -- by hitting Ctrl+Z, thereby sending the SIGTSTP signal to the editor, which suspends the process, as shown by the shell here. Now we can run other commands and for example create a whole new pipeline running in the foreground. If we decide that we'd like to switch to do something else, we can again hit Ctrl+Z and suspend this entire foreground process group, and 'jobs -l' will show us the status of all of our current processes: 'proc1 | proc2' is still running in the background; vim is suspended, and so is this pipeline. We can now place the a.out pipeline into the background by running 'bg' and specifying job number 3, and 'jobs -l' shows them to now be no longer suspended. We can also bring any of the currently backgrounded process groups to the foreground, which then allows us to interrupt it via the keyboard sequence Ctrl+C. But we can do more: We can suspend any process, including one running in the background, by sending it the TSTP signal using the kill(1) command... ...and 'jobs -l' shows us that this had the expected effect. We can then ask the job to continue as well via the SIGCONT signal, or we can kill it completely. Since we just killed one command that was part of a three-command pipeline, the other two processes are also affected, although each one differently: The first 'a.out' was terminated because the read-end of the pipe it was writing to disappeared -- it received a SIGPIPE signal and exited. The last 'a.out' command on the other hand simply encountered EOF when the second 'a.out' was killed, so this command ended normally. All three commands reported their changed status to the parent process, allowing the 'jobs' command to display their status accurately. --- Ok, let's summarize: - both background and foreground process groups may report a change in status to the login shell - the foreground process group can perform I/O on the controlling terminal - the controlling terminal can generate signals via keyboard interrupts to send to the foreground process group We saw at least EOF (via Ctrl+D), SIGINT (via Ctrl+C), and SIGTSTP (via Ctrl+Z), but there are others that we may revisit in our next video. - the background process group may be able to write to the controlling terminal if the terminal permits this; if not, then the background process group is suspended and the foreground process notified. - That is, the background process group may generate a signal to send to the controlling terminal if it needs to perform I/O. - The shell may move process groups into the foreground or background, suspend or continue them, as we've seen by several examples. - Finally, we can send any signal to any process via the kill(1) command. We'll take a closer look at this command and how signals are delivered as well as what you can do when you receive a signal in our next video. Until then, thanks for watching - cheers!