Wednesday, February 22, 2017

Linux - Hunting of the Shellz with System Tap

If there is one binary that's called the most on Unix platforms it's probably the bash shell. This obviously makes perfect sense given that pulling up a single shell also pulls thousands of tools to your finger tips. Legitimate programs make calls to the shell to execute commands instead of writing their own logic which saves a ton of time and keeps programs lightweight. The ability to pull up a shell is also perhaps one of the most important and consistent features across all malware. Since both legitimate and malicious programs are constantly pulling up shells, how do we know what we should actually be looking for? We'll discuss that here in what is likely to be the first of many different shell hunting blog posts.

A great place to start is by looking for new ttys. A lot of us ignore the importance of the tty even though the creation of one tells us there is hands on activity happening somewhere on the system. The only question that needs to be asked after seeing a new tty get created is do we trust the process that created it? For those unfamiliar with what exactly a tty is, just picture it as the underlying program that's allowing you talk with a shell. For instance, when you open the Terminal application, a new tty is opened allowing you to interact fully with the stdin and stdout. Ttys are actually just special character devices stored in the dev folder. You can view your current tty with the 'tty' command.

There are also items called 'PTY's which are Pseudo-Terminals. This can be thought of shell being used over something like SSH. Where the user is connected to tty device, but that device exists somewhere outside of the local system. We will be able to look at both tty an pty creation using the System Tap language as we did in the last blog post.  Sure, you can craft up a nice little script that looks for tty files being created in the /dev folder, but we could also be more thorough by monitoring for new tty devices at the kernel level using System Tap. Here's the basic code we can use to see when a new tty is created. Upon the creation of a tty the script will return the timestamp, the UID of the user responsible for opening the tty, the name of the process that opened the tty, and its process id.

Let's run this and open a new terminal. We will also login to the system via SSH to ensure that both generate a notification.

Opening a terminal shows that gnome-pty-helpe (presumebly gnome-pty-helper) is the responsible process when opening a terminal via the GUI. When we login via SSH we see that sshd is the responsible process. Don't be confused by the UID 0. This does not mean I logged in as the root user. It means the process that created the TTY is running as root. This makes sense since we're dealing with a daemon.

Based on my system, I would expect to see both of these processes spawning ttys. Therefore, a notification would not be incredibly helpful. Don't get me wrong, logs are always handy for post-mordem analysis, but what I want is a tool that provides notifications when suspicious shells spawn. One approach to take here is to look for ttys spawning from oddball processes on your system. In my use case I'd simply call it normal for "gnome-pty-helpe" and "sshd" to create ttys. Here's my System Tap code to do just that.

Cool. We now have a working script to notify us of ttys from processes we aren't expecting them from.  Nothing too tricky here.

Before we get ahead of ourselves, let's make sure to note the use cases where this script doesn't apply. The last thing you want is some pentester who thinks he's God's gift to hacking telling you he has a way around your detections without understanding what it's actually looking for. Let's demonstrate using a little netcat example. We'll start a listener, connect to it, and run some commands...

Ah. What is this madness? Our tty tracker can't even catch a simple netcat listener? No. THIS IS NOT ITS PURPOSE. The purpose of our script is to catch shells with ttys which netcat does not use. This tty tracker script is not meant to catch all malicious shells, rather it's suppose to catch shells that are guaranteed to be suspicious. If we were looking for all shells without ttys we would have caught this netcat use case, but in the course of about 10 minutes we probably would have caught about a thousand other processes.

However, if the attacker refuses to use a tty he does lose the availability to use interactive programs. For instance, let's say our attacker is in a situation where his backdoor is running as a standard user, but he has figured out the root password and wishes to escalate using sudo.

When running the tty command via netcat we see that we are in fact lacking a tty. When running sudo to escalate to root, the password prompt ends up appearing in the terminal that has the tty. This isn't to say that nothing can be accomplished. We can basically still manage the same amount of damage without a tty, we will just have to work harder for it. But a tty makes life simpler by providing full functionality and therefore is a desirable built in feature for a backdoor.  We can actually get ourselves a tty through this netcat session multiple different ways. Perhaps the easiest is spawning a bash shell using python.

Here we see that the attacker was successfully able to escalate to root using sudo since he had a tty. Of course, now we should have a heads up from our system tap script.

So again, although there are many ways to run malicious commands from shells, this is a way to catch highly interesting hands on activity. It wouldn't be hard to modify this script  to climb process trees allowing us to look for ttys spawning under  web server processes which would also be incredibly abnormal and interesting, but maybe more on that later. :D