<![CDATA[Old Code. New Tricks.]]>https://oldcodenewtricks.com/https://oldcodenewtricks.com/favicon.pngOld Code. New Tricks.https://oldcodenewtricks.com/Ghost 3.3Wed, 15 Jan 2020 15:52:23 GMT60<![CDATA[Engineers, Interruptibility, and Inception Layers]]>https://oldcodenewtricks.com/engineers-interruptibility-and-inception-layers/5a847f75338cf30018124571Sun, 17 Apr 2016 06:16:37 GMT

Interruptibility Chart

"Hey, Scott. You got a sec?"

For years, when I needed to get Scott's attention, I'd use this question. I thought it was polite. I wanted to honor his time and not bother him if he was busy. At the same time, I was usually blocked. I tried not to disturb him needlessly. The answer could have well been "no" and my response would have been, "No worries. Get back to me when you can." But it never happened that way. Instead, I got complete and utter frustration.

And I would be confused as to why.

Maybe this is happening to you now as you try to communicate to your engineers.

After about five years, we started to figure out what was going on. One day, after Scott got frustrated, he responded, "I was at level Nine."

"Level Nine?"

"Yeah. Like in the movie Inception."

A light bulb clicked in my brain, and I immediately understood my mistake. I had crumbled his focus just like how dreams collapse in the movie Inception. What I saw when Scott got angry was frustration that I had taken him out of a place where he had built so much focus was so tantalizingly close to a solution, and then had it all ripped away in an instant.

This concept is compounded when you're working on complex problems. In the movie Inception, there's the idea of a "dream within a dream." This happens in engineering too: a mental model within a mental model. The more models you have to keep in your mind at one time, the more time it takes to build up to that state. If you come out of it too quickly, you almost get the mental equivalent of decompression sickness.

We started using this analogy and realized it worked beautifully. The problem with asking engineers if they "have a sec" is that the amount of mental energy needed to evaluate that question will inherently take them out of their flow state. They have to set aside their fragile mental model an attempt to construct something completely different along side it. A series of assessments starts to take place:

  • How long did it take me to build my current mental model?
  • How long will it take me to reconstruct it if I get pulled out of it?
  • Does she mean "a sec" or is this likely going to last twenty minutes?

By the time the engineer starts to focus on the answer to the question "Do you have a sec?" they start to see the mental model of the complex solution break apart, just like dreams do in the movie Inception. In their minds, they shift back over to their original question, but it's too late. That world is crumbling, and it could take hours or sometimes days to get back to that state.

When Scott explained this, I felt for him. I understood how frustrating it is to have a solution so close but then it's torn away. I also understood that context switching is our biggest cost. The more our engineers can focus on a singular problem without having to switch contexts, the better value we deliver to our clients.

At the same time, I needed a way to determine if Scott was interruptible because there was no outward physical sign to tell me if he was at level zero or level nine. So we started experimenting with asking about the state of the inception level before anything else. Now, instead of asking "Got a sec?" I ask, "Inception level?"

Identifying a current inception level takes far less cognitive power than "Got a sec" but it gives the same information. In our framework, anything above a Level Two is uninterruptible.

As we started using this framework, we discovered this works just as well for me when I'm focused on a strategic problem. Since we've started using inception layers to describe our interruptibility, our team is more productive, our engineers are happier, and our desks are intact.

This post originally appeared on Corgibytes.com.

<![CDATA[The Pyramid of Automated Tests]]>

So you've decided that automated testing is a good thing for your team. After all, it's an excellent way to improve productivity and reduce risk. Who wouldn't want that, right?

But then you decide to research how to get started on building an automated testing framework for your existing application,

https://oldcodenewtricks.com/the-pyramid-of-tests/5a847f75338cf3001812456dTue, 29 Mar 2016 06:37:07 GMT


But then you decide to research how to get started on building an automated testing framework for your existing application, and things get a little tricky. Integration tests? Unit tests? Acceptance tests? How do all these tests work together and what should you focus on first?

At Corgibytes, we see this issue frequently. We're often brought on to help a team introduce automated testing into an existing application.

We like to think of testing as a pyramid. The goal is to build an entire pyramid and keep it growing at scale. This means that over time, you'll end up having a lot more unit tests than acceptance tests, but I'm getting a little ahead of myself. First, let's dive into the three different types of automated tests and why they're important.

Pyramid of Tests

Types of Tests

There are three different types of tests to consider when you're building your testing suite:

  • Unit Tests - validates the functionality of a small unit of code in complete isolation from all external systems.
  • Integration Tests - tests the interaction between two or more units
  • Acceptance Tests - ensures all the units involved in a particular feature or workflow work as expected

Unit Tests

What in the world is a unit?
Before we go any further, it's useful to define the word "unit". Pay attention. Units are the building block for the rest of the concepts.

A unit is an intentionally vague term that's meant to be a stand-in for whatever your unit of focus is. If you're in a functional language, your focus would be, guess what — a function! For object-oriented languages, this could be an object or a method. You get the idea. The idea here is that you want to be consistent, so make sure you document what a unit test means to you and your team.

Okay, so with that, now we can proceed.

So, what's a unit test and why is it useful?

A unit test is designed to validate the functionality of a small unit of code in complete isolation from all external systems.

Read that last part again. We emphasized it for a reason. It's cool. I'll get some tea while you let the concept sink in.

Alright, I'm back. You good?

Let's parse that sentence a little bit. Unit tests are the foundation of our pyramid because they validate behavior in very discrete chunks. You don't need access to a file system. You don't need access to a database. You don't need access to a network. You don't need access to any functionality provided by a third party API or framework. If you do connect to any of these things, you do not have a unit test. You have an acceptance or integration test. Very important.

Why keep things so separated?
Well, you want there to be only one reason a test can fail. This way, if a test fails, you know exactly what you need to do to make it pass. Keeping unit tests small and independent is vital for maintaining the speed and focus that make developers more productive. If your unit tests are discrete and not tied to other aspects of your code, your test suite will be zippy, and your poor developers will not be able to rely on the "Sorry, my test suite is running" excuse any longer. Instead, they'll be focused. Keeping the problem domain in their brain and minimizing context switching, which is expensive.

XKCD Comic

Integration and acceptance tests rely on having a group of unit tests already in place, so if you have zero tests, you'll want to focus on getting some unit tests in place first.

A note just for Rails developers
Sorry to pop your balloon, but chances are your app has very few unit tests. That's because it's VERY common for tests that are called unit tests by rails to include database interactions. But as we've learned, a unit test needs to be discrete and not tied to another system. This means that you have lots of shiny integration and acceptance tests, but that you're probably frustrated by how long it's taking your suite to run. We love Rails, but this is one area where it falls short in that the easiest way of doing something isn't the best. I'll plan on going over specific strategies for attacking this in a later post, but if you have questions in the meantime, just let me know.

Integration Tests

Whew! If this were a talk, this would be the point where everyone gets up and stretches. Go ahead. I'll wait.

Okay. And we're back.

As you'll recall from earlier, integration tests validate the connections between two units and the way data flows from one unit to another. These tests are incredibly useful but are slower to run than unit tests because there's more code that needs to run.

When clients complain that their test suite takes too long to run, one of the first things we look at is the ratio between integration and unit tests. When your integration tests lack a solid foundation of unit tests, your test suite becomes incredibly slow. Why? Let's take a familiar example from Rails to illustrate the point.

In Rails, a significant amount of application logic lives directly in controllers and the only way to reliably test controller actions is by simulating an HTTP call. This means that if you only rely on integration tests (which most Rails apps do), then you're repeatedly executing the same logic over and over and over again. Kind of goes against that whole DRY principle thing.

A better way to approach this scenario is to have unit tests that cover all of the different behaviors that are accessed through the controller. Then you'll only need one, maybe two, integration tests that ensure that data is flowing correctly from the HTTP request and the pieces that are already covered by unit tests.

Acceptance Tests

Acceptance tests make sure all of the components in your system are behaving when they all work together. If it sounds like a comprehensive and all-encompassing type of thing, you're spot on. Acceptance tests are useful because they're a great way of translating the criteria that a human would use to determine if a feature is acceptable or not. Product Owners LOVE these, and for good reason. Acceptance tests help ensure the app that they envisioned is working as expected.

There's a downfall, though. Acceptance tests are incredibly slow to run, precisely because they're so comprehensive. So if your test suite is bogged down, super slow, and you're questioning its value, take a look at the ratio between the types of tests you have.

Balance Your Pyramid

The goal here is not to build a test suite entirely out of unit tests. That would be useless. Instead, the idea is to make sure you have enough foundational tests to support the suite and make it run as efficiently as possible.

What's a good ratio? In general, you should aim for two to three times more tests for each descending level. So for each acceptance test you want to write, you'll have around three integration tests and nine unit tests.

If that's ever not true, then your pyramid is out a balance. Imagine what a pyramid would look like if it was upside down. I've worked with test suites that were structured that way: there were more acceptance tests than anything else.

Another way to measure balance is with test coverage. If you collect the test coverage of each level in your pyramid, you should find that your unit tests cover a larger percentage of your app than your integration tests, and your integration tests cover a greater percentage of tests than your unit tests.

You can also measure balance with execution time. Your entire unit test suite should run faster than all of your integration tests, which should run faster than your acceptance tests.

By using the pyramid of tests to design your test suite, you can keep your tests running quickly, mitigate risk, and keep all the members of your team happy and productive.