author avatar
By Joshua CurtisSoftware Engineer

*Views, thoughts, and opinions expressed in this post belong solely to the author, and not necessarily to SemanticBits.

Tips and tricks for folks who are new to unit testing

When I started with SemanticBits many moons ago, I was fairly confident in my ability to write good, functional code and to account for how changes that were introduced would interact with other elements of my codebase. After all, I’d come from a place where I was the sole developer on a substantial project that I’d built from the ground up. I had a very solid understanding of Node.js and MySQL, a good understanding of PHP, and absolutely no exposure to the newfangled front-end technologies like React.js and Angular.js, which seemed unnecessarily complex for my simple app (I’d taken that time to learn DOM manipulation).

So, when I was asked about testing methodology during my interview with SemanticBits, I froze. I’m good enough at improv that I was able to come up with a reasonable-sounding answer, but I ultimately had to admit to my team after I was hired that I knew nothing, and they’d have to help me. It was fascinating to learn that I wasn’t the only one who’d started that way. As it turns out, writing darn good tests is something that ought to come naturally to anyone who has ever done an ounce of quality control in their own codebase. Prior to SemanticBits, my testing consisted of generating an input that would hit the code I’d changed and running it through the functions—usually through a REPL or Postman—to make sure that the output had changed appropriately.

I was then overjoyed to learn that it worked that way in automated testing suites, as well.

Why Do We Test?

On a recent work trip, I checked into my hotel room after several hours of travel, and decided to take a shower. The bathroom had been recently renovated. There was no tub to climb over. The tile work looked completely new. And the shower fixture featured two shower heads: one on a hose and one fixed at the top of a long shaft. It was fancy—fancier than I’m used to seeing in most private bathrooms, let alone a hotel bathroom.

After my shower, I was greeted with a small lake on the floor of the bathroom, and I needed to scramble to use all of the towels that housekeeping had provided just to keep it contained as it edged towards the hallway. As I cleaned up the mess whose blame split between myself and whichever contractor had created this space, a thought occurred to me:

This entire situation could have been avoided if the contractor had tested what they made. Instead, they opted to sign off on it when it looked complete. As a result, a tired traveler who just wanted to sleep was forced to contain the result of their error, which made the hotel and the contractor look bad to me.

But this is why we test—to prevent avoidable mistakes from making it out into the wild. And it’s ever-important in software development. 

How To Write Some Darn Good Tests

To boil it down, a good test should take a specific input condition and assert a specific output condition (rather than the entirety of a complex output), and a good testing suite should cover as many input conditions as there are logical switches in this way. This first principle is called specificity, and the second is thoroughness.

So, darn good tests must be specific, and they must be thorough.

It’s a tall order, but there is a methodology that you can follow to help fulfill both of these principles. The gist is that you need to map out your test cases by determining what inputs should produce what results, and the way we accomplish specificity and thoroughness is by examining our inputs and logical conditions.

Let’s take a look at this sample function that we want to test:

def flagDiv(num: Int, den: Int, flag: Boolean) = {
    if (flag
  || den == 0 || num == 0
  || den == null || num == null) {
    return 0
  } else {
    val fraction = num/den
    if (fraction > 10) {
      return 10
    } else if (fraction < 0) {
      return 0
    } else 
      return fraction
    }
  }
  }

The novice test author might make an assertion that testing num = 1 and den = 2 returns a result of 0.5. But this is woefully incomplete. That only covers one line of the function!

If we want to test this function specifically and thoroughly, the first thing we need to do is look at our inputs and test conditions. We have three inputs: num, den, and the boolean flag. So, let’s make a table with those inputs to track their values, and then an extra column for the output:

num den flag output

When feasible, you should always test all possible inputs, which specifically relates to inputs that have a small set of possible values. In this case, the boolean flag parameter is the only one that can be completely mapped, which gives us two values to consider in our test cases:

num den flag output
true 0
false

Note how when flag is true, the output that we’re going to test is 0. This harkens to the first if statement, where flag being true will trigger that output. While it may seem asinine to assert this result, it is important to do in case this function or code block is ever refactored later. You want to make sure that any changes will honor and preserve that logic.

Now, what other logical conditions can produce an output of 0? If we look at that block again, we get a handful of conditions: den == null, den == 0, and num == null, and num == 0. Mapped to the table, we get:

num den flag output
null 0
0 0
null 0
0 0

And here’s where specificity comes in: we need to work through this to cover every possible scenario for that code block. Going back to the discussion about the boolean flag, we know that there are three relevant values for both den and num: null, 0, and not zero. So, we can count it up, creating the table as follows:

num den flag output
null null true 0
null null false 0
null 0 true 0
null 0 false 0
null 1 true 0
null 1 false 0
0 null true 0
0 null false 0
0 0 true 0
0 0 false 0
0 1 true 0
0 1 false 0
1 null true 0
1 null false 0
1 0 true 0
1 0 false 0
1 1 true 0
1 1 false != 0

So, to create this table, we effectively rotated through our input values at each step and then determined what the result should look like. What are the possible values of num? For each value of num, what are the possible values for den? For each value of num and den, what are the possible values of flag? It’s an iterative process, and it’s complete. And since we’re only testing the first logic block, once we have a collection of inputs that results in a non-zero value, we only care that the function should not return 0—not necessarily a specific value at this point. While you could make that assertion, that deviates from specificity.

So, then, what other conditions do we need to test in this function? Once again, if we look at the code block, we see that the two other conditions to test are fraction > 10 and fraction < 0. Since fraction is defined as num / dem, we can use that to populate our inputs to find values that can satisfy those conditions. As such:

num den flag output
15 1 false 10
-1 1 false 0

And then, finally, once we’ve gotten to the point where all other logical conditions have been tested, we can test the division operation:

num den flag output
6 2 false 3

So, then, what does our total collection of tests look like?

num den flag output
null null true 0
null null false 0
null 0 true 0
null 0 false 0
null 1 true 0
null 1 false 0
0 null true 0
0 null false 0
0 0 true 0
0 0 false 0
0 1 true 0
0 1 false 0
1 null true 0
1 null false 0
1 0 true 0
1 0 false 0
1 1 true 0
1 1 false != 0
15 1 false 10
-1 1 false 0
6 2 false 3

Is it a lot of tests? Yes, but this is what a thorough, specific test suite looks like. So, what do the assertions look like?

describe("flagDiv tests") {
    it("should properly handle cases that return 0") {
      assert(flagDiv(null, null, true) == 0)
      assert(flagDiv(null, null, false) == 0)
      assert(flagDiv(null, 0, true) == 0)
      assert(flagDiv(null, 0, false) == 0)
      assert(flagDiv(null, 1, true) == 0)
      assert(flagDiv(null, 1, false) == 0)
      assert(flagDiv(0, null, true) == 0)
      assert(flagDiv(0, null, false) == 0)
      assert(flagDiv(0, 0, true) == 0)
      assert(flagDiv(0, 0, false) == 0)
      assert(flagDiv(0, 1, true) == 0)
      assert(flagDiv(0, 1, false) == 0)
      assert(flagDiv(1, null, true) == 0)
      assert(flagDiv(1, null, false) == 0)
      assert(flagDiv(1, 0, true) == 0)
      assert(flagDiv(1, 0, false) == 0)
      assert(flagDiv(1, 1, true) == 0)
      assert(flagDiv(1, 1, false) != 0)
    }
    it("should handle minimum and maximum values") {
      assert(flagDiv(15, 1, false) == 10)
      assert(flagDiv(-1, 1, false) == 0)
    }
    it("should properly divide the inputs") {
      assert(flagDiv(6, 2, false) == 3)
    }
  }

This produces a long set of tests and assertions compared to a relatively short function, but it is the best way to ensure that your testing suite covers as many cases as possible.

Handling Business Rules

The trickier part of all of this is taking your code and your tests and mapping them to business rules. For a function like this, what if the business rule was to never return a value below 1? Then we’d need to write a test for that, making sure to model our data such that it would produce the desired result, and also make sure that our assertions are targeted and specific to the result we want:

describe("function return is never less than 1") {
    it("should not return less than one if num is less than den") {
      assert(flagDiv(2, 4, false) == 1) // this will fail
    }
  }

Why would we write tests for rules that we know will fail? The logic for the function above clearly doesn’t support this. So, why would we even bother to write the test?

The demonstration function is an extremely simple item. Often in our development, we will encounter incredibly complex workspaces with immense call stacks and constantly mutating data spaces. We may make a change in the code, thinking we’ve satisfied a given requirement, but our change might be wrong or else in the wrong place. Writing the test helps us check ourselves, and make sure that the requirements are met.

In this case, the business rule was for the divFlag function to never return a result less than 1. The function was not written to match that, so, naturally, the test for it will fail. But if we didn’t write that test, thinking that our function was correct, we might have shipped bad code (oh no!).

As a note, there is something to be said for the overhead of having to write a separate assertion method for every requirement and test case, which can quickly make your testing library unwieldy. There is an answer to that, but that’s a subject for a different blog post (please look forward to it!).

Conclusion

Testing is important. If the contractor who finished the shower in my hotel room had opted to test the full range of the shower head, they would have recognized the potential for failure, which could have caused a more careless tenant to damage the hotel itself. They might have been able to request more resources from the hotel to install a drain outside the shower or to use a glass shower door instead of a hanging curtain. If they had done more than make sure the shower turned on and the temperature could be controlled, the problem I encountered could have been prevented entirely.

We test our work because we want to cover our own skin, and because it helps other developers work with the logical patterns we’ve established. If we write darn good tests that are specific and thorough, we can make sure that our code shines, that we look good, that our team looks good, and that our company looks good. Darn good tests are a staple of darn good developers, which is something I hope we’re all aspiring to be.