Just What Are You Testing? Write Tests Intently

15 08 2009

Unit tests need to be robust and reliable. If your tests frequently raise false positives, or worse, fail to report actual errors, then they are not providing the level of comfort they should.

Effective test code, like production code, expresses its intent. Test code that is too general can be brittle, or worse, outright fail.

Here’s a simplified example, based on some code I’ve been working on in TriSano.

class Loinc < ActiveRecord::Base
  validates_presence_of :loinc_code
end

What we have here is a simple ActiveRecord class, named Loinc. Its only validation ensures that a loinc code is present. If we were to write a spec for this validation, it might look like this:

  it "should not be valid if loinc code is blank" do
    Loinc.create(:loinc_code => nil).should_not be_valid
  end

This test is actually very expressive. In fact, the code reads almost the same as the description. “A new loinc (with no value for loinc code) should not be valid.”

What we’ll start to see, however, is that this code is not actually expressing the proper intent of the test. Let’s make a change:

class Loinc < ActiveRecord::Base
  validates_presence_of :loinc_code
  validates_presence_of :scale_id
end

Now we’ve added a second validation, on the scale_id field. Our spec still passes, so everything’s hunky-dory, yes? Well, no.

The intent of our test code is to verify that a blank loinc code makes the instance invalid. The code actually tests that a blank loinc code *or* a blank scale id makes the instance invalid.

Pragmatically, this means that we haven’t properly isolated this test.

Nothing is broken yet, but, when merging in the commit that includes the scale id change, let’s assume the developer accidentally merges away the loinc code validation (hey, it happens). So we have:

class Loinc < ActiveRecord::Base
  validates_presence_of :scale_id
end

When we run our test, it still passes! That wasn’t what we intended at all.

To fix the test, consider what behavior we are expecting. Since this is a Rails app, we are expecting that, If a user tries to create or update a Loinc instance with a blank loinc code, they will receive an error message. That should be the intent of our test.

In code, our spec might look like this:

  it "should produce an error if loinc code is blank" do
    Loinc.create.errors.on(:loinc_code).should == "can't be blank"
  end

Our test is still expressive (well, maybe, a little less expressive), but now it is expressing our programs actual intent, and the validation tests for the loinc code field are isolated from other fields’ validations. If we run our test now, we receive the failure we’d expect.

Links

  • TriSano on GitHub
  • Practices of an Agile Developer: Practice 25 – Program Intently and Expressively
  • Advertisements

Actions

Information

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s




%d bloggers like this: