Unit tests need to be robust and reliable. If your tests frequently raise false positives, or worse, fail to report actual errors, then they are not providing the level of comfort they should.
Effective test code, like production code, expresses its intent. Test code that is too general can be brittle, or worse, outright fail.
Here’s a simplified example, based on some code I’ve been working on in TriSano.
class Loinc < ActiveRecord::Base validates_presence_of :loinc_code end
What we have here is a simple ActiveRecord class, named Loinc. Its only validation ensures that a loinc code is present. If we were to write a spec for this validation, it might look like this:
it "should not be valid if loinc code is blank" do Loinc.create(:loinc_code => nil).should_not be_valid end
This test is actually very expressive. In fact, the code reads almost the same as the description. “A new loinc (with no value for loinc code) should not be valid.”
What we’ll start to see, however, is that this code is not actually expressing the proper intent of the test. Let’s make a change:
class Loinc < ActiveRecord::Base validates_presence_of :loinc_code validates_presence_of :scale_id end
Now we’ve added a second validation, on the scale_id field. Our spec still passes, so everything’s hunky-dory, yes? Well, no.
The intent of our test code is to verify that a blank loinc code makes the instance invalid. The code actually tests that a blank loinc code *or* a blank scale id makes the instance invalid.
Pragmatically, this means that we haven’t properly isolated this test.
Nothing is broken yet, but, when merging in the commit that includes the scale id change, let’s assume the developer accidentally merges away the loinc code validation (hey, it happens). So we have:
class Loinc < ActiveRecord::Base validates_presence_of :scale_id end
When we run our test, it still passes! That wasn’t what we intended at all.
To fix the test, consider what behavior we are expecting. Since this is a Rails app, we are expecting that, If a user tries to create or update a Loinc instance with a blank loinc code, they will receive an error message. That should be the intent of our test.
In code, our spec might look like this:
it "should produce an error if loinc code is blank" do Loinc.create.errors.on(:loinc_code).should == "can't be blank" end
Our test is still expressive (well, maybe, a little less expressive), but now it is expressing our programs actual intent, and the validation tests for the loinc code field are isolated from other fields’ validations. If we run our test now, we receive the failure we’d expect.