Just What Are You Testing? Write Tests Intently

15 08 2009

Unit tests need to be robust and reliable. If your tests frequently raise false positives, or worse, fail to report actual errors, then they are not providing the level of comfort they should.

Effective test code, like production code, expresses its intent. Test code that is too general can be brittle, or worse, outright fail.

Here’s a simplified example, based on some code I’ve been working on in TriSano.

class Loinc < ActiveRecord::Base
  validates_presence_of :loinc_code

What we have here is a simple ActiveRecord class, named Loinc. Its only validation ensures that a loinc code is present. If we were to write a spec for this validation, it might look like this:

  it "should not be valid if loinc code is blank" do
    Loinc.create(:loinc_code => nil).should_not be_valid

This test is actually very expressive. In fact, the code reads almost the same as the description. “A new loinc (with no value for loinc code) should not be valid.”

What we’ll start to see, however, is that this code is not actually expressing the proper intent of the test. Let’s make a change:

class Loinc < ActiveRecord::Base
  validates_presence_of :loinc_code
  validates_presence_of :scale_id

Now we’ve added a second validation, on the scale_id field. Our spec still passes, so everything’s hunky-dory, yes? Well, no.

The intent of our test code is to verify that a blank loinc code makes the instance invalid. The code actually tests that a blank loinc code *or* a blank scale id makes the instance invalid.

Pragmatically, this means that we haven’t properly isolated this test.

Nothing is broken yet, but, when merging in the commit that includes the scale id change, let’s assume the developer accidentally merges away the loinc code validation (hey, it happens). So we have:

class Loinc < ActiveRecord::Base
  validates_presence_of :scale_id

When we run our test, it still passes! That wasn’t what we intended at all.

To fix the test, consider what behavior we are expecting. Since this is a Rails app, we are expecting that, If a user tries to create or update a Loinc instance with a blank loinc code, they will receive an error message. That should be the intent of our test.

In code, our spec might look like this:

  it "should produce an error if loinc code is blank" do
    Loinc.create.errors.on(:loinc_code).should == "can't be blank"

Our test is still expressive (well, maybe, a little less expressive), but now it is expressing our programs actual intent, and the validation tests for the loinc code field are isolated from other fields’ validations. If we run our test now, we receive the failure we’d expect.


JRuby on Rails Service No Go (This Time)

19 09 2007

As an official JRuby fanboy, I have been looking for a project at work to use JRuby on Rails on, and finally I thought I’d found one.

I need to write a little service to manage user preferences. Clients need to be able to save and load their list of favorites for a particular activity, along with a list preferred favorites determined by “someone in charge”. In my head, I was seeing a slick little REST service, feeding lists back to the client as XML. Since our application uses a Swing client, I don’t even need to provide an HTML view. It just need to run it on Tomcat against an embedded H2 database.

It almost worked.

There is definately a promising future for this development style. Some things work really well:

  • The service ran in Tomcat just fine. Thanks to Warbler (you rule Nick), it was dead simple to get this running in a Java container. And this was before that whole Glassfish gem whatchacallit was released. I can’t wait to try that!
  • I added some code to environment.rb that would run the migrations in production on start up if it detected that the schema was out of date. Convenient for deploying new releases to clients (we don’t host our applications) but migrations have to be rock solid.
  • Using JNDI, it’s possible to configure where the embedded database files are stored, which is great when twitchy sysadmins start asking about making back ups.

All and all, it would have made a pretty slick solution.

In the end, the death knell for JRuby on this project was that, at the time I had to make the final call, I could not get the script\console or unit tests to work with H2. No tests, in particular, is a big deal. Our shop is pretty open to adopting new technologies. At this time last year we were putting JRuby 0.9.1 into our application to support customizable ETL precesses. But when it came time to commit on this project, I wasn’t comfortable building in JRuby on Rails if I couldn’t deliver a complete Rails application. I have a reputation to maintain, after all.

Oh well, maybe next time.