Just What Are You Testing? Write Tests Intently

15 08 2009

Unit tests need to be robust and reliable. If your tests frequently raise false positives, or worse, fail to report actual errors, then they are not providing the level of comfort they should.

Effective test code, like production code, expresses its intent. Test code that is too general can be brittle, or worse, outright fail.

Here’s a simplified example, based on some code I’ve been working on in TriSano.

class Loinc < ActiveRecord::Base
  validates_presence_of :loinc_code
end

What we have here is a simple ActiveRecord class, named Loinc. Its only validation ensures that a loinc code is present. If we were to write a spec for this validation, it might look like this:

  it "should not be valid if loinc code is blank" do
    Loinc.create(:loinc_code => nil).should_not be_valid
  end

This test is actually very expressive. In fact, the code reads almost the same as the description. “A new loinc (with no value for loinc code) should not be valid.”

What we’ll start to see, however, is that this code is not actually expressing the proper intent of the test. Let’s make a change:

class Loinc < ActiveRecord::Base
  validates_presence_of :loinc_code
  validates_presence_of :scale_id
end

Now we’ve added a second validation, on the scale_id field. Our spec still passes, so everything’s hunky-dory, yes? Well, no.

The intent of our test code is to verify that a blank loinc code makes the instance invalid. The code actually tests that a blank loinc code *or* a blank scale id makes the instance invalid.

Pragmatically, this means that we haven’t properly isolated this test.

Nothing is broken yet, but, when merging in the commit that includes the scale id change, let’s assume the developer accidentally merges away the loinc code validation (hey, it happens). So we have:

class Loinc < ActiveRecord::Base
  validates_presence_of :scale_id
end

When we run our test, it still passes! That wasn’t what we intended at all.

To fix the test, consider what behavior we are expecting. Since this is a Rails app, we are expecting that, If a user tries to create or update a Loinc instance with a blank loinc code, they will receive an error message. That should be the intent of our test.

In code, our spec might look like this:

  it "should produce an error if loinc code is blank" do
    Loinc.create.errors.on(:loinc_code).should == "can't be blank"
  end

Our test is still expressive (well, maybe, a little less expressive), but now it is expressing our programs actual intent, and the validation tests for the loinc code field are isolated from other fields’ validations. If we run our test now, we receive the failure we’d expect.

Links





Rcov for JRuby Gem Available

31 07 2008

It’s been a long time coming, but, thanks to Jay, who’s really kicked some ass to make this happen, the long suffering Rcov port to JRuby finally is bearing fruit. The first gem is now available.

There are a few known issues. Performance isn’t that great. And there is some slight variation between what Rcov reports in JRuby vs. MRI. But Jay’s been using it in his build environment, and so should you. So go ahead and grab the gem and help us make it great!





Ruby TestCase: Resumable Assertions

14 02 2008

It’s hard to believe that, for all of Smalltalk’s elegance, it isn’t more widely accepted. I just read the SUnit chapter in Squeak By Example. The SUnit TestCase class has a resumable assert method (TestCase#assert:description:resumable:). The use case for this is testing objects in a collection. Here’s a code example:

(1 to: 30) do: [ :each |
    self assert: each even description: each printString, ' is odd' resumable: true]

This code produces a test failure and outputs each object in the collection that fails the test (every odd number, in this example).

I decided that I wanted something like that in Ruby. Well, I want it in Java too, but coding in Java’s no fun 🙂

I was thinking of something like this:

1.upto 30 do | each |
       assert_and_resume each % 2 == 0, "#{each} is odd"
end

After poking through the TestCase source, I decided the easiest approach would be to write a custom assert, wrapping the assert_block method. I just needed to be able to catch the failed assertion and report the failure without stopping the test method.

So I wrangled a little code and came up with a solution that works like this:

1.upto 30 do | each |
       assert_and_resume( "#{ each } is odd" ) { each % 2 == 0 }
end

Pretty close to what I was thinking.

And here’s the implementation:

module Test
  module Unit

    # Resumable assertions are in a different module then the other
    # assertions because they have a dependency on
    # TestCase#add_failure.  The standard assertions can be included
    # anywhere, so including Resumable assertions in the Assertions
    # module might break existing code.
    module ResumableAssertions

      require 'test/unit/assertions'
      def assert_and_resume( description, &block )
        begin
          assert_block description, &block
        rescue AssertionFailedError => e
          add_failure e.message, e.backtrace
        end
      end

    end

    # Now we hook our assertion up to the test case. I could have just
    # re-opened TestCase, but I wanted to keep the Assertion logic
    # separate from the TestCase logic, as was the original author's
    # design.
    require 'test/unit/testcase'
    TestCase.send( :include, ResumableAssertions )

  end
end

Not much to it. The only other change that I might like to make is to modify the output so that it’s clear that one method failed many times. On a lark, I ran this through ci_reporter and then used Antwrap to generate a junit report. Here’s an snapshot of what that report looks like:

Resumable Assert Test Report





Book Avalanche: Design Patterns In Ruby

9 02 2008

Sitting on my desk at work is a copy of trusty the old GoF book. Whenever a new programmer asked me about patterns, that was the book I would hand him. The next time, I think I’ll hand him Design Patterns in Ruby instead. One reason is because I take any chance I can to introduce developers to Ruby, but the main reason is that Russ Olsen does a great job of describing commonly used patterns.

Each pattern description is concise and easy to understand. The code samples lean towards simplistic, but are more then adequate for demonstrating the technique being described. Olsen starts with a code example that resembles how the implementation was describe in GoF. He then morphs the pattern into a more Ruby-ish implementation. Along the way he makes sure to describe the pros and cons of each implementation.

In addition to covering fourteen of the GoF patterns, the book also covers three additional patterns that are popular in the Ruby community. Those are Domain Specific Languages, Convention Over Configuration, and Creating Custom Objects with Meta-programming. Again, the pattern descriptions are clear and the code examples nicely demonstrate the concepts.

If you’re just getting started a developer, and you’re interested in seeing what patterns are all about, Design Patterns in Ruby is a good place to start. If you’re already familiar with patterns, but not so familiar with Ruby, this book gives some good examples of how using Ruby impacts the patterns you’re already familiar. Even if you’re an experienced Rubyist, you may want to pick it up anyway. It’s a quick read and a nice book to have in your lending library.





More “Monkey Patch”-ed Java Classes

28 02 2007

Last time, I demonstrated how to re-open Java objects from JRuby and I used a completely worthless example. This time, the example is only mostly worthless. By that I mean, we’re going to add some Ruby-like aesthetics to a very useful Java library called XOM. The perceived value of this code is directly proportional to how much you believe you should be able to use each, any?, all?, include?, or grep on all collection-like objects from JRuby code.

XOM is a well designed and very opinionated Java library. It is also my preferred library for general purpose XML wrangling. However, because of the principles under which XOM was developed (not implementing collections API, I’m looking at you), certain Ruby conveniences don’t get applied to XOM objects in JRuby. In particular, XOM’s collections don’t implement the Java collections framework. Because of this, XOM collections don’t get decorated with Ruby-like iterators and such. We’re going to change that.

Before we get started, make sure you can reach XOM from JRuby:

1) Download XOM, of course.
2) Set or export your CLASSPATH so that it includes the XOM jar.
3) Fire up jirb.

The classes we’re looking at are Nodes and Elements. I would like for both of these classes to behave more like Ruby array objects. For the Nodes class, which is a modifiable collection, I want [] to be an alias for get, I want << to be an alias for append, and I want to mix in the Ruby Enumerable module. For Elements, which is a read-only collection, I want the [] alias and Enumerable.

So first things first:

require 'java'

XomNodes = Java::nu.xom.Nodes

XomElement = Java::nu.xom.Element

XomElements = Java::nu.xom.Elements

This should look familiar, but you’ll notice that we used the Java module to reference the XOM class objects. This is because the magic that makes JRuby understand that java.lang.String is a Java class, doesn’t work for nu.xom.Nodes, because nu is not a common Java package starter. So we use Java::nu.xom.Nodes, and everything works great.

Now, if we want our collections to behave more like Ruby arrays, then we need to mix Enumerable into our Java objects. Enumerable will not work unless the object has an each method that iterates over the collection, passing each iterated item to yield, in turn.

module XomCollectionExtensions
  include Enumerable

  # Implements each method as described in the Enumerable contract
  def each
    (0...size).each { |i| yield get(i) }
  end
end

What I’ve done here is created my own mixin module. I’ve included Enumerable in my module, and I’ve created the each method. My each implementation just uses an end exclusive Range to iterate over the Nodes in the collection.

Now I can either re-open the Java class and include my module (as well as aliasing the append and get method)…

# modifications to the nu.xom.Nodes to make it behave more
# like a Ruby Array
class XomNodes
  include XomCollectionExtensions

  alias_method :[], :get
  alias_method :<<, :append
end

… or I can send the include and alias_method messages to the class.

XomElements.send :include, XomCollectionExtensions
XomElements.send :alias_method, :[], :get

If you’ve been following along, you should now be able to check out your handiwork:

nodes = XomNodes.new
nodes << XomElement.new('first')
nodes << XomElement.new('second')

nodes.each { |n| p n.to_xml }

Also, if you’re interested in what else can be done with this sort of black magic (besides hacking up your Java objects at runtime), errtheblog has a some nice examples of using alias_method_chain. A welcome bit of warmth that was on this cold and snowy evening, I can tell you.





“Monkey Patch” Java Objects from JRuby

24 02 2007

Sometimes, when you’re programming Java objects in JRuby, you just wish the object acted more… Rubily. JRuby does a pretty good job creating a more Ruby-like interface for Java objects (by aliasing method names and whatnot), but sometimes you want more. Sometimes you want to Monkey Patch those Java objects!

To follow along, install or build JRuby and then fire-up jirb. And, before we go on, please note:

1) I’m using JRuby trunk (3072 I believe). These examples should work in 0.9.2 and 0.9.1, but I haven’t tested them in those environments. Just be aware.

2) “Monkey Patched” Java methods are only available in Ruby code.

3) Aspects of this code are likely to change in the next release.

With that in mind, let’s look at a completely contrived example. Let’s patch java.lang.String. There’s no good reason to do this sort of patching because JRuby marshals strings between Java and Ruby pretty well. But I picked String because everyone should be down with what a String is and how it works, so I can focus on the technique without having to explain the API.

First things first:

require 'java'

This is one of those aspects of the code that will change. In the next release, Java features will be handled in a module, so they will be mixed in, instead of required. But just use require ‘java’ for now. Trust me 🙂

JString = java.lang.String

Here we assign the String class to a constant (JString). We will refer to Java Strings through this constant from now on. Because Ruby objects are just Class objects assigned to constants, this allows us to treat the Java object just like a Ruby object.

So, what do we want to fix about Java Strings? How about this: Ruby strings have size and length methods. Java Strings only have length. If we want JString to behave more like a Ruby String, we’ll have to fix that.

class JString
  def size
    length
  end
end

That’s it. Now take it for a test drive:

s = JString.new('monkey')
p "#{s} | #{s.size}"

You should see something like this:

monkey | 6

Now, just to prove that we did change the behavior of java.lang.String…

s = java.lang.String.new('patch')
p "#{s} | #{s.size}"

Gives you:

patch | 5

That’s how you re-open a Java class in JRuby.

For this particular change, where we just created an aliased name for an existing method, we did things the hard way. Ruby objects accept a message named alias_method that will, as the name suggests, create a second alias for an existing method.

We could re-open the class and alias the method:

class JString
  alias_method :size, :length
end

or, we could just send the alias_method message to the class object.

JString.send :alias_method, :size, :length

This looks like a good place to stop for now, but never fear. I have enough sample code left over for another post. The next one will demonstrate a more practical use of these techniques.





ActiveRecord-JDBC Bug: Numerics retrieved using getInt

31 10 2006

Here’s a fun little bug in the JRuby JDBC adapter for ActiveRecord. Numeric data is retrieved from the JDBC resultset using getInt. This could probably produce some nasty bugs. I created a bug report, and a patch is attached. If anyone else is having this problem, you may want to apply the patch now.