Well Crap! A Tech Debt Post

1 09 2009

Well crap! Several weeks ago I was laughing with my co-workers about people who have nothing better to do then argue about the what ‘tech debt’ is. So now I’m posting about what ‘tech debt’ is.

So, I’m an asshole.

Uncle Bob had tweet about messes. This tweet.

I and others debated with him on twitter.

Things quieted down. Then DocOnDev posted this.

I was going to leave a long and rambling comment on Doc’s post. When I finished, I realized that it was long enough and rambling enough to be a blog post. So here we are.

Since arguing with Uncle Bob over that tweet, I’ve had time to think about this, so I’m going to indulge in some rambling commentary:

Writing software is a human endeavour. There will be messes. We may find them distasteful, but they are normal. And you won’t eliminate them by fiat.

If you have more then one committer on a project, you are at risk for accidental code duplication; new developers to the team will make naive mistakes; someone is going to try to sneak that multi-purpose Util class in there somewhere (arrrgghh); and a lot of code just isn’t going to express its intent.

When we go to refactor this code, will we treat it differently because its “distasteful”? Nope. We will test, and we will refactor, until the code is clean, just like any other debt. We need to account for these messes when we are considering how much of our budget goes towards paying back debt.

That being said, if you’re intentionally taking on debt by leaving your code a mess, and excusing it by saying “I’m delivering value faster”, then you’re a fool. How many opportunities will your project miss out on because developers will be cleaning up your mess? After all, this is debt. If you don’t pay for now, you’ll pay for it later, with interest.

Tech Debt isn’t supposed to be about taking a crap in your code base and calling it value. I think that may be what Doc and Uncle Bob are getting at.e


Just What Are You Testing? Write Tests Intently

15 08 2009

Unit tests need to be robust and reliable. If your tests frequently raise false positives, or worse, fail to report actual errors, then they are not providing the level of comfort they should.

Effective test code, like production code, expresses its intent. Test code that is too general can be brittle, or worse, outright fail.

Here’s a simplified example, based on some code I’ve been working on in TriSano.

class Loinc < ActiveRecord::Base
  validates_presence_of :loinc_code

What we have here is a simple ActiveRecord class, named Loinc. Its only validation ensures that a loinc code is present. If we were to write a spec for this validation, it might look like this:

  it "should not be valid if loinc code is blank" do
    Loinc.create(:loinc_code => nil).should_not be_valid

This test is actually very expressive. In fact, the code reads almost the same as the description. “A new loinc (with no value for loinc code) should not be valid.”

What we’ll start to see, however, is that this code is not actually expressing the proper intent of the test. Let’s make a change:

class Loinc < ActiveRecord::Base
  validates_presence_of :loinc_code
  validates_presence_of :scale_id

Now we’ve added a second validation, on the scale_id field. Our spec still passes, so everything’s hunky-dory, yes? Well, no.

The intent of our test code is to verify that a blank loinc code makes the instance invalid. The code actually tests that a blank loinc code *or* a blank scale id makes the instance invalid.

Pragmatically, this means that we haven’t properly isolated this test.

Nothing is broken yet, but, when merging in the commit that includes the scale id change, let’s assume the developer accidentally merges away the loinc code validation (hey, it happens). So we have:

class Loinc < ActiveRecord::Base
  validates_presence_of :scale_id

When we run our test, it still passes! That wasn’t what we intended at all.

To fix the test, consider what behavior we are expecting. Since this is a Rails app, we are expecting that, If a user tries to create or update a Loinc instance with a blank loinc code, they will receive an error message. That should be the intent of our test.

In code, our spec might look like this:

  it "should produce an error if loinc code is blank" do
    Loinc.create.errors.on(:loinc_code).should == "can't be blank"

Our test is still expressive (well, maybe, a little less expressive), but now it is expressing our programs actual intent, and the validation tests for the loinc code field are isolated from other fields’ validations. If we run our test now, we receive the failure we’d expect.


How Fetchmail (and Mutt) Saved Me From Email Mediocrity.

9 08 2009

I’ve always had an ambivalent relationship with email. I used pine in college, but since then, it’s been mostly windows (and windows flavored) email clients.

At the start of my career, it was Outlook. Outlook could read email, but it’s primary feature seemed to be this calendaring thing. Largely it seemed to schedule meetings for me at the busiest times of day, and then canceled them at the last minute. It was easily thwarted by scheduling all day events or just ignoring it altogether.

When Thunderbird stabilized (it’s name; Minotaur? Pbbbbttt!) I decided to switch to that. It also read mail, but had two important features over Outlook. A mediocre feed reader, and not-being-Outlook.

I’ve been using Thunderbird ever since then, though I’ve occasionally tried other clients. Evolution was pretty good. It read mail and integrated with Gnome. It also had this crash-to-desktop feature that I could live without, so I went back to Thunderbird.

At the end of the day, these were all just windows-y applications. You claw at the screen with your mouse all day, dragging little sprite mails into little sprite folders. Configuration is a clicky-clicky exercise of navigating annoying menu tress to overcrowded little dialogs. Setting up one mail box, complete with filtering and signature wiz-bang, is just annoying. Setting up two or three? Even more so.

Why is it so annoying? Why can’t I grab the email from all my accounts and sort it into one set of folders? Why isn’t there a simple text file configuration for all this?

With those thoughts in mind, I started looking for the next great mail client. But in one of the first blogs I read, I stumbled across the magic words: ‘Fetchmail’ and ‘Maildirs’. *HEAD SLAP*

I feel the need to take a short Yegge here to defend my nerd cred.

Please understand that I spent most of my early career on Windows machines. My first programming job was at a VB6 shop.

Windows and Windows applications seem to have a general approach to users that says “Here, let me do that for you, before you break something.” Like how I take the TV remote from my mother when she’s about to “switch it over to DVD”.

Windows says “here, let me install that for you”. Or “here, let me scour the network for printers so all you need to do is pick one from a list” Or, everyone’s favorite, “you seem to be having trouble with that letter. Why don’t we turn it over to this anthropomorphic, animated paper clip. We think he can do it without spilling coffee on the keyboard.”

The point here is not to turn this into a “Windows makes you stupid” rant. The point is, that after years of that learned helplessness, I still sometimes have trouble getting in tune with the Tao of Linux. But I’ve got it now. I still get to be in the nerd club, right? We cool, right?

End Yegge.

So where was I? Oh yeah, ‘Fetchmail’ *SLAP*

Fetchmail is super easy to install and setup. I wish someone had told me about this before.

*Note: All my steps are Ubuntu flavored*

$ sudo apt-get install fetchmail fetchmailconf

I installed fetchmailconf. It was kind of crappy, but it helped me get the global settings configured. After that, I just edited the run control file myself.

So here’s what my ~/.fetchmailrc looks like (sorta)

# Configuration by fetchmailconf
set postmaster "my_username"
set bouncemail
set no spambounce
set properties ""

# don't keep on server because it's my hotmail account and who cares
poll pop3.live.com with proto POP3
       user 'user@hotmail.com' there with password 'secret' is 'my_username' here options ssl

# keep, just because (see, it's IMAP)
poll mail.01.com with proto IMAP
       user 'user@work.com' there with password 'secret' is 'my_username' here options keep ssl

# no keep. gmail is configured to archive after pop access.
poll pop.gmail.com with proto POP3
       user 'user@gmail.com' there with password 'secret' is 'my_username' here options ssl

mda "/usr/bin/maildrop"

This configures fetchmail to pull from three different accounts and deliver all the mail to me. The line that starts with ‘mda’ tells fetchmail not to bother going to port 25, just hand it off to a program called maildrop.

Oh, yeah, I also installed maildrop

$ sudo apt-get install maildrop

Maildrop is what actually sorts the mail. It uses a file called ~/.mailfilter to decide where to deliver the mail. This file contains the the regular expressions that decide what mail folder to deliver a message to.


logfile "$HOME/maildrop.log"

if (/^(To|Cc|Bcc):.*user@work/)
  to $WORK

if (/^X-Apparently-To:.*trisano-dev/)

if (/^(To|Cc|Bcc|Delivered-To):(.*user@hot|.*user@gmail)/)
  to $PLAY


This is an example of a ~/.mailfilter file based on my first attempt. It sets up a default folder for mail. In this case, I’m using the Maildirs format, which I prefer over a single spoolfile. This file also specifies a few subfolders and uses simple regular expressions to decide which folder(s) to deliver a message to.

The regular expressions, by the way, are applied against the entire message, including the headers. So the expression /^X-Apparently-To:.*trisano-dev/, looks for a line starting with the header X-Apparently-To (an awesome header) and then the word trisano-dev (a google groups mailing list, in this case) anywhere else on the line. If it finds a match, it drops that message in the $TRISANO folder.

To make this work, I needed to create the maildirs folders. A utility installs with maildrop that takes care of that. I ran maildirmake once for each folder.

$ maildirmake ~/Maildir
$ maildirmake ~/Maildir/.Trisano
$ maildirmake ~/Maildir/.Play
$ maildirmake ~/Maildir/.Work

While I was getting set up, I ran fetchmail from the command line. Some of the early runs took a while because I had a lot of email laying around on servers.

Once everything worked, I set up cron to run fetchmail every few minutes. I actually use gnome-schedule for that, which is handy, because I don’t speak fluent cron.

$ sudo apt-get install gnome-schedule

Well, I have nicely filtered messages going into my new maildirs. Problem is, Thunderbird can’t read maildirs. But that’s OK, because I was tired of Thunderbird anyway (isn’t that how all this got started?).

I was looking for a nice emacs mail client to use long term but, by happy coincidence, eggyknap was chatting up Mutt in IRC one day, so I decided to give it a try.

$ sudo apt-get install mutt

A simple Mutt configuration looks like this (in ~/.muttrc)

set mbox_type=Maildir

set realname='Ryan L. Bell'

set folder="~/Maildir"
set mask="!^\\.[^.]"
set mbox="~/Maildir"
set record="+.Sent"
set postponed="+.Drafts"
set spoolfile="~/Maildir"
set signature="~/.sig"
set sort=threads

This tells Mutt I’m using Maildirs and that my ~/Maildir folder should be opened by default. It also tells Mutt what folders to use for Sent messages and saving email Drafts. These folders will be in Maildir format, but Mutt handles creating them, so Bob’s your uncle.

The last two lines tell Mutt which signature file to use for outgoing messages, and tells Mutt to sort using threads, by default.

I didn’t think I could face a sendmail configuration, so for simplicity’s sake, I just have Mutt talking directly to my smtp server. One day I may try using Lamson for this. We’ll see.

Again, from the ~/.muttrc file:

# Outgoing
set smtp_url="smtp://user@smtp.someserver.com:587"
set smtp_pass="secret"

Next, I set up Mutt to know about all my maildirs. This was not strictly necessary, but it makes navigating between folders in Mutt much more pleasant.

# Mailboxes
mailboxes "=.Play"
mailboxes "=.Work"
mailboxes "=.Trisano"
mailboxes "=.Sent"
mailboxes "=.Drafts"

The last things I added to my mutt configuration were these folder hooks.

# Folder hooks
folder-hook .*Play    set from="user@home.com"
folder-hook .*Play    set signature="~/.sig"

folder-hook .*Work    set from="user@work.com"
folder-hook .*Work    set signature="~/.sig-work"

folder-hook .*Trisano set from="user@work.com"
folder-hook .*Trisano set signature="~/.sig-work"

These hooks set my Reply-To header and signature file based on what folder I am in.

This barely scratches the surface of what you can configure Mutt to do. I’ve included a link to the Mutt configuration page in the links section.

Now I have exactly the email configuration I desired. I can control my mail filtering with regular expressions and a simple text file. I can grab mail from all my accounts and sort it into a simple set of folders. With Mutt, I have plenty of hooks to change the behavior of the client to suit my needs. I can even modify the message headers when a message is sent.

So what will I do with all this new found power and control? Who knows? But it won’t be boring.

Whew! Ran a little long this time. I guess I’ve been storing up some blogging action points. As always, corrections, additions, and enhancements are always welcome in the comments.


Run IE in VirtualBox

20 01 2009

I tweeted this earlier today, but decided to blog it too. Here is an excellent screen cast on running IE (well, Windows too) in VirtualBox. Since the vhd’s are from Windows, and for the express purpose of cross browser testing, I’m assuming everything is above board.

A couple of notes: The screen cast is for OS X, but everything worked fine on Ubuntu. I just did a couple of things differently. I used wine to extract the exe’s, rather then 7zip. I also received a funky error from VirtualBox when I tried to run the image, but the error description told me exactly how to fix it.

The last thing I’ll note is that many comenters had problems w/ VirtualBox 2.1.0, so stick to 2.0.6.

Run Skype on Ubuntu 8.04

13 01 2009

I’ve had Skype working on Ubuntu for a while, but I thought I’d blog it so it would be easier to find if I had to do it again.

To get Skype, I just downloaded the Ubuntu bundle from Skype’s site. Or you can go and checkout their other linux downloads.

For the record, I was not having trouble making calls. Once I was on a call, dialing numbers produced funny echo sounds on the line. This made it impossible to participate in conference calls because I couldn’t navigate phone trees.

Here’s how I fixed it:

Get the pulseaudio-utils.

sudo apt-get install pulseaudio-utils

Then, run skype through padsp.

padsp skype

That is all.

Rcov Update For JRuby 1.1.4

31 08 2008

Just a quick post. An update that makes rcov compatible with JRuby 1.1.4 has been released. Here’s the announcement on the user’s list. Again, I’d like to commend Jay on his work on this. I really have not had the time to work on this at all, and Jay is the only reason anything has been released at all. Thanks Jay!

Rcov for JRuby Gem Available

31 07 2008

It’s been a long time coming, but, thanks to Jay, who’s really kicked some ass to make this happen, the long suffering Rcov port to JRuby finally is bearing fruit. The first gem is now available.

There are a few known issues. Performance isn’t that great. And there is some slight variation between what Rcov reports in JRuby vs. MRI. But Jay’s been using it in his build environment, and so should you. So go ahead and grab the gem and help us make it great!