How Fetchmail (and Mutt) Saved Me From Email Mediocrity.

9 08 2009

I’ve always had an ambivalent relationship with email. I used pine in college, but since then, it’s been mostly windows (and windows flavored) email clients.

At the start of my career, it was Outlook. Outlook could read email, but it’s primary feature seemed to be this calendaring thing. Largely it seemed to schedule meetings for me at the busiest times of day, and then canceled them at the last minute. It was easily thwarted by scheduling all day events or just ignoring it altogether.

When Thunderbird stabilized (it’s name; Minotaur? Pbbbbttt!) I decided to switch to that. It also read mail, but had two important features over Outlook. A mediocre feed reader, and not-being-Outlook.

I’ve been using Thunderbird ever since then, though I’ve occasionally tried other clients. Evolution was pretty good. It read mail and integrated with Gnome. It also had this crash-to-desktop feature that I could live without, so I went back to Thunderbird.

At the end of the day, these were all just windows-y applications. You claw at the screen with your mouse all day, dragging little sprite mails into little sprite folders. Configuration is a clicky-clicky exercise of navigating annoying menu tress to overcrowded little dialogs. Setting up one mail box, complete with filtering and signature wiz-bang, is just annoying. Setting up two or three? Even more so.

Why is it so annoying? Why can’t I grab the email from all my accounts and sort it into one set of folders? Why isn’t there a simple text file configuration for all this?

With those thoughts in mind, I started looking for the next great mail client. But in one of the first blogs I read, I stumbled across the magic words: ‘Fetchmail’ and ‘Maildirs’. *HEAD SLAP*

I feel the need to take a short Yegge here to defend my nerd cred.

Please understand that I spent most of my early career on Windows machines. My first programming job was at a VB6 shop.

Windows and Windows applications seem to have a general approach to users that says “Here, let me do that for you, before you break something.” Like how I take the TV remote from my mother when she’s about to “switch it over to DVD”.

Windows says “here, let me install that for you”. Or “here, let me scour the network for printers so all you need to do is pick one from a list” Or, everyone’s favorite, “you seem to be having trouble with that letter. Why don’t we turn it over to this anthropomorphic, animated paper clip. We think he can do it without spilling coffee on the keyboard.”

The point here is not to turn this into a “Windows makes you stupid” rant. The point is, that after years of that learned helplessness, I still sometimes have trouble getting in tune with the Tao of Linux. But I’ve got it now. I still get to be in the nerd club, right? We cool, right?

End Yegge.

So where was I? Oh yeah, ‘Fetchmail’ *SLAP*

Fetchmail is super easy to install and setup. I wish someone had told me about this before.

*Note: All my steps are Ubuntu flavored*

$ sudo apt-get install fetchmail fetchmailconf

I installed fetchmailconf. It was kind of crappy, but it helped me get the global settings configured. After that, I just edited the run control file myself.

So here’s what my ~/.fetchmailrc looks like (sorta)

# Configuration by fetchmailconf
set postmaster "my_username"
set bouncemail
set no spambounce
set properties ""

# don't keep on server because it's my hotmail account and who cares
poll pop3.live.com with proto POP3
       user 'user@hotmail.com' there with password 'secret' is 'my_username' here options ssl

# keep, just because (see, it's IMAP)
poll mail.01.com with proto IMAP
       user 'user@work.com' there with password 'secret' is 'my_username' here options keep ssl

# no keep. gmail is configured to archive after pop access.
poll pop.gmail.com with proto POP3
       user 'user@gmail.com' there with password 'secret' is 'my_username' here options ssl

mda "/usr/bin/maildrop"

This configures fetchmail to pull from three different accounts and deliver all the mail to me. The line that starts with ‘mda’ tells fetchmail not to bother going to port 25, just hand it off to a program called maildrop.

Oh, yeah, I also installed maildrop

$ sudo apt-get install maildrop

Maildrop is what actually sorts the mail. It uses a file called ~/.mailfilter to decide where to deliver the mail. This file contains the the regular expressions that decide what mail folder to deliver a message to.

DEFAULT="$HOME/Maildir/"
TRISANO="$DEFAULT/.Trisano"
WORK="$DEFAULT/.Work"
PLAY="$DEFAULT/.Play"

logfile "$HOME/maildrop.log"

if (/^(To|Cc|Bcc):.*user@work/)
{
  to $WORK
}

if (/^X-Apparently-To:.*trisano-dev/)
{
  to $TRISANO
}

if (/^(To|Cc|Bcc|Delivered-To):(.*user@hot|.*user@gmail)/)
{
  to $PLAY
}

to $DEFAULT

This is an example of a ~/.mailfilter file based on my first attempt. It sets up a default folder for mail. In this case, I’m using the Maildirs format, which I prefer over a single spoolfile. This file also specifies a few subfolders and uses simple regular expressions to decide which folder(s) to deliver a message to.

The regular expressions, by the way, are applied against the entire message, including the headers. So the expression /^X-Apparently-To:.*trisano-dev/, looks for a line starting with the header X-Apparently-To (an awesome header) and then the word trisano-dev (a google groups mailing list, in this case) anywhere else on the line. If it finds a match, it drops that message in the $TRISANO folder.

To make this work, I needed to create the maildirs folders. A utility installs with maildrop that takes care of that. I ran maildirmake once for each folder.

$ maildirmake ~/Maildir
$ maildirmake ~/Maildir/.Trisano
$ maildirmake ~/Maildir/.Play
$ maildirmake ~/Maildir/.Work

While I was getting set up, I ran fetchmail from the command line. Some of the early runs took a while because I had a lot of email laying around on servers.

Once everything worked, I set up cron to run fetchmail every few minutes. I actually use gnome-schedule for that, which is handy, because I don’t speak fluent cron.

$ sudo apt-get install gnome-schedule

Well, I have nicely filtered messages going into my new maildirs. Problem is, Thunderbird can’t read maildirs. But that’s OK, because I was tired of Thunderbird anyway (isn’t that how all this got started?).

I was looking for a nice emacs mail client to use long term but, by happy coincidence, eggyknap was chatting up Mutt in IRC one day, so I decided to give it a try.

$ sudo apt-get install mutt

A simple Mutt configuration looks like this (in ~/.muttrc)

set mbox_type=Maildir

set realname='Ryan L. Bell'

set folder="~/Maildir"
set mask="!^\\.[^.]"
set mbox="~/Maildir"
set record="+.Sent"
set postponed="+.Drafts"
set spoolfile="~/Maildir"
set signature="~/.sig"
set sort=threads

This tells Mutt I’m using Maildirs and that my ~/Maildir folder should be opened by default. It also tells Mutt what folders to use for Sent messages and saving email Drafts. These folders will be in Maildir format, but Mutt handles creating them, so Bob’s your uncle.

The last two lines tell Mutt which signature file to use for outgoing messages, and tells Mutt to sort using threads, by default.

I didn’t think I could face a sendmail configuration, so for simplicity’s sake, I just have Mutt talking directly to my smtp server. One day I may try using Lamson for this. We’ll see.

Again, from the ~/.muttrc file:

#
# Outgoing
#
set smtp_url="smtp://user@smtp.someserver.com:587"
set smtp_pass="secret"

Next, I set up Mutt to know about all my maildirs. This was not strictly necessary, but it makes navigating between folders in Mutt much more pleasant.

#
# Mailboxes
#
mailboxes "=.Play"
mailboxes "=.Work"
mailboxes "=.Trisano"
mailboxes "=.Sent"
mailboxes "=.Drafts"

The last things I added to my mutt configuration were these folder hooks.

#
# Folder hooks
#
folder-hook .*Play    set from="user@home.com"
folder-hook .*Play    set signature="~/.sig"

folder-hook .*Work    set from="user@work.com"
folder-hook .*Work    set signature="~/.sig-work"

folder-hook .*Trisano set from="user@work.com"
folder-hook .*Trisano set signature="~/.sig-work"

These hooks set my Reply-To header and signature file based on what folder I am in.

This barely scratches the surface of what you can configure Mutt to do. I’ve included a link to the Mutt configuration page in the links section.

Now I have exactly the email configuration I desired. I can control my mail filtering with regular expressions and a simple text file. I can grab mail from all my accounts and sort it into a simple set of folders. With Mutt, I have plenty of hooks to change the behavior of the client to suit my needs. I can even modify the message headers when a message is sent.

So what will I do with all this new found power and control? Who knows? But it won’t be boring.

Whew! Ran a little long this time. I guess I’ve been storing up some blogging action points. As always, corrections, additions, and enhancements are always welcome in the comments.

Links

Advertisements