How I Manage Mailing List Subscriptions

I participate in a lot of mailing lists, including Python and Zope related lists, local interest groups, various open source projects, and even the high traffic Linux Kernel Mailing List.  Mailing list participation is important for working with people.  I’m subscribed to about 50 lists right now.  So many subscriptions would be practically impossible to manage with an ordinary email setup, because they would dilute my inbox.  I have tried many solutions and have finally settled on a method that makes me happy.

Here are some ways I have tried in the past to manage mailing list subscriptions, all of them eventually failing in different ways.

• The filter solution. I set up a complex system of mail filters in my desktop mail client.  The mail client would try to match each incoming email with a mailing list, putting the email in a folder for that mailing list.  This had two big problems.  First, mailing lists sometimes changed their configuration behind the scenes, breaking my filter, causing all of the email for that mailing list to spill into my inbox until I changed my filter.  Second, if I accessed my email from a different computer that didn’t have the filters installed, the filters would not apply and all mailing list traffic would spill into my inbox.  Spills happened much too often.
• The newsgroup solution. People used to participate in newsgroups rather than mailing lists, and my mail client has great support for newsgroups, so I wrote some software that bridged email to NNTP newsgroups.  This was workable except it had two big problems.  First, I could no longer delete spam from mailing lists, since NNTP has no provision for doing so.  Deleting spam is important for subscribing to lists that aren’t managed very well.  Second, my NNTP software had to mangle email headers so that mailing list participants would not get confused about where the email had been sent.  I’ll note that I tried gmane and found that it has exactly the same problems with bridging email to NNTP.
• The multiple account solution. I created an email account just for subscribing to mailing lists, then I set up mail filters again.  This had most of the old problems, except that spills went to the mailing list inbox instead of my main inbox.  Then I started creating email accounts for each mailing list, but the number of client/server connections that required was problematic.  My mail client had to open a separate connection to the server for each account, leading to slow downloading.

I spent years (literally) with each of these solutions.  In every case, I was able to work around the problems, but over time, the problems built up to the point that I had to try something new.

The latest solution uses email address extensions, also known as address aliases.  I am using Postfix, but address extensions are possible with Gmail and probably others.  I set my server to use the character “.” for extensions, so that my solution can work seamlessly with services that think “+” is not a valid character in an email address.

On the server running Postfix, I set up Procmail with a simple .procmailrc file:

ARG = $1

:0
* ARG ?? list
$HOME/Maildir/.$ARG/

This file is cryptic, but it does a lot in only 4 lines.  I learned how to do this from freebsddiary.org and pm-doc.sf.net.  It tells my mail server to put every email that uses an address extension starting with the word list in a named email folder.  For example, an email sent to shane.list.gardening@example.com would be placed automatically in the list/gardening folder of my inbox.

This filtering happens on the server, rather than my mail client program, so I can read my mailing lists from any computer where I have set up a mail client, including a Squirrelmail server.  I always connect to my mail server using IMAP, so the information about which emails I have read is stored on the server.

I subscribe to every mailing list twice.  First I subscribe using an address extension like I just described, then I subscribe using my normal email address, but with delivery disabled.  It’s easy to do that on mailing lists run by Mailman.  I subscribe my normal email address because many mailing lists disallow or delay postings by people not subscribed to the list; that measure reduces spam for the mailing list.  The second subscription ensures the mailing list recognizes me as a member.

This system has none of the problems of the earlier solutions.  Mailing list emails have never spilled into my inbox.  It no longer matters if mailing list administrators reconfigure or rename their list.  I can delete spams.  I don’t have to alter any email headers at all.  The system causes no extra burden on my server or my mail client; in fact, putting mail in different directories probably reduces the burden for both.  I can move emails that were sent to the wrong mailing list.  If the mailing list provides a downloadable archive of old posts (many do), I can even import that archive using perfect_mbox and start browsing as if I had been subscribed since the beginning.

This solution is worth a look for anyone who participates on many mailing lists, particularly software developers.  Perhaps webmail providers will eventually provide features to make this solution accessible to more people; it could be a competitive advantage.