How I Manage Mailing List Subscriptions

I participate in a lot of mailing lists, including Python and Zope related lists, local interest groups, various open source projects, and even the high traffic Linux Kernel Mailing List.  Mailing list participation is important for working with people.  I’m subscribed to about 50 lists right now.  So many subscriptions would be practically impossible to manage with an ordinary email setup, because they would dilute my inbox.  I have tried many solutions and have finally settled on a method that makes me happy.

Continue reading How I Manage Mailing List Subscriptions

Ubuntu 9.04 Versus Gentoo

Ubuntu 9.04 is now in beta and I’m tempted to upgrade. I want to help test it. I even started “update-manager-kde -d”, but it warned me that since the new version is not yet fully tested, it could make my computer unstable, release a new strain of computer virus, kill babies, and reinstate Bush Junior as president. So I canceled. Am I chicken?

That situation makes me awfully uncomfortable.  It happens with every binary Linux distribution I have tried.  It never occurs in Gentoo, however, where all upgrades are incremental. Breakage happens, but when it does, I only have to fix or revert a relatively small part, not the whole system. It appears that such incremental upgrades are only manageable with a source-based distribution, but of course a source-based distribution requires more technical skills.

I think I’ll stick with Ubuntu right now because its support for KDE 4 is better than what Gentoo currently offers.  From what I can tell, Gentoo got in a big fight over how to package KDE 4, causing them to fall behind in stabilizing KDE 4.  That fight seems to be over now, so perhaps Gentoo will catch up and lead again. I will seriously consider a switch when Gentoo marks the latest version of KDE 4 as stable.

How to Install Plone with RelStorage and MySQL

These step by step instructions describe how to install Plone on Ubuntu with RelStorage connected to MySQL as the main database. Familiarity with Linux systems administration is expected. Update: These instructions were revised in August 2009 for Plone 3.2.3 and RelStorage 1.2.0.

Continue reading How to Install Plone with RelStorage and MySQL

The Irony of Google AdSense

I set up AdSense on my blog to learn about serving ads and maybe earn some pennies. According to the terms of service, I am not allowed to click any of the ads. That rule certainly makes sense from Google’s business perspective, yet I find that of all the ads on the whole web, the ads Google puts on my pages are the most relevant to me. They are among the few that I want to click, but I can’t! Gah.

RelStorage for Sessions?

I’ve been working on a document explaining how to install Plone with RelStorage, starting from a basic Linux server.  As always, the basic procedure is simple, but there are all sorts of interesting little complications.  One detail that bugged me today is the need for a shared session database.

Session storage is a little different from normal storage because keeping a history of session state becomes expensive quickly.  For session storage, I think we still want all the goodness of ZODB transactions, conflict detection, distributed caching, and so on, but in this case, the ability to undo is pointless and the need to pack is a liability.

The database schema I’ve been using in RelStorage is a mismatch for this need.  A history-free storage should not have a “transaction” table, there should be no need for MD5 sums and prev_tid pointers, and the compound primary keys consisting of oid and tid should become simple primary keys indexed by oid.  A history-free storage still needs garbage collection, but not packing.

I’m thinking that the main RelStorage class will need few changes to support history-free storages, while the database adapter class will change so much that it would be best to just create a different adapter.  I like that.  It won’t be possible to switch history on and off without an export and import operation, but that seems reasonable.  To put a positive spin on the new adapters, I think I’ll call the new adapters “packless”, like the old BerkeleyDB storage.

I think that’s a good plan.  While the purpose of the packless adapters is initially session storage, they will certainly be also usable for other databases, including the main database.  I expect them to be slightly simpler and faster than the history-preserving adapters we have now.

I encourage anyone interested to leave a comment, even if all you wish to say is “I want that!” 🙂

Why Keep a Journal?

I need to keep a journal because as I change over time, my method of comprehending everything also changes, and some memories no longer fit my new system of comprehension.

If my memory were a computer database, then I would be adding rows to giant tables continuously, while changing the schema every time I find a better way of understanding things.  Over time, some of those rows would become so far out of sync with the schema that no query would find them.  Some of those practically forgotten rows would remain important, though.  Keeping a journal is like copying vital data to an external store with a more stable schema.  (The English language is more stable than my system of comprehension.)

I wonder if I ought to express that without a computer analogy.  My kids need to learn it, but they certainly aren’t versed in all this jargon.

Streaming General Conference from MythTV

Every 6 months, the Church of Jesus Christ of Latter-day Saints broadcasts on television a series of meetings called General Conference.  There are usually 6 meetings, each 2 hours long, each meeting consisting primarily of talks by about 6 people.  On General Conference weekends, members of the Church watch the conference instead of attending local meetings.

I like to use MythTV to record the public sessions from KSL, a local TV station.  That way, I get a pristine, high definition picture that I can review anytime I like.

Two of the meetings are on Saturday.  During the Saturday meetings, I prefer to multitask while listening to conference.  I want my wireless laptop to play the audio and a miniature version of the video.  I finally figured out how to do that.

I started with the directions on parker1.co.uk.  That page describes how to use VideoLAN to stream a video even as it’s being created.  Unfortunately, those directions seem to be out of date, as is the mythlink script the page links to.   So here is my version of the mythlink script (with the database password replaced with ‘fakepw’):

#!/bin/sh

# mythlink.sh - Symlinks mythtv files to more readable versions
# by Dale Gass

if [ ! -d /tv/links ]; then mkdir /tv/links; fi
rm -f /tv/links/*

echo "Done RM"
mysql -umythtv -pfakepw mythconverg -B --exec "select chanid,starttime,endtime,title,subtitle from recorded;" >/tmp/mythlink.$$
perl -w -e '
        my $mythpath= "/tv";
        my $altpath= "/tv/links";
        if (!-d $altpath) {
                mkdir $altpath or die "Failed to make directory: $altpath\n";
        }
        <>;
        while (<>) {
                chomp;
                my ($chanid,$start,$end,$title,$subtitle) = split /\t/;
                $start =~ s/[^0-9]//g;
                $end =~ s/[^0-9]//g;
                $subtitle = "" if(!defined $subtitle);
                my $ofn = "${chanid}_${start}.mpg";
                do { print "Skipping $mythpath/$ofn\n"; next } unless -e "$mythpath/$ofn";
                $start =~ /^(............)/;
                my $nfn = "$1_${title}";
                if ($subtitle) {$nfn="${nfn}__${subtitle}"}
                $nfn =~ s/ /_/g;
                $nfn =~ s/&/+/g;
                $nfn =~ s/[^+0-9a-zA-Z_-]+/_/g;
                $nfn = "${nfn}.mpg";
                print "Creating $nfn\n";
                unlink "$altpath/$nfn" if(-e "$altpath/$nfn");
                symlink "$mythpath/$ofn", "$altpath/$nfn" or die "Failed to create symlink $altpath/$nfn: $!";
        }
' /tmp/mythlink.$$
rm /tmp/mythlink.$$

This handy script generates links to make it easy to find recorded TV programs on the filesystem. After running the script, I started a VLC streaming server like this:

vlc 200904041400_LDS_General_Conference.mpg --sout \
'#transcode{vcodec=mp4v,vb=2048,scale=0.25,acodec=mp3,ab=128,channels=2}:std{access=http,mux=ts,dst=:8081}'

I learned what options are available from the VLC streaming documentation.  The command I used creates a 2 megabit per second streaming server using the MPEG-4 and MP3 codecs, with the video scaled from high definition (1920×1080) to 480×270. Then I launched the client on my laptop:

vlc http://tiger:8081

This worked very well. Both the video and audio were very clean and never stalled.  The MythTV server has an AMD 5200+, which is more than fast enough to simultaneously record the high definition stream to a hard drive, play it back with scaling, transcode down to 480×270, and send the transcoded stream to multiple clients.

Of course, this will all be much easier when MythWeb gains the ability to generate Flash videos.

NewEgg, You Amaze Me

Last week, an important desktop computer used by my family died. The connection between the power supply and motherboard fried somehow. I’ve never seen a connection fail like this before: all of the connections attached to red wires had burned the surrounding plastic within the connector. It was difficult to detach the connector, and even when I cleaned up the motherboard and used a new power supply, the motherboard refused to power on. Nothing else was damaged.

I suspect the motherboard was drawing too much current and the connector failed over time. Newer motherboards have separate 12V connectors which could solve the problem.

Anyway, I immediately found a better motherboard (PCCHIPS A15G (V1.0) AM2+ MCP61P) and a faster, lower power CPU (AMD|A64 X2 5200 2.7G AM2 65N R) on NewEgg for a total of $108 including shipping. The computer will be better than ever. NewEgg sure knows how to keep me as a customer.