I am pleased to announce the beta release of RelStorage 1.0! RelStorage is a storage implementation for ZODB that stores pickles in a relational database.
Features in this release:
- Full support for PostgreSQL 8.1+, Oracle 10g, and MySQL 5.0+.
- Supports whole-database migration to and from FileStorage, including all history.
- A number of optimizations have been implemented, making RelStorage performance comparable to that of FileStorage.
- There is now a poll-interval option, which reduces the frequency of database polls and helps the database scale better for read-intensive loads.
- There is now a pack-gc option, which makes it possible to retain at least one revision of every object during packing.
Please help test right away so that we can make a solid 1.0 release.
Get it here:
For more information, see the wiki:
I was interested in adding MySQL 5.0+ support to RelStorage, so I went ahead and did it. The code is in Subversion now.
The tests I’ve run suggest the MySQL adapter is already a lot faster than ZEO and both of the other adapters. The MySQL adapter has consistently finished the tests 40% faster than the PostgreSQL adapter. I haven’t tested PostgreSQL 8.3 yet; perhaps that will make a difference. I don’t know yet how the reliability of each adapter compares.
The port took all day. (My wife and I both have a cold, making it unwise to go to work today.) I didn’t think it would take that long, but I had to slow down to figure out the strange relationship between locks and transactions in MySQL. The LOCK TABLE statement is full of surprises! I ended up using GET_LOCK and RELEASE_LOCK and row-level locks instead.
So after editing queries to fit MySQL’s rules and fixing miscellaneous details, RelStorage now has a third database adapter. I’m excited to see what happens next. For one thing, MySQL support could have a major positive effect on Plone.
By the way, MySQLdb version 1.2.2 is required. Version 1.2.1 has a bug involving BLOBs; RelStorage depends heavily on BLOBs.
I just upgraded arcutah.org to Plone 3.0 and put a new skin on it. Check it out if you have a minute. Pay special attention to the Bridge to the Future training materials; the Arc of Utah has published a great deal of information for caregivers of mentally disabled people.
I’m writing RelStorage, a new storage implementation for ZODB / Zope / Plone. RelStorage replaces PGStorage. I’ve put up a RelStorage Wiki page and the zodb-dev mailing list has been discussing it. There is no stable release yet, but a stable release is planned for this month.
While performance is not the main goal (reliability and scalability are more important), I was pleasantly surprised to discover last week that creating a Plone 3 site in RelStorage on PostgreSQL 8.1 is a bit faster than doing the same thing in FileStorage, the default ZODB storage. Clearly, the PostgreSQL team is doing a great job!
Several years ago I put together an early prototype of PGStorage. I recall discovering that PostgreSQL was terribly slow at storing a lot of BLOBs. I read about the soon-to-come TOAST feature, but I wasn’t sure it would solve the problem, so I discarded the whole idea for years. Today, PostgreSQL seems to have no problem at all with this kind of work. It sure has come a long way.
RelStorage also connects to Oracle 10g. According to benchmarks, Oracle has a slight performance advantage, perhaps due to the “read only” isolation mode that Oracle provides. It might be useful for PostgreSQL to get that feature too.
I’m considering setting up a MySQL adapter for RelStorage as well. When the database is in MySQL and Zope is running in mod_wsgi, we could say that the “P” in LAMP stands for Plone!
Check out this musical talent. My little niece is so good that she can predict the accompaniment my brother is going to play, then she fits a melody with it!
I was going to move the posts from my old weblog, but I decided to just leave the old site up instead. I can change my mind about that later if there’s any reason to.