Tag: PyPI

RelStorage 1.1.2

This release has two new useful features, one for performance, one for safety.

The performance feature is that if you use both the cache-servers and poll-interval options, RelStorage will use the cache to distribute basic change notifications.  That means we get to lighten the load on the database using the poll-interval, yet changes should still be seen instantly on all clients.  Yay! 🙂

The only drawback I expect is that caching makes debugging more difficult.  Still, this option should help people build enormous clusters, like the one my current customer was planning to build, although I got word today that they have changed their mind.

The new safety feature is the pack-dry-run option, which lets you run only the nondestructive pre_pack phase to get a list of everything that would be deleted by the pack phase.  This should be particularly useful if you’re trying out packing for the first time on a big database.  My current customer would have benefited from this too.

I also fixed a bug that caused the pack code to not remove as much old stuff as it should and I started using PyPI instead of the wiki as the main web page.  Using PyPI means I have to maintain only one README, which gets translated automatically into the PyPI page.  Until now I’ve had to maintain both the README and the wiki page.

http://pypi.python.org/pypi/RelStorage/1.1.2

Patched ZODB3 Eggs Available

This week, I put up some ZODB3 eggs and source distributions with the patch required for RelStorage already applied. I built both ZODB3-3.8.1-polling and ZODB3-3.7.3-polling.  I even made eggs for Windows developers who have not yet taken the time to set up MinGW. 😉

http://packages.willowrise.org/

Developers can use this web site in buildout.cfg to incorporate RelStorage in their applications.  Feel free to mirror the site if you need to.

Egg Patching Solution #3

Martijn Faasen suggested this solution in a comment on my previous post and I think it’s the best.  I created a new service:

http://packages.willowrise.org

I simply posted a patched ZODB3 source distribution on a virtual-hosted server.  The first tarball, “ZODB3-3.8.1-polling-serial.tar.gz”, includes both the invalidation polling patch and the framework I created for plugging in data serialization formats other than pickles, but in the near future I plan to also post distributions with just the polling patch and some eggs for Windows users.

It would not make sense for me to post the patched tarballs and eggs on PyPI because I don’t want people to pull these patched versions accidentally.  Pulling these needs to be an explicit step.

Thanks to setuptools and zc.buildout, it turns out that creating a Python code distribution server is a piece of cake.  The buildout process scans the HTML pages on distribution servers for <a> links.  Any link that points to a tarball or egg with a version number is considered a candidate.  A static web site can easily fulfill these requirements. I imagine it gets deeper than that, but for now, that’s all I need.

To use this tarball, buildout.cfg just needs to include lines something like:

[buildout]
find-links = http://packages.willowrise.org
versions = versions

[versions]
ZODB3 = 3.8.1-polling-serial

zc.buildout does the rest.

It took a while to find this solution because, upon encountering the need to distribute patched eggs, I guessed it would be difficult to set up and maintain my own package distribution server. I also guessed setuptools had no support for patches in its versioning scheme. I’m glad I was completely wrong.

By the way, Ian suggested pip as a solution, but I don’t yet see how it helps. I am interested. I hope to see more of pip on Ian’s great blog.

RelStorage 1.1.1 Released

For those just tuning in: RelStorage is an adapter for ZODB (the database behind Zope) that stores data in a relational database such as MySQL, PostgreSQL, or Oracle.  It has advantages over FileStorage and ZEO, the default storage methods.  Read more in the ZODB wiki.

Download this release from PyPI.  As I mentioned in another post, this release works around MySQL performance bugs to make it possible to pack large databases in a reasonable amount of time.

If you are upgrading from previous versions, a simple schema migration is required.  See the migration instructions.