This release has two new useful features, one for performance, one for safety.
The performance feature is that if you use both the cache-servers and poll-interval options, RelStorage will use the cache to distribute basic change notifications. That means we get to lighten the load on the database using the poll-interval, yet changes should still be seen instantly on all clients. Yay! 🙂
The only drawback I expect is that caching makes debugging more difficult. Still, this option should help people build enormous clusters, like the one my current customer was planning to build, although I got word today that they have changed their mind.
The new safety feature is the pack-dry-run option, which lets you run only the nondestructive pre_pack phase to get a list of everything that would be deleted by the pack phase. This should be particularly useful if you’re trying out packing for the first time on a big database. My current customer would have benefited from this too.
I also fixed a bug that caused the pack code to not remove as much old stuff as it should and I started using PyPI instead of the wiki as the main web page. Using PyPI means I have to maintain only one README, which gets translated automatically into the PyPI page. Until now I’ve had to maintain both the README and the wiki page.
All, I am in a bit of a pickle here, pun intended. We have a fairly extensive setup of Zope/Plone running relstorage to connect to MySQL on the backend. Took us some time to get things imported because zodbconvert wouldn’t work. Now that the system is up the database is around 15GB in size. Reading from it is wonderful, blows away the performance we had on ZODB. However, writing to the database seems to take a loooooong time….and heaven forbid we import anything of substantial size. Any ideas on what we are doing wrong?
We are running Python2.4.4, Zope2.10.5, Plone3.0.6, MySQL python 1.2.2, ZODB 3.7.1, relstorage 1.1.2, and MySQL 5.0.x
To be honest we had some substantial problems getting all the pieces to play nice so if there happens to be a HOWTO on installing all these pieces together I’d really be interested in seeing it.
I appreciate any input anyone has to offer. If you would rather respond to me in e-mail you can do so at: paul.fiero ciaustintxus
PFiero
The performance measurements I’ve done suggest that large writes to MySQL, in the default configuration, are indeed quite slow. I imagine the MySQL buffer sizes need to be tuned. I can try a few simple ideas. I am hoping for a customer who needs this. 🙂
BTW, what went wrong with ZODBConvert? I am not aware of any bugs.