How to get a fast response from me

I’m swamped with work.  I can’t yet talk much about the work I’ve been doing until we go public, but it’s a cool project, although its coolness may not be immediately obvious at first. 😉

Meanwhile, I have been rather slow to answer some emails.  Here is what you need to do to get a fast email response from me: present me a viable option that allows someone other than me to do the work.  For example, if you want one of my open source projects changed, the best thing to suggest is that you will do the work and all I have to do is review a patch.  (Ideally, include the unified diff in the email so I can review instantly.)

RepStrap Notes for my Past Self

Here are some bits of wisdom I have gained about building and using a Bits From Bytes RepStrap 2.0, in stream-of-thought order.  I wish I had known this stuff before!  I imagine much of this applies to newer machines as well.

  • Essential tools that you may not realize are essential: a small benchtop drill press (AKA pillar drill) if you don’t have a lathe, a good soldering iron (a Hakko 936 is great), solder wick (size #1), a heat gun (Black and Decker makes a good one) and heat shrink, a range of drill bits that differ in size by 1/64″, a bench vice, a good hack saw, a chisel, a metal file, a set of small diamond files, a multimeter with a temperature probe, and a dremel-style tool.  BTW, a drill press is much quieter than a hand-held drill.
  • As a beginner, buy only PLA.  ABS and HDPE are more difficult because they shrink too much while cooling, pulling the lower layers upward, warping objects with dimensions of about 5 cm or larger.  ABS also smells bad while building.  Warping happens with PLA objects by about 15 cm, but most objects in the community aren’t that big.
  • Warping can break the extruder mount.  Don’t accept warping.  If you break the extruder mount, fix it with super glue and tie the extruder to its mount in more than one place.
  • Loctite Super Glue for All Plastics, #01-07011, works well on acrylic.  Apply it generously and always allow the full 8 hour set time.  If the lid gets filled with glue, drill it out.
  • Loctite Vinyl and Plastic, #01-06998-01, works well for flexible plastic and rubber, such as the Z axis belt.
  • Kapton tape is helpful, although I don’t trust it enough to apply it directly to the nichrome wire on the heater.  I’ve heard that Kapton tape can catch fire.  I use Kapton tape to insulate the copper leads connecting to the nichrome.
  • When building the heater, let the nichrome dangle outside the barrel a bit to avoid breaking the nichrome wire.  The copper/nichrome connection does not need to be inside the heater.
  • Twisting copper and nichrome together, then applying solder, makes a strong bond.
  • Build a 4 ohm heater, not an 8 ohm heater.  A 4 ohm heater can reach the specified temperature more quickly and maintain it better.  The standard transistor can handle 5 amps; a 4 ohm heater at 12 volts consumes only 3 amps.
  • If you short the PWM transistors, they will stay on forever, which could cause a fire.  Use Nophead’s suggested replacement.
  • Running the object cooling fan continuously (with a PWM setting of 64) seems to prevent jams and burned filament.  To compensate for the fan, add extra cement as insulation and raise the temperature by about 10 degrees.  I used to add fiber glass insulation to the heater, but that made it harder to fix the heater.
  • Ace Hardware sells an 8oz tub of Chimney Sweep Furnace Cement.  I’ve been happy with it.  It cures in minutes with a heat gun. 🙂
  • The copper pipe on the heater seems unnecessary.  I have left it off and I’m happy.
  • Don’t bother with the 30mm washer holding the heater in place.  Cut a piece of thin aluminum instead.
  • The 5V motor can run at 12V, but it’s loud and has a short life (about 2 weeks when I used the machine a lot).  If you use a Solarbiotics DC motor, it *must* be the 12V type.
  • The 12V DC motor, running at full speed, extrudes about 1.6 mm3/s, translating to about 4 or 5 cm3/h, including pauses.  The 5V motor runs faster and can extrude 3 mm3/s.  Too bad it doesn’t work for long.  Trying to extrude more than 3 mm3/s leads to frequent jamming.
  • The extruder motor has more than enough power.  The much harder problem to solve is gripping the filament; the motor tends to turn the same speed whether grip is achieved or not.  My latest solution for gripping is to sharpen the stainless steel (SS) M8 screw with a diamond file (the drill press is handy for this process) and move the SS washer upward by filing a small slot, so that the washer doesn’t dull the important part of the screw.  This has worked well so far.
  • Make the extruder detachable.  A 9 pin connector is good.  You will remove and improve the extruder dozens (hundreds?) of times before you’re satisfied that it is reliable.
  • Make the heater detachable too.
  • To make a thermocouple, get type K thermocouple wire (with glass insulation) and twist the wires together at the point where you want to measure the temperature.  I don’t think welding the thermocouple is necessary at all.
  • The alarm LED on the thermocouple PCB is very useful.  If it flickers, you probably have broken thermocouple wire.
  • Build on blue painter’s tape (the non-stick side).  I stuck the tape to the acrylic base that came with the BitsFromBytes 2.0 kit.
  • The primary source of wobble is the 608ZZ bearings.  Don’t rely on their accuracy.  I had to change both the Y and Z axes from the standard design so that the bearings could wobble without affecting the machine.  (The X axis might need a similar adjustment.)
  • Some of the acrylic takes too much load and breaks.  I replaced the acrylic piece that connects to the X axis stepper with hacked 1/4″ aluminum.  The fix has been quite reliable.
  • Run the machine from a server so that you can get other work done on your laptop/desktop at the same time.
  • Art of Illusion is very buggy, Blender is just as buggy when performing boolean operations, and OpenSCAD is great once you’ve gone through the slightly difficult process of compiling it.  Use OpenSCAD for anything that requires exactness.  Use Blender for artwork.
  • Both the RepRap host software and Skeinforge are hard to configure correctly.  Many of the defaults are wrong.  Tweaking them takes a lot of time.  Skeinforge is much more flexible.  Get Skeinforge from Subversion and plan to edit the code to fix bugs such as division by zero.
  • Use a G-code firmware, but plan to tweak the firmware.  I hope you can program in both C and Python. 🙂
  • The Oozebane plugin is no good.  In my experience, a much better solution is to add code to the firmware that pauses when starting a segment and temporarily reverses the motor (while the steppers are moving!) after stopping a segment.  The length of the pause depends on how much time has passed since the extruder was last turned off.  A Skeinforge plugin could emulate this behavior, but it’s simpler to do it in the firmware.
  • A new Sanguino board requires you to upload a bootloader, which requires either a programmer or a breadboard and a parallel port.  Most new computers don’t have a parallel port.  I got by with Xubuntu loaded on an old Pentium II that had a parallel port.
  • Both OpenSCAD and Skeinforge often need lots of CPU (2-3 GHz) and RAM (1 GB or more).  OpenSCAD needs a modern 3D video card.  A $30 fanless nVidia card from NewEgg is sufficient.
  • I wrote my own Python scripts for running the machine.  The scripts prepare the machine before starting a build, and can pause and resume a large build.  I’m not sure how others are handling that part.  Maybe I should post my scripts on some server.

Having learned all that and more, I think I am finally ready to build a Mendel. 🙂

RelStorage 1.4.0b1 and zodbshootout

I just released RelStorage 1.4.0b1.  New features:

  • More documentation.
  • Support for history-free storage on PostgreSQL, MySQL, and Oracle.  This reduces the need to pack and makes RelStorage more appropriate for session storage.
  • Speed.  New tests prompted several optimizations that reduced the effect of network latency in both read and write operations.  Memcached support is now integrated in a much better way.
  • Support for asynchronous database replication.  Previous versions of RelStorage worked with MySQL replication, but did not keep ZODB caches in sync when failing over to a slave that was slightly out of date.
  • The Oracle adapter now uses PL/SQL for speed and lock timeouts.  Lock timeouts are important for preventing cluster lockup.
  • Moved the speed test script into a separate package named zodbshootout, making it easier for developers and administrators to run comparative performance tests.
  • The adapter code is more modular, making it easier to support new kinds of databases and database adapter modules.

The zodbshootout script tells me this release of RelStorage is faster than ever.  It reports objects read or written per second, so unlike the previous charts I’ve made, bigger is now better.  Here are the results:

relstorage-speed

PostgreSQL now beats MySQL in some of the tests.  Oracle (not on this chart) is now looking pretty good too.

The new features led to far more automated tests.  My private Buildbot, which tests RelStorage with several combinations of Python, ZODB, and operating systems (in virtual private servers), now takes 2 hours to run all the tests.  Maybe I need to upgrade that server or investigate the possibility of making Buildbot launch an Amazon EC2 instance.

The previous release was 1.3.0b1, which added ZODB blob support.  Several customers asked for new features right after I released 1.3.0b1, so I decided to jump to version 1.4.0b1 rather than finalize the 1.3 series.  The 1.2 series has had more extensive testing, so use that for a while if you have troubles with 1.4.0b1.

This new release should be particularly interesting for Plone users, since Plone is always hungry for faster infrastructure.

PyCon 2010: I Want to Present Something

I’ve been racking my brains to find something to present at PyCon 2010. I have been trying to find something good to present since PyCon 2003, when I last presented at PyCon.  (I talked about Ape, the Adaptable Persistence Engine for Zope.)  I really liked the experience of presenting and it led to a lot of interesting conversations. Since then, however, nothing has really struck me as a good idea for a presentation. The right idea has to fit at least these criteria:

  • It’s something I’m good at.
  • At least a handful of PyCon attendees would like to learn more about it in a presentation.
  • I can’t spend weeks to prepare for the presentation.

This year, I was planning to do a really fun presentation by writing some Python scripts for controlling a RepRap, then I was going to present the hardware and software.  That didn’t work out, however, because the warping issues with ABS (a type of plastic) are just too severe to print useful parts, so my RepRap has sat idle.  (I’m now considering PLA, but it’s a newish and expensive material.)

Here are a few other ideas for a presentation topic.  What do you think?  Any other ideas?

  • RelStorage.  I could talk about future plans, why I think it is an improvement over ZEO, and why you should use it.  There seem to be a lot of people with questions about RelStorage, so it would be nice to have some time to answer the questions in a big room so others can hear the answers.
  • KARL and BFG.  This would probably be a team presentation.  KARL is some fairly interesting software that I got to help develop this year.  I could talk about the software, along with the development experience and style.  In KARL we made a conscious choice to ignore certain apparent DRY violations, leading to significant productivity gains.
  • A friendly introduction to Buildout.  Buildout is a tool that a lot of developers need, but don’t know it yet.  The function that Buildout performs is as important as version control and automated testing.  Come find out why Buildout is far better than a pile of Makefiles.
  • An introduction to Buildout (zc.buildout) for people familiar with Apache Maven.  Buildout and Maven fill approximately the same niche, but for different audiences.  (Buildout for Python, Maven for Java.)  Maybe there are Mavenites at the conference who would like to switch to a more Python centric system.
  • A discussion of text indexing in Plone and BFG.  This might be a narrow topic, but I find it interesting and important.  I have found ways to reduce complex 90 second text searches to 1 second.  The solution is not pure Python, unfortunately. 😉  I have also thought about how to expand into areas like faceted search/browse functionality.

Feedback encouraged!

RelStorage 1.3.0b1, Now With Blob Support

I have just released two versions of RelStorage. Version 1.3.0b1 adds full support for ZODB blobs stored on the filesystem. Version 1.2.0 is currently the better choice if you’re upgrading a production system and don’t need blob support.

People have been asking for blob support for months. I am glad to finally get it done, with a little help from a customer. With blob support, now we can easily store large artifacts on the filesystem, while keeping all metadata in the database.

To celebrate the new release, I have created a sample buildout.cfg that builds Plone with RelStorage, PostgreSQL, and blob support. (Thanks goes to Hanno Schlichting, who released a compatible version of plone.recipe.zope2instance only moments after I requested it.) Here it is:

[buildout]
parts = plone zope2 instance zopepy
find-links =
    http://dist.plone.org
    http://download.zope.org/ppix/
    http://download.zope.org/distribution/
    http://effbot.org/downloads
    http://packages.willowrise.org
eggs =
    elementtree
    PILwoTk
    RelStorage
    psycopg2
versions = versions

[versions]
ZODB3 = 3.8.3-polling
RelStorage = 1.3.0b1

[plone]
recipe = plone.recipe.plone

[zope2]
recipe = plone.recipe.zope2install
url = ${plone:zope2-url}

[instance]
recipe = plone.recipe.zope2instance
zope2-location = ${zope2:location}
user = admin:admin
products = ${plone:products}
eggs =
    ${buildout:eggs}
    ${plone:eggs}
    plone.app.blob
zcml = plone.app.blob
rel-storage =
    type postgresql
    dsn dbname='plone' user='plone' host='localhost' password='plone'
    blob-dir var/blobs

[zopepy]
recipe = zc.recipe.egg
eggs = ${instance:eggs}
interpreter = zopepy
extra-paths = ${instance:zope2-location}/lib/python
scripts = zopepy zodbconvert

P.S. I have been told that a very prominent Plone developer recently configured RelStorage with master/slave replication on MySQL, and that it works smoothly. I expect him to announce his success soon!

Maximize the Failure Rate

This is a reminder to myself.  It’s about failure, which is a paradoxically fantastic thing if you have the right attitude.  Let me describe the attitude I am working to gain.

  • If you fail three times in a week, make a goal of trying (and possibly failing) five times the next week.  Failure is the right thing as long as you’re trying sincerely and the rate of failures is increasing, or you’re failing less each time.
  • You won’t succeed at really complex things until you’ve failed a lot first, so your best bet is to fail as fast as possible.  Enjoy the journey!
  • Mere repetition isn’t sufficient.  After each failure, try something significantly different.
  • Report and graph the failures.  Talk about them.  Things that get reported tend to improve.
  • Goals need to be simple, but some goals are inherently complex.  You can simplify a goal by maximizing the rate of failing to achieve the goal.
  • If rapid iterations of failure are not possible due to circumstances outside your control, fail in parallel!  Try many things at once.  Don’t forget to report on all of the failures.
  • When you finally do succeed, continue to try other things.  In other words, keep failing!

This is particularly applicable to small business.  Discouragement resulting from failures can spread like a disease.  However, the “maximize the failure rate” attitude can make you immune to that kind of discouragement.

Wrong attitude: “We failed.  Maybe we should give up.”

Right attitude: “We failed again?  Ye-haw, that means success is closer!  Now how can we fail faster (or less expensively) next time?”

Book Review: Practical Plone 3

Packt Publishing asked me to review their new book, Practical Plone 3: A Beginner’s Guide to Building Powerful Websites. The book impressed me, but not in the way I expected at first.

As I read the instructions in chapter two about how to install Plone, I considered the experience my Dad would have gone through if he had this book when we were setting up Plone to run his company’s web site. My Dad is a power user, but not a programmer or systems administrator, so with this book, he probably would have installed Plone himself on a spare Windows computer. This book would have provided him enough direction to set up a lot of the functionality he needed, without my help. He would have immediately started publishing pages with Plone’s many features.

However, I imagine that a short time later, something would go seriously wrong. The computer’s IP address would change because the DHCP lease expired, the database would lose some transactions due to some misbehaving application, or a mischevious virus would rename files with a “py” extension to “rb”. All of those problems are outside Plone’s control, so this book does not try to address them.

Plone beginners like my Dad are not prepared to handle the problems that occur when a computer is used as a web server. In that light, I wondered if it really is possible to run Plone (or any content management system) without deep technical experience. I thought for a moment that this book is not for beginners after all.

Even after that logic, I decided I still want very much to give my Dad a copy of this book next time we set up a Plone web site. He will read it to find out what the latest version of Plone can do. He will install it on his own computer for his own education, but I will set up the production web site on a server.

The first twelve chapters (250 pages) are intended for Plone users. Beginners will enjoy all of those chapters, I think. As I read them, I even picked up a few things I haven’t learned, like how to use content rules.

I think beginners might struggle the most with chapter nine, which explains how to control workflow. Controlling workflow in Plone is not as easy as other Plone functions, because Plone falls back to the less polished Zope Management Interface for workflow design. Matt Bowen handled the difficult topic gracefully.

The rest of the book (almost 300 pages) is for developers, not power users. The contrast is sharp. While the first half of the book tells the reader what buttons to push, the second half tells the reader how to modify their Buildout configuration and what to type in a terminal session.

Each chapter is written by a different author. I noticed two interesting effects of multiple authorship. First, each author is enthusiastic about the particular topic, so even LDAP (which I generally find quite boring) gets a chapter of quality treatment. Second, there is more redundancy than you would find in most technical books, but redundancy is probably good in this case.

I do have one quibble with the book’s organization. When the technology behind Plone was invented, CSS was still a baby and browsers did not support it well. Back then, changing a site’s appearance meant changing nested tables in HTML, so the developers of Zope (including myself) invented ways to manage that task. That is how the portal_skins tool came about. The theming chapters explain how to use the latest version of that technology.

Today, we can expect all of our customers to use browsers that support CSS, so the chapters on theming should start by explaining how to customize the web site’s CSS. Developers will make much faster progress that way than if they have to learn the many theming-related abstractions Plone has today.

In conclusion, Practical Plone 3 is more than just a beginner’s book. I plan to use this book as a communication tool with my Plone customers. The book is a menu telling my customers who are beginners to Plone what we can set up together without a lot of work. I will also use it to help developers come up to speed on Plone.

RelStorage 1.2.0b2 Released

This release works with unpatched versions of ZODB 3.9!  A big thank-you to Jim Fulton for including support for RelStorage in ZODB.  This release also continues to support patched versions of ZODB 3.7 and 3.8.

I have been doing a lot of testing, and I have found MySQL 5.1.34 to be a lot more stable than earlier releases of MySQL 5.1, so I am now declaring MySQL 5.1.34 and above supportable, meaning that if you ask questions about it, I am no longer going to request that you revert to MySQL 5.0. 🙂

Finally, I recently expanded my private RelStorage Buildbot to include a Windows XP slave.  After solving a couple of minor test glitches, the test results are now all consistently green on 4 platforms:

  • Debian Etch, 32 bit (Python 2.4.4, MySQL 5.0.32, PostgreSQL 8.1.17, Oracle 10g XE)
  • Debian Lenny, 32 bit (Python 2.5.2, MySQL 5.0.51a, PostgreSQL 8.3.7, Oracle 10g XE)
  • Debian Lenny, 64 bit (same as above but no Oracle)
  • Windows XP, 32 bit (Python 2.6.2, MySQL 5.1.34, PostgreSQL 8.3.7)

I’m thinking about adding another Linux slave that runs MySQL 5.1 and Python 2.6.

Anyway, enjoy the release!

P.S. You may be wondering why I released 1.2.0b2 instead of 1.2.0b1.  A little slip ruined the web page on PyPI, so I fixed the slip and skipped to the next version number.

Save the Art!

My sister, Maria, is an excellent artist.  Her paintings make you sit up and think about how people feel.  Her art captures life better than a photograph.

But what does she do when a piece doesn’t fit her high quality standards?  Not long ago, she finished a fine painting of a cowboy.  Everyone liked it, but Maria said the eyes were too big.  Ok, maybe the eyes were a bit larger than normal, but does that justify what she did next?

A little bit of the history of an important artist is gone for good.