The Holy Grail server framework

… and I think I just wrote it.

For years, I have wanted a Python TCP server framework with the following features.

  • Multi-process: I want the framework to be able to use all available processors and cores with no GIL (global interpreter lock) contention.  This is important for CPU-bound applications.
  • Multi-threaded: Each process should have a fixed number of threads so it can multiplex tasks.  This is important for I/O-bound applications.
  • Graceful shutdown: when I issue the kill signal, I want the server to close its server socket immediately, but I want existing connections to have a chance to finish their work.  This is important if the connections are involved in transactions.
  • Graceful code reloading and reconfiguration: when I issue the HUP (hangup) signal, I want the server reload its configuration, spin up new processes, and tell the old processes to stop accepting new connections and shut down only when finished.  Also, the server should only load application code in child processes, so that loading new code is simply a matter of issuing a HUP signal.
  • Transaction integration, including two-phase commit: If the connection dies, I want to roll back the associated transaction automatically, unless the connection dies after the first phase of two-phase commit, in which case I want to leave the transaction in a partially committed state and sort out the issue at a higher level.
  • No proxying: I don’t want all traffic to flow through the master process, which would be bad for both reliability and performance.  Linux (and maybe any Unix system) allows multiple processes to listen on the same server socket; I want to use that feature.

What I have described is well beyond what the Python standard library provides.  In theory, it will let me have all the joys of transactions and zero-downtime updates in a high performance environment.  I have been searching for an implementation of this design for years.  This week, I have been reworking some WingCash code to make better use of transactions and I realized I really need this design.

So, on Monday, I wrote my dream server framework and tried it out… and to my surprise, it actually worked!  I was surprised because I thought that some Python hacker, somewhere, must have tried this experiment by now, and must have failed; otherwise the experiment results would surely be published or mentioned somewhere.  Perhaps my requirements are not very common, the design is not obvious, or I did not type the right search terms.

I am now writing tests for the new server framework, which I am tentatively calling fdtserver (forking distributed transaction server).  I won’t put it into production until it has 100% statement coverage like the rest of WingCash.  I wonder whether fdtserver should be open source.  Doesn’t the need for all those features, particularly graceful code reloading, exist elsewhere?

4 thoughts on “The Holy Grail server framework”

  1. Quite nice, I had a similar situation recently when I was reworking our billing internals. We have a secure cluster that distributes various tasks, and the requirements given to me did not allow me to use Apache (and a nice RESTful interface). So we had to do something similar, in Perl of course :-p

  2. OK, now I’m really intrigued how you’ve managed to enable the GIL-less multi-processor bits! Beyond that, it seems like most of the remaining feature set is included in the combination of Eventlet + Spawning:

    Eventlet: “programming that is similar to threading, but provide the benefits of non-blocking I/O”

    Spawning: “wsgi server which supports multiple processes, multiple threads, green threads, non-blocking HTTP io, and automatic graceful upgrading of code”

  3. My little server framework uses blocking I/O, actually. For my specific application, blocking I/O is simpler and faster. I can see how non-blocking I/O is better for other situations, of course.

  4. Just curious, what happened to this server?

    I have the exact same requirements, and would love to play with your implementation.

    Incidentally, I came to your blog using other search terms 🙂

Comments are closed.