25 Jan 2001 emk   » (Master)

XML-RPC Problems: Spoke with Adrian about RedHat's experience with XML-RPC. It seems that they've run into a very big problem: round-trip HTTP message latency. They tried to make lots of little XML-RPC calls, and their performance died miserably.

I did some research on HTTP pipelining (which xmlrpc-c already supports), but this doesn't fix the problem. It seems that HTTP always requires a minimum of one round-trip packet per request. So if you've got a 250ms ping time, you can't make more than two XML-RPC method calls per second.

This is totally unacceptable. Oh, sure, you can work around it by reducing your number of function calls to a minimum (which is what the RedHat Network does), but your APIs will still be bletcherous.

A Proposed Fix: Let's say you're invoking a simple-minded addition function:

>>> import xmlrpclib
>>> multi = xmlrpclib.Server("http://localhost/cgi-bin/multi-cgi.cgi")
>>> multi.sample.add(1, 1)

Now, if you need to perform a zillion additions, this is really going to suck. But since XML-RPC is so dynamic, there's no reason why you can't just do something like this:

>>> add_2_2 = {'methodName': 'sample.add', 'params': [2, 2]}
>>> add_4_4 = {'methodName': 'sample.add', 'params': [4, 4]}
>>> multi.system.multicall([add_2_2, add_4_4])
[[4], [8]]

On the back end, my new server code unpacks this transparently, calls all the appropriate handlers, and packages up the result.

If you're a serious RPC hacker, I'd really appreciate some feedback.

On related note, Ryan Dietrich is hacking on an asynchronous XML-RPC message queue. Way cool!

Latest blog entries     Older blog entries

New Advogato Features

New HTML Parser: The long-awaited libxml2 based HTML parser code is live. It needs further work but already handles most markup better than the original parser.

Keep up with the latest Advogato features by reading the Advogato status blog.

If you're a C programmer with some spare time, take a look at the mod_virgule project page and help us with one of the tasks on the ToDo list!