2010/09/19

Code is soft, interface is hard

I was expecting to push 0.9.1 out of the door sometime in June or July, which has long past. The development trunk has 0.9.1_pre1 version tag for months, and I'm using new features are actively without a problem. So what's pulling back the release?

There are couple of new API sets that I'm not sure are right. I like an API that are simple but powerful; easier to do typical things but allowing to do complicated things if you want.

Thanks to the flexibility of Scheme it is relatively easy to have simple calls that covers easy use cases, and later adding options that augment them to handle atypical situations.

However, that approach is susceptible to creeping featurism; ending up tons of options that modifies the behaviour of API in various ways, sometimes incompatibly each other. It forces a programmer to look up references constantly, and when a new atypical situation arises, it is likely that he can't deal with it with existing options and end up modifying the library code to support a new option.

So, it is often the case that difficult challenges are not introducing the first version of a new module that supports the most basic functionalities, but releasing a second version to support complicated use cases.

And while code is malleable, interface is not. Once you release a public interface and other people's writes code on top of that, it becomes very difficult to introduce incompatible changes. The difficulty is that, an interface design needs to be evaluated by the actual applications that uses it, but to have enough applications you have to make the interface widely available, hence making the change more difficult.

At this moment, the biggest issue I'm stuck at is in rfc.http. The original API covers simple use cases and works well for the job. But it has shown its shortcomings as I write more applications; especially HTTP becomes a substrate of more involved protocols.

What I want in the revised rfc.http module:

• Persistent connections: This hasn't been implemented, but the infrastructure is there, and I expect it can be introduced as a natural extension of the exiting API. The current API will be extended to take <http-connection> object in place of the 'server' argument.
• Secure connections: There are practical needs for this, so I hacked up an easy solution, though it may be extended later. Right now I call "stunnel" as external process and delegates SSL handling to it; at API level, a new keyword argument :secure switches the use of SSL. In future, if Gauche supports SSL as in-process extension, we can just switch to it. This option is orthogonal to other options, so it's acceptable.
• Parameterized http method: Current API has different functions for each http method, e.g. http-get, http-post etc. This is good if I just want to retrieve an html page or post an html form. However, when some other protocol uses http as a lower layer, it gets a bit awkward; it is more natural for the code to give http method as a parameter to a single http handling function. This motivates me to create a single point of entry (in the trunk, it is http-request), and redefine existing http-get etc. as convenience procedures on top of it.
• Flexible handling of message body: This is the biggest issue, and I haven't resolved it yet. I'll discuss it in a separate entry.

Tags: rfc.http, API