2023/09/30

Pipeworks

Ports are very handy abstraction of data source and sink. In Gauche libraries, you can find many utitlies that reads from input port or writes to output port, and then another utilities (e.g. convert from/to string) are built on top of them.

While they are useful, it becomes tricky when you want to compose those utilities. Suppose you have a procedure f that writes to an output port, and a procedure g that read from an input port. You want to feed the output of f to g while make f and g run concurrently, so some threading is involved. You can write such a pipe using procedural ports but it is cumbersome to do so for every occasion. I want something that's as easy as Unix pipe.

So I initially started to writing a pipe utility using procedural ports. Then I realised I also want a device dual to it; while a pipe flows data from an output port to an input port, the co-pipe, or pump, pulls data from an input port and push it to an output port. An example is that you run a subprocess and feed its error output to your current output port. When you invoke a subprocess (ref:gauche.process), you can get its error output from an input port. So you need to read it actively and feed the data to your current output port.

Then you might want to peek the error output to find out a specific error message appears. So your contraption reads actively an input port, and feed the data to an output port, and you can read whatever data flows through it from another input port to monitor.

There are many variations, and mulling over it for some time, I wrote a library that abstracts any of such configurations. I call the device plumbing (draft:control.plumbing).

You can also create an output port that feeds the data to multiple outputs, or gather multiple input port into one input port. Refer to the manual to see what you can do.

Tags: 0.9.13, control.plumbing