2012/03/18

Consumers for generators

I've been trying out generators for day-to-day scripts and have repeatedly seen a common pattern--- to repeat reading from a generator and process values until it is exhausted. I wrote a bunch of utilities in gauche.generator but they are mostly a sort of filters, taking generators and yielding another generator. What I needed is a kind of "terminator" of the chain of generators, where the generated values were actually consumed.

Since a generator is just a thunk, I can use already existing tools---actually, port-fold, port-map and port-for-each expect thunks (in spite of their names), and we can pass a generator directly.

(port-for-each (^v ...) generator-expr)

Probably I should rename them to indicate they are actually about generators, not ports.

(Another option is to convert a generator to a lazy sequence by generator->lseq, so that we can use all list-handling utilities without wasting space for intermediate lists. But it still has some overhead (not only CPU-wise, but mentally for programmers who read and write the code). So I think it makes sense to have specialized utilities for generators.)

For lists, we have for-each. But sometimes it is handy and more readable to use dolist macro, taken from CL. We can have its equivalent for generators:

(do-generator [v generator-expr]
  ... do something with v ...)

What if we want to run across multiple generators, or even to mix generators and other sequences? Instead of inventing more macros, we can extend the existing srfi-42 mechanism to handle generators.

(do-ec (:parallel (: x '(a b c))
                  (: y (gdrop-while skip-preamble
                                    (file->generator "foo"))))
  (do-something x y))

For the latter, we added :generator EC-qualifier (so the second clause of :parallel in the above example can also be written as (:generator y ...)). The generic dispatcher of : dispatches to the :generator if its argument is an applicable entity taking zero arguments.

Those changes have been pushed to the master.

Tags: Generator, gauche.generator, srfi-42