C API and promise
This ate up my whole afternoon so I write it down not to fall into it again.
I've got a really weird bug. We have a parameter (say P). P has a default value, but it may not be available at the initialization time. Basically, what we want is to delay evaluation of EXPR below until the value of P is actually taken:
(define P (make-parameter EXPR))
Simply wrapping EXPR with
delay did't cut it, for the user of P expected it to contain a value which wasn't a promise. We couldn't go to every place where P was used to wrap it with
So we added a special flag in the parameter, which applies
force on the value whenever the value is taken. The feature isn't available from the Scheme world, though. It's only through C API, for we're not sure
if such feature is a good idea yet.
Anyway, P got such a flag, so we could also say
(P (delay EXPR)) to alter the value of P, with the actual computation of EXPR is delayed. And it seemed working.
However, we ran into an issue when some code takes the value of P from C API. The internal of parameter object is a bit complicated, but you can assume there's an C API that retrieves the value of the given parameter.
Through C API, however, P's value looked like
#<closure ...>, whereas when I took P's value from the Scheme world, it returned the value of EXPR.
I started tracking it down and it was like
a rabbit hole. Scheme interface eventually calls the internal Scheme
%primitive-parameter-ref, which directly calls
Scm_PrimitiveParameterRef. I inserted a debug stub
to show the result of C call. The C API returns the mysterious
closure, yet in the Scheme world it returns the desired value.
Does Gauche runtime intercept the return value from C world to Scheme
world? Nope. It's directly returned to the Scheme world. I have
no idea where this
#<closure...> came from, neither
how the value changes to the desired one.
Furthermore, I found that if I evaluate
(P) second time,
C API returns the desired value. But no code is called to actually
replacing P's value!
I poke around C stub generators, VM code, parameter code,... in vain.
Finally, I opened up the source of
Scm_Force, the C API
force. And BANG! The answer was there.
C runtime doesn't like
call/cc. C procedures return either
exactly once, or never. So, when you call back Scheme code from C,
you have to choose one of these two strategies:
- Restrict the called Scheme code to returns at most once. If a continuation captured within the Scheme code is invoked again later, and tries to return to the C code again, an error is thrown.
- Split your C code to two, before the callback (A) and after the callback (B). Both A and B are ordinary C function. A arranges B to be called after the Scheme callback returns. Effectively, you write it as a continuation-passing style. With this, a continuation captured within the Scheme callback can be re-invoked, which just calls B again.
Most of Gauche runtime in C adopts the latter strategy, so that
call/cc works seamlessly. By convention, the C API functions
that use the strategy are named
Scm_VM***. The caller
of such C API can't expect to get the final result as the C return
value, since such function may need more calculation (Scheme code
and B part) to get the final result.
Scm_Force is that type of function, too. I only forgot to name
Scm_PrimitiveParameterRef casually called
when it has the delayed evaluation flag, expecting that it returns
the final value. But in fact,
Scm_Force can only be used
in conjunction of Scheme VM to obtain the final result.