2013/09/18

Macro system extension

I finally added syntax-rules extensions (srfi:46) to Gauche, that makes Gauche's hygienic macro system compatible to R7RS (except a few known bugs).

The current hygienic macro expander is written in C which is an ugly pile of spaghetti. Originally I planned to ditch the legacy code and to write an explicit-renaming macro expander as the new basis of our hygineic macro system, then to implement syntax-rules on top of it.

I like ER-macro since it's transparent to what it is doing for hygienity. It doesn't necessary to be the easiest one to use--- destructuring the input form, then renaming identifiers explicitly would be cumbersome for day-to-day programming. But those things can be easily alleviated by combining other tools. For example, we can just use util.match matcher to destructure the input form (instead of yet another pattern matcher tied to macro system).

In fact, in er-macro branch in the repo I implemented ER-macro expander to some extent. But it turned out I need some more time to substitute the low-level macro layer completely. A major issue is to keep compatibility between ER-macro, which allows raw symbols inserted by the macro expander to capture symbols in macro calls, and the current syntax-rules implementation which turns all symbols into identifiers. (The same issue is described by mjt here, in Japanese.)

Since I'd like to push out R7RS compatible release sooner, I just went into the legacy code and added some more spaghetti to make it work as srfi:46.

＊＊＊

I realized this enhancement makes syntax-rules a lot more useful. I also adapted define-values form to R7RS, which allows generic formals, as follows:

(define-values (x y . z) (values 1 2 3 4))

z => (3 4)

With R7RS syntax-rules it's not difficult to distinguish proper list and inproper list (see Gauche:lib/gauche/defvalues.scm).

Tags: macro, r7rs, srfi-46

Past comment(s)

Peter Bex (2013/09/18 21:40:37):

Hey Shiro,
If you're concerned about the usability of ER macros, check out IR macros (http://wiki.call-cc.org/man/4/Macros#ir-macro-transformer). They're basically the same, but inverted: stuff that's injected needs to be explicitly injected, all "bare" symbols are automatically escaped. They can be implemented reasonably easily in an ER-based system.
If you would like to bounce some ideas about the implementation off of someone, feel free to contact me. I think this is one of the more fascinating topics in Scheme, and would like to gain some more perspective.

shiro (2013/09/19 03:30:46):

Hi Peter. Yeah, I've been looking into various low-level macros. My goal is to implement one low-level macro natively, then build others on top of it, without compromising performance.
IR macro is as simple as ER macro, but I wonder how automatic renaming is realized without traversing entire macro output (or input, if we delay renaming until another macro is called). I should look at how Chicken does it.

Peter Bex (2013/09/19 17:50:07):

IR macros currently indeed traverse the entire input *and* output. That's the main disadvantage of this system.
I believe the advantage in safety and convenience are worth the performance hit, and many users of CHICKEN prefer IR macros for the same reason. It's a tradeoff, of course, and it's important to leave the choice to the user; that's one reason why CHICKEN will always provide both ER and IR macros.

shiro (2013/09/20 15:25:22):

This traversal thing is what bothers me about hygienic macros. The reason syntax-case needs built-in pattern matcher is that they want to delay propagating syntactic context to subtrees (they'd claim other reasons, but without this issue of delaying traversal, there's little reason to have pattern matcher tied to the macro system---you can provide it as a separate component.)
I'm still wondering if there are alternatives, even stepping outside of the realm of standard Scheme (e.g. Gauche already has implicit-forcing lazy pair. Can I also have "wrapped pair"? That you can take car and cdr as ordinary pair, but the associated context info is automatically propagated to the leaf as it is decomposed. Without sacrificing normal pair operations, of course.)

Peter Bex (2013/09/20 16:39:57):

I agree the situation of needing a "special" pattern matcher which recognises syntactic information is rather ugly.
Of course you could build something like the "lazy trees" you mentioned, but it sounds tricky when you end up with one tree being shared as a subtree of multiple other trees, where the higher nodes contain conflicting information. I'm not sure if the user can control exactly which of the trees gets forced before the other.
In CHICKEN we associate the syntactic info with symbols using plists, which causes similar problems unless you only attach context info to gensyms (which is what the expander does). Of course this has the disadvantage we already discussed, and the disadvantage of filling up the symbol table like mad, which brings other performance issues with it.
Having special "syntax objects" solves a lot of these problems (and is why the syntax-case proponents are so smug about it), but it's very unlispy since code is no longer list data. That's eminently ugly in my opinion.

Sam Tobin-Hochstadt (2013/12/12 03:58:14):

Shiro, the lazy propagation you describe is exactly how syntax-case works.
Also, the pattern matcher isn't built-in in any fundamental sense. The only reason it's built in to the original syntax case system is to implement the somewhat odd (but useful) communication between syntax-case binding and syntax templates. Other than that, it's just a pattern matcher.

shiro (2013/12/12 04:40:39):

Sam, in syntax-case macros, you can't use standard car/cdr to decompose a syntax object. You can't use Wright's match to decompose it, either. The fact that you need to use a different way to work on the input form is exactly the point I don't like about the syntax-case system.
What I was talking was to treat the input form as if it's just a plain old sexpr, yet avoiding to traverse input form completely for every time a macro is applied.

Gauche Devlog

About

Author

Recent Entries

Recent Comments

Recent Releases