2014/05/03

Keyword-symbol integration

Aargh, Already May?　Unbelievable. Final stretch turned out to be a long marathon.

If you monitor github repo https://github.com/shirok/Gauche you know it's actively developed, though most changes are minor fixes. I cleared the items on the table I listed last time for 0.9.4 release, but I realized I left out one major item: Keywords.

Gauche has a keyword type, disjoint from other types. I think I inherited it from STk. I also speak Common Lisp and having :keyword is so natural that I tend to forget that treating keywords specially actually conflicts RnRS spec. In RnRS, they must be symbols.

In fact, it became an issue when we adopted srfi:42, in which symbols beginning with : is used. Alex Shinn employed a clever trick during macro expansion to work around the issue.

Now, with more R7RS libraries coming out, I expect this design would be a bigger obstacle.

Some implementations that have keywords avoid this conflict by adopting a syntax different from symbols, e.g. #:keyword. In fact, my original plan was to make the keyword syntax customizable, :keyword, #:keyword or even keyword: (DSSSL style; anybody remember?). That's why we don't have the colon in keyword's name slot. However, mixing these syntax is just confusing, and I strongly prefer :keyword anyway. #:keyword has a merit of not conflicting with symbols, but they are visually too intrusive, and it has been used as uninterned symbols in CL and Gauche.

Technically we don't need to make keywords separate from symbols; indeed, in CL, keywords are just symbols in the keyword package. We can just do the same:

• We can make <keyword> a subtype of <symbol>. Identifiers beginning with ':' are read as an instance of <keyword>, but they're also symbols. So keyword? still works, but keywords also return #t for symbol?.
• We can have gauche.keyword module, that acts as if it has constant bindings of every possible keywords, bound to itself. In reality, we create such bindings on the fly as keywords are read.
• Then the gauche module can inherit gauche.keyword, so that existing usage of Gauche keywords will be preserved. They are symbols, but evaluate to themselves. It also allows programmers to rebind keywords (locally or at the toplevel of their modules) but the effect stays local, and those who conduct such an act should know what they are doing.
• The r7rs module does not inherit gauche.keyword, hence symbols starting with : are just ordinary symbols. All portable R7RS code should run.
• If one wish to use Gauche features and import gauche.core, the bindings of those keywords come with it, so you can call Gauche procedures with keyword arguments just like you're using Gauche.

This looks reasonable to me. There are a couple of catches, but I think they are minor and easily worked around:

• If you import some of Gauche's procedures into R7RS code by (import (only (gauche core) foo ...)), keywords are not imported, so you should call it something like (foo ':bar b ...) i.e. quoting keywords.
• If you import both an R7RS library that exports bindings of symbols that begins with : and gauche.core into R7RS code, whichever comes later would shadow the former. Generally you need to import gauche.core first.

However, this change needs to break the backward compatibility of certain cases, and it turned out it's a bit complicated. I'll describe it later.

Tags: r7rs, keyword