2014/05/03

Keyword-symbol integration

Aargh, Already May?　Unbelievable. Final stretch turned out to be a long marathon.

If you monitor github repo https://github.com/shirok/Gauche you know it's actively developed, though most changes are minor fixes. I cleared the items on the table I listed last time for 0.9.4 release, but I realized I left out one major item: Keywords.

Gauche has a keyword type, disjoint from other types. I think I inherited it from STk. I also speak Common Lisp and having :keyword is so natural that I tend to forget that treating keywords specially actually conflicts RnRS spec. In RnRS, they must be symbols.

In fact, it became an issue when we adopted srfi:42, in which symbols beginning with : is used. Alex Shinn employed a clever trick during macro expansion to work around the issue.

Now, with more R7RS libraries coming out, I expect this design would be a bigger obstacle.

Some implementations that have keywords avoid this conflict by adopting a syntax different from symbols, e.g. #:keyword. In fact, my original plan was to make the keyword syntax customizable, :keyword, #:keyword or even keyword: (DSSSL style; anybody remember?). That's why we don't have the colon in keyword's name slot. However, mixing these syntax is just confusing, and I strongly prefer :keyword anyway. #:keyword has a merit of not conflicting with symbols, but they are visually too intrusive, and it has been used as uninterned symbols in CL and Gauche.

Technically we don't need to make keywords separate from symbols; indeed, in CL, keywords are just symbols in the keyword package. We can just do the same:

• We can make <keyword> a subtype of <symbol>. Identifiers beginning with ':' are read as an instance of <keyword>, but they're also symbols. So keyword? still works, but keywords also return #t for symbol?.
• We can have gauche.keyword module, that acts as if it has constant bindings of every possible keywords, bound to itself. In reality, we create such bindings on the fly as keywords are read.
• Then the gauche module can inherit gauche.keyword, so that existing usage of Gauche keywords will be preserved. They are symbols, but evaluate to themselves. It also allows programmers to rebind keywords (locally or at the toplevel of their modules) but the effect stays local, and those who conduct such an act should know what they are doing.
• The r7rs module does not inherit gauche.keyword, hence symbols starting with : are just ordinary symbols. All portable R7RS code should run.
• If one wish to use Gauche features and import gauche.core, the bindings of those keywords come with it, so you can call Gauche procedures with keyword arguments just like you're using Gauche.

This looks reasonable to me. There are a couple of catches, but I think they are minor and easily worked around:

• If you import some of Gauche's procedures into R7RS code by (import (only (gauche core) foo ...)), keywords are not imported, so you should call it something like (foo ':bar b ...) i.e. quoting keywords.
• If you import both an R7RS library that exports bindings of symbols that begins with : and gauche.core into R7RS code, whichever comes later would shadow the former. Generally you need to import gauche.core first.

However, this change needs to break the backward compatibility of certain cases, and it turned out it's a bit complicated. I'll describe it later.

Tags: r7rs, keyword

2014/02/07

Almost there?

Just committed changes to cover R7RS-small support.

• Exact complex numbers are not in yet. In next version, I hope.
• Problems have been reported regarding hygienic macro-defining macros (Macros that expands to let-syntax). It's a long-standing issue, but I'm going to give one more try before releasing 0.9.4. If it turns out a lot of work to fix, though, I'll postpone it to the next version.
• Currently, toplevel import is Gauche's by default, which is different from R7RS import. So you need -r7 option for gosh to run R7RS scripts. However, as R7RS gets adopted more widely, users would expect R7RS code just works on Gauche, so requiring -r7 option may confuse them.
  I can't change Gauche's import semantics for the backward compatibility, and I don't want to extend Gauche's import to support R7RS semantics, since those two have different functions (R7RS import ≒ Gauche's use ＝ require + Gauche's import).
  However, I had another idea. The import issue only arises when a Scheme script uses it on toplevel. When a Scheme script is given, gosh start evaluating it in user module. So we can provide the third import form in user module, which checks its arguments and dispatch to either R7RS import or Gauche import.
• And of course, I need to go through documents to update for R7RS.

Tags: r7rs, 0.9.4

2013/11/12

Getting closer

Just added r7rs test suite (taken from ChibiScheme) to the repo.

About two dozens of tests fail, because of the following:

• Lack of exact complex number support. Most of test failures, including number writer tests, are due to this. This isn't required in r7rs, but I plan to implement it anyway just for my satisfaction. (If I don't have time, though, I may put it off).
• Behavior of 'write'---currently it works like write-circular. I'm rewriting the writer submodule and that will address this. (In the current HEAD, you may sporadically see a test failure in gauche.threads, telling assertion failure in the writer submodule. It'll be also fixed in this rewrite.)
• Behavior of 'equal?' (this isn't tested in r7rs test suite.) --- Gauche's current implementation doesn't conform r7rs since it diverges on circular structures.
• Feature 'r7rs' ---- I'll turn this on when above issues are addressed.

This is all I'm aware of to support r7rs. Once done, I'll call it 0.9.4.

My work project is in crunch time, though, so I can't guarantee when it'll be done.

Tag: r7rs

2013/09/18

Macro system extension

I finally added syntax-rules extensions (srfi:46) to Gauche, that makes Gauche's hygienic macro system compatible to R7RS (except a few known bugs).

The current hygienic macro expander is written in C which is an ugly pile of spaghetti. Originally I planned to ditch the legacy code and to write an explicit-renaming macro expander as the new basis of our hygineic macro system, then to implement syntax-rules on top of it.

I like ER-macro since it's transparent to what it is doing for hygienity. It doesn't necessary to be the easiest one to use--- destructuring the input form, then renaming identifiers explicitly would be cumbersome for day-to-day programming. But those things can be easily alleviated by combining other tools. For example, we can just use util.match matcher to destructure the input form (instead of yet another pattern matcher tied to macro system).

In fact, in er-macro branch in the repo I implemented ER-macro expander to some extent. But it turned out I need some more time to substitute the low-level macro layer completely. A major issue is to keep compatibility between ER-macro, which allows raw symbols inserted by the macro expander to capture symbols in macro calls, and the current syntax-rules implementation which turns all symbols into identifiers. (The same issue is described by mjt here, in Japanese.)

Since I'd like to push out R7RS compatible release sooner, I just went into the legacy code and added some more spaghetti to make it work as srfi:46.

＊ ＊ ＊

I realized this enhancement makes syntax-rules a lot more useful. I also adapted define-values form to R7RS, which allows generic formals, as follows:

(define-values (x y . z) (values 1 2 3 4))

z => (3 4)

With R7RS syntax-rules it's not difficult to distinguish proper list and inproper list (see Gauche:lib/gauche/defvalues.scm).

Tags: macro, r7rs, srfi-46

2013/08/01

.gaucherc

When gosh is started in the interactive REPL mode, it loads ~/.gaucherc if it exists. I suppose it may be handy if the user needs his own local setup, even though I personally haven't used the rc file yet---I guess it's a sort of traditional Unix culture.

Recently I realized this feature interferes with R7RS mode. The .gaucherc file is loaded into #<module user>, but what's visible from the user module differs greatly when gosh is invoked with -r7 option. It'll be quite difficult to write .gaucherc that can work both in traditional Gauche mode and r7rs mode.

(Note: I say r7rs mode and Gauche mode, but it's not that there are two separate modes, except the planned reader compatibility modes. You can load R7RS library from standard Gauche program and load Gauche library from standard R7RS program, no matter whether you start gosh with -r7 option or not. The -r7 option merely specifies which environment you're in at the time interactive REPL starts.)

I considered a few options:

• If -r7 option is given, try to load a different rc file, e.g. ~/.gaucherc-r7. This option is less appealing: It scatters more rc files in the home directory. Besides, I expect things you want to do in rc file are likely to need to access Gauche-specific features (e.g. add-load-path) and you can't do that easily from R7RS environment. You would need to create a separate module, e.g. mysetup.scm for the setup code, then (import (mysetup)) from .gaucherc-r7.
• Let rc file be loaded in a module other than user, say, gauche.user module. Then you can use Gauche features in .gaucherc, regardless of -r7 option. This is clean, but adding a new module just for the rc file seems a bit overkill. Besides, it is incompatible to the current version if a user defines something in .gaucherc and expect it visible from the user module.
• Drop .gaucherc support. This is a tempting solution, for it makes things simpler. But who knows? Sometimes this kind of hook comes handy unexpectedly.

Eventually I settled on somewhat compromised design.

• We load .gaucherc to #<module user>, as we have been doing.
• When gosh is started with -r7 option, the initial module will be #<module r7rs.user>, not #<module user>.

It looks a bit ad-hoc solution, but let's give a shot.

Tags: gosh, r7rs, gaucherc