Ventonegro

I am currently reading the third chapter of Lisp in Small Pieces. It is really a wonderful book. By teaching how to implement Lisp, it teaches a lot about using the language too. Moreover, reading it sometimes I feel the urge that almost every Schemer has at one time or another felt: The urge to write his own Scheme system.

Ok, I know the World is already sick of Schemes, it only takes a look at this list to know why. But I still think I have to do it, if only for my personal learning. There is no need to bore the World to death with another half-done Scheme. Sometimes I think about features that could make my Scheme useful, though. They are probably never going to be implemented, but here they are:

Full R5RS compliance

The R5RS is a piece of art (R6RS, on the other hand, is a stain, a mess). Of course it does not describe a terribly useful system, but it beautifully describes the core of one. Everything else can then come from there. For more compliance with other systems, the SRFIs can be used as the basis for a standard library.

Bytecode virtual machine

Although Scheme is a dynamic language, there is a lot that can be known about the program before executing it. Lexical variables, never assigned variables, closure and continuation creation etc. This can be processed at compile time, allowing for a simple and fast virtual machine. Interpreting the source code directly is easier but very inefficient for any serious system.

Incremental garbage collection

Stop-and-go garbage collectors are easier to implement, but in some applications the pause is inacceptable. For applications like interactive games, the smooth user experience only can be achieved by an incremental garbage collector. It would be nice if it was a moving collector too, to avoid too much fragmentation of the heap (some naive C programmers think the use of malloc and free can beat a garbage collector in any circustances, until they are hit in the face by heap fragmentation. There is a reason the Apache web server uses memory pools). If I remember correctly the CLR garbage collector is very good and compacts the memory without the use of from- and to-spaces, which is cool, because uses less heap memory. Hmm, while I am dreaming about my ideal garbage collector, make it concurrent too (see below).

Concurrency

These days the hot topic is concurrency. New personal computers with two and four cores sell more and more every day. The programming languages/environments that makes easy for the programmer to use these new cores are going to be big. Erlang is the hottest thing right now in this regard, although Haskell is a strong candidate too. In the mainstream Java has a concurrent VM, but Java uses the old lock/unlock paradigm of other mainstream languages, Clojure is the new language on top of the JVM that leverage that power for the programmer. But although this is all so hot, the only thing that seems to be happening to Scheme is Termite, which as far as I know can only use green threads. My hypotetical Scheme would have a concurrent VM, some Scheme compilation techniques already give a start in that direction. There would be maybe a spawn-continuation form that would take a continuation captured with call/cc and resume it in parallel with the current one, in a M:N model (M continuations spread on N OS threads).

Just-in-time compilation

Well, once dreaming, why not go all the way? One of the nice things about Scheme is the bottom-up, incremental development. Developing at the REPL gives instant feedback and ease debugging. What if it could give the speed of compiled-to-CPU languages as well? Another advantage is hot-swapping code in a running application. This can’t be done with Chicken or Gambit, which are fast, but Scheme-to-C compilers. Larceny, Chez and Ikarus can do this. By using a bytecode VM, the system is portable to wherever there is a C compiler, and by adding JIT support for some architectures it can be made fast too.

Escape continuations

Full, first-class continuations with indefinite extent are a powerful and mind-bending feature, but unfortunately, are usually hard to implement and causes pauses when invoked. Besides them, my hypotetical Scheme would have escape continuations. Instead of completely overwriting the execution stack, they simply unwind it until some recorded point, which can be made very fast. Exceptions would them be implemented as escape continuations rather than full ones.

Easy FFI

I am a long-time Lua user, and it is amazing how they could get the C API so good. Not only it is very easy to add new C/C++ libraries to Lua, it is trivial to embed the Lua interpreter itself in a legacy application, or an application that needs to be mostly in C/C++, such as games. As Scheme, unfortunately, is not one of the World’s favorite languages, the capacity to talk to the outside World is fundamental. And by being embeddable (which is the goal of Guile, but it sucks for Windows), it would be easier to take Scheme to another domains where it is not so popular today.

Well, I guess this is it. Thinking about it, there seems to be no Scheme system today with all these features. Will I have the skills to write something like this after LiSP? I doubt it, only the garbage collector would take me ages. But it would be nice if such thing existed…