[This is part 4 of a 9-part series of articles on macro expansion algorithms in Scheme. Parts 1 and 3 can be found in the Scheme Repository, available for anonymous FTP on nexus.yorku.ca.] [Parts 1 through 4 were written near the end of an 11-month stretch when I was unemployed and had plenty of time for writing and research. This is no longer the case, and so parts 5 through 9 will probably be a bit slower in coming; please bear with me. Thanks to everybody who's given feedback on parts 1-3.] ------------------------------------------------------------------------------ 4. So What's Wrong Now? So far we've looked at naive macro expansion, Kohlbecker's hygienic macro expansion, and syntactic closures. Kohlbecker's algorithm and syntactic closures both solved the capturing problem inherent in the naive algorithm. Everything should be wonderful now, shouldn't it? Well, not quite. 4.1. Transformer Procedures are Hard to Write In presenting solutions to the hygiene problem, we've ignored a problem that's equally severe: Even though we now find it possible to write macro transformations that Do The Right Thing, it's still a problem to write them and to understand them once they've been written. In fact, it's a real pain, and that goes double for syntactic closures. (macro-transformer-set! 'let (lambda (form) `((lambda ,(map car (cadr form)) ,@(cddr form)) ,@(map cadr (cadr form))))) Even this simple macro transformation takes a bit of work to understand. It's not immediately obvious what part of the original expression all the `car's and `cadr's are meant to be accessing, for instance. Constructing the transformed output expression is a bit easier, thanks to the use of quasiquotation. (Indeed, quasiquotation was introduced into LISP largely for this purpose.) Without it, the transformer would be an even more illegible jumble of calls to `cons' and `list'. A facility like Common LISP's `defmacro' often makes the destructuring part much easier, but it places limitations on the sorts of forms that can be easily defined. What's more, the transformer definition above is itself somewhat naive. If the user forgets a level of parentheses and writes: (let (foo 1) (write foo)) then he or she will receive an error message like ERROR: wrong arg type in cadr (or worse, no error message at all) instead of a useful message that would say exactly what was wrong with the `let' expression. Naturally, error checks can be added to the transformer procedure definition, but these also serve to make the definition much harder to understand, and the "paranoid" programming mindset required to catch all of the cases where things might go wrong distracts one from the real problems at hand. In light of this, many programmers seem to ignore error checking entirely, thus leaving the users to puzzle out the error messages (and the workings of the macro transformer) for themselves. 4.2. Kohlbecker's Thesis: What We Want is a Pattern Language Eugene Kohlbecker (and his Ph.D. thesis advisor Dan Friedman) had the following two Great Insights: 1. We want a method of abstracting away all of the details of "destructuring" macro calls, error checking, and assembling the transformed expression. Ideally, we even want to be able to forget that expressions are represented as lists, symbols, etc., and that macro transformations are represented as transformer procedures, and instead concentrate on the transformation to be performed. 2. We already have just such a method, namely the notation for the "derived expression type" rewrite rules found in the Scheme reports (for example, section 7.3 of [9]). This notation expresses the transformation concisely, without unnecessary reference to lists and symbols. With this in mind, Kohlbecker designed and implemented a tool that would make it possible to use this notation in a concrete way. The resulting tool became `extend-syntax'. We can easily express our `let' transformation using `extend-syntax': (extend-syntax (let) () ((let ((name val) ...) body1 body2 ...) ((lambda (name ...) body1 body2 ...) val ...))) The first argument is a list of the names which will be considered as reserved words in the macro pattern definition. In this case, the only reserved keyword is the macro keyword itself. The second argument is a list of names that should be captured; in this case there are none since the definition of `let' is hygienic. The remaining arguments are production rules of the form (