Portable syntax-case

R. Kent Dybvig Oscar Waddell

The syntax-case macro system implements the restricted high-level syntax-rules macros of the Revised⁵ Report as well as the equally high-level but general-purpose syntax-case macros. It also supports a compatible module system that allows import and export of macro bindings as well as bindings for variables and modules.

The system is documented in Section 3.1 and Chapter 8 of The Scheme Programming Language, third edition [1] and in Chapter 9 of the Chez Scheme User's Guide [2]. The former has more introductory material and examples, while the latter documents some aspects of the system not covered by the former.

Aspects of the system and its implementation are described in Syntactic abstraction in Scheme [3] and Extending the Scope of Syntactic Abstraction [4]. Additional examples of its use are given in Writing hygienic macros in Scheme with syntax-case [5].

The portable system is designed to be adaptable with minimal effort to any Revised⁵ Report implementation of Scheme, with the provision of a small set of implementation-dependent hooks to install the expander. Included with the portable expander are definitions of each of the syntactic forms supported by the Revised⁵ Report. Along with a reader, the system forms a complete front-end for Scheme. If the reader and evaluator support source annotations (e.g., file and line number information), these are handled as well to provide source-object correlation.

The portable syntax-case implementation consists of one file, psyntax.ss. The file psyntax.pp is an expanded version of the file that may be used for bootstrapping the system. See the porting notes in psyntax.ss

Please let us know if you successfully port this code to a new Scheme implementation, and also please allow us to include the hooks you develop in doing the port in this directory for use by others.

Please also contact us if you have difficulty porting to a new Scheme implementation or if you discover that the implementation depends on nonportable features of Chez Scheme in some undocumented way.

Release Notes

Version numbers refer to the version of Chez Scheme from which the portable version was extracted.

Version 7.3 (2/26/2007)

[psyntax.ss, psyntax.pp]

• The quasiquote form now guarantees that new pairs or vectors will be allocated any time a nonempty unquote or unquote-splicing form is used, even if the unquoted object is itself a constant. For example, while `(a . b) is equivalent to '(a . b) and returns a constant pair (the same one each time the quasiquote expression is evaluated, e.g., in a loop or procedure called multiple times) `(,'a . ,'b) is equivalent to (cons 'a 'b) so that a new pair is allocated each time the quasiquote expression is evaluated.

• The generate-temporaries procedure no longer creates a gensym but instead creates an ordinary symbol. The symbols differ in name one from each other over the course of a run and are given a special "tmp" mark to distinguish them from other identifiers.

Version 7.1 (8/1/2006)

[psyntax.ss, psyntax.pp]

• syntax now disallows vectors starting with an ellipsis, e.g., #'#(... foo).

• A new syntax->vector procedure, similar to syntax->list has been added.

• Support for quasisyntax, unsyntax, and unsyntax-splicing forms has been added. Implementations are encouraged to supply hash-backquote ( #` ), hash-comma ( #, ), and hash-comma-at ( #,@ ) abbreviations for the three new forms.

Version 7.0 (9/2/2005)

[psyntax.ss, psyntax.pp]

• quasiquote now leaves unquote untouched in `#(a ... unqoute b) rather than trying to treat it as the equivalent of (list->vector `(a ... . (unquote b))).

• Some portability problems have been eliminated: #4(---) in syntax-case is now #(---); a format call has been replaced with a call to symbol->string; a call to list* has been replaced with a quasiquote form; remq is now defined where it is used (update-mode-set); and dummy definitions have been added (but are commented out) for the variables defined (assigned) by the system for implementations that require them.

• syntax-rules now allows fenders.

• Two new derived forms, with-implicit and datum, and one new procedure, syntax->list, have been added. See the Chez Scheme Version 7 User's Guide for descriptions.

• import-only at top level is officially treated as an import.

• An identifier x imported from a module m is now treated in the importing context as if the corresponding export identifier had been present in the import form along with m. See the Chez Scheme Version 7 User's Guide for a description of the now current import semantics.

Version 6.9c (9/2/2004)

[psyntax.ss, psyntax.pp]

• Added a meta prefix for definitions that tells the expander that any variable definition resulting from the definition is to be an expand-time definition available only on the right-hand sides of other meta definitions and, most importantly, to transformer expressions. It is used to define expand-time helpers and other information for use by one or more syntax-case transformers. See the comments in the code for examples of its use.

• Added a new definition form alias which may be used to create aliases from one identifier to another. (alias new-id old-id) makes new-id behave like old-id. It is implemented by mapping new-id to the same label, which may be old-id itself if old-id is a top-level identifier. (This feature was added to support the new import syntax below.)

• Extended the syntax of import to allow for selection and renaming of imports. An import form now has the following syntax:

  (import spec ...)

  Where spec is defined by the following grammar:

  spec		id
  	|	(only spec id*)
  	|	(except spec id*)
  	|	(add-prefix spec id)
  	|	(drop-prefix spec id)
  	|	(rename spec (new-id old-id)*)
  	|	(alias spec (new-id old-id)*)

  only imports only the listed identifiers; except imports all but the listed identifiers; add-prefix adds the given prefix to the imports; drops-prefix drops the given prefix to the imports, which must all share the given prefix; rename provides new names for the listed imports; and alias provides additional names for the listed imports.

• Eliminated separate per-body-form r and mr and with them the need to destructively update environments. The per-body-form r and mr did nothing more than control the extent of let-syntax/letrec-syntax bindings to prevent things like

  (let ()
    (let-syntax ((a (identifier-syntax 3)))
      (define-syntax b (identifier-syntax a)))
    b)

  where a is not actually run until expand-time control has left the let-syntax form. This kind of thing is harmless, though, and in any case, a is visible only within the body of the let-syntax form by virtue of the wrap.

• Replaced some uses of continuation-passing style in the chi and parse procedures.

• Cleaned up chi-top-module.

Version 6.9a (5/13/2003)

[psyntax.ss, psyntax.pp]

• Made several modifications to support environments. Changed name of put-global-definition-hook to put-cte-hook. Added new put-global-definition-hook that actually does what you'd expect, i.e., install a binding that can be retrieved by get-global-definition-hook. Added third top-token argument to both $sc-put-cte and build-cte-install, identifying the top-level environment into which the binding should be installed. Added new read-only-binding? hook that can be used to identify bindings that are read only (currently always returns false). Added put-import-binding, which installs bindings that can be retrieved by get-import-binding.

• The new user-visible environment features are an environment? predicate and interaction-environment. Another new procedure, $make-environment, is also defined by the system but is intended as a low-level feature to be used by the host implementation to define other environments, such as ieee, scheme-report, and null, and possibly even user-defined environments, that are not directly supported by the portable syntax-case expander.

• import-only is now supported at top level.

• define-syntax now supports the "defun" syntax, e.g.:

  (define-syntax (foo x) (syntax-case x ---))

• module, import, and import-only are now macros that expand into $module, $import, and $import-only. This change was made primarily to simplify the expander by moving the handling of anonymous modules into the macro versions.

• The build procedures, e.g., build-application, now receive the entire annotated expression rather than just the source extracted from the source annotation. Along with this change, renamed source, src and s to ae (for annotated expression) in many places, since the object passed around is now the annotated expression rather than the source extracted from the annotated expression. annotation-stripped is now expected to return the stripped version of an annotated expression, i.e., strip-annotation simply grabs for the stripped version rather than traversing the (possibly cyclic) annotated expression.

• Added a new built-lambda? predicate that is used to recognize lambda expressions built by build-lambda. (This is used in handling deferred transformer bindings; see below.)

• Modified build-begin to weed out leading calls to void and suppress the begin wrapper when only one subexpression is present. It assumes built void calls are represented as ordinary s-expressions; change this if your build-application uses some other representation for applications.

• make-binding now builds quoted structure if type is known and value is false.

• transformer-env now strips out syntax variables, as it should have been doing all along.

• So we can't be caught being lazy, we defer the evaluation of a local transformer only if it is a lambda expression.

Version 6.9 (7/12/2002)

[psyntax.ss, psyntax.pp]

• Changed chi-top-module so that definitions and initialization forms in compiled files are not evaluated (by default) at visit time.

• Recoded generate-id so that generated symbols have a standard print representation and to avoid ASCII dependencies.

• Extended the syntax of syntax-case patterns to allow a fixed number of items after an ellipsis. Ellipses are therefore no longer constrained to appear only at the end of a list- or vector-structured form, but only one ellipsis can appear at a given level of a list- or vector-structured form. For example, (a ... b) matches a list of one or more elements, with b bound to the last element and a ... to all but the last element.

• Added a local definition for a one-clause version of let-values and recoded uses of call-with-values using let-values.

• Fixed two bugs in the portable expander. One resulted in an error in vector-ref when redefining a variable at top level that was previously imported from a module. The other caused the wrong identifier to be exported in certain circumstances, perhaps resulting in an inappropriate identifier out of context error.

Version 6.8 (2/6/2002)

[psyntax.ss, psyntax.pp]

• Added support for visit and revisit procedures, which are relevant for systems that support compile-file: visit is like load but loads only compile-time information, and revisit is like load but loads only run-time information.

• Cleaned up and extended eval-when to handle new situations visit and revisit. A translation table that clearly shows what happens with nested eval-when forms is given in the source.

• Added literal-identifier=?. literal-identifier=? is similar to free-identifier=? except that the former equates top-level identifiers that come from different modules, even if they do not necessarily resolve to the same binding. syntax-rules and syntax-case employ literal-identifier=? to compare identifiers listed in the literals list against input identifiers. literal-identifier=? is intended for the comparison of auxiliary keywords such as else in cond and case, where no actual binding is involved.

• Fixed a bug in the portable expander that caused an application of cdr to the empty list in build-sequence for top-level modules with no exported definitions other than reexports, e.g.,

  (module m1 (a) (define a 3))
  (module m2 (a) (import m1))
  (let () (import m2) a)  3

• Fixed a bug in the portable expander that prevented use of "hidden" top-level definitions in the output of a macro, e.g.:

  (define-syntax counter
    (syntax-rules ()
      ((_ ref incr e)
       (begin
         (define hidden e)
         (define ref (lambda () hidden))
         (define incr (lambda (n) (set! hidden (+ hidden n))))))))
  (counter get bump 17)
  (get)  17
  (bump 4)
  (get)  21
  hidden  unbound error

• An incompatible change to the expander has been made to reduce the loss of source information when one macro generates another macro definition: List and vector structure in the subexpression of a syntax form is no longer guaranteed to be list and vector structure in the output form except where pattern variables are contained within that structure. For example, #'(a ...), where a is a pattern variable, is guaranteed to evaluate to a list, but the constant structure (a b c d), none of a, b, c, and d are pattern variables, may not. The practical consequence of this change is that constant structures must be deconstructed using syntax-case or syntax-rules rather than car, cdr, and other list-processing operations.

Version 6.3 (8/30/2000)

[psyntax.ss, psyntax.pp]

• The portable syntax-case macro system is now up-to-date with respect to The Scheme Programming Language, second edition [6] and the Chez Scheme User's Guide [2].

References

[1] R. Kent Dybvig. The Scheme Programming Language, third edition. MIT Press, 2003. Introduction and reference manual for ANSI Standard Scheme with numerous short and extended examples and exercises. [full text, bibtex]

[2] R. Kent Dybvig. Chez Scheme User's Guide. Cadence Research Systems, 1998. User's guide and reference manual for Chez Scheme. Complements [1]. [full text, bibtex]

[3] R. Kent Dybvig, Robert Hieb, and Carl Bruggeman. Syntactic abstraction in Scheme. Lisp and Symbolic Computation, 5(4):295-326, 1993. Describes the syntax-case macro system and its implementation and argues why it is superior to previous attempts at hygienic macro systems. [full text, abstract, bibtex]

[4] Oscar Waddell and R. Kent Dybvig. Extending the scope of syntactic abstraction. In Conference Record of the Twenty Sixth Annual ACM Symposium on Principles of Programming Languages, 203-213, January 1999. Describes the Chez Scheme module system and its interaction with the syntax-case expander. [full text, abstract, bibtex]

[5] R. Kent Dybvig. Writing hygienic macros in Scheme with syntax-case. Technical Report 356, Indiana Computer Science Department, June 1992. Introduces the syntax-case macro system through a series of examples. Some overlap with [3]. [full text, abstract, bibtex]

[6] R. Kent Dybvig. The Scheme Programming Language, second edition. Prentice Hall, 1996. Second edition of [1]. [full text, bibtex]