▼ Smalltalk Syntax Macros
• by John Dougan
• Abstract
An initial design and implementation of a syntax macro system for Smalltalk is described.

▼ Motivation
• Initial
The initial motivation for looking into macros was somewhat trivial. I had heard the Common Lisp zealots talk about macros and how they made various things easier and how any language without proper syntax macros was inferior and beneath consideration. Being a self confessed Smalltalk zealot, I found this state of affairs annoying and decided that I would see if I could build a macro system for Smalltalk if for no other reason than to show that it could be done.

I received a number of interesting reactions to this plan when I mentioned it to other Smalltalk developers. Most of the reactions I read as a "Blub" response, the reaction Paul Graham described as happening when a programmer encounters a language or language feature higher up the power scale and doesn't appreciate it. These people would typically claim that it was not a necessary feature, everything you could do with macros could be done using various features of Smalltalk. I do not dispute this, save to observe that the same could be said for a Turing machine.

• Discards
Possibly the biggest use of macros in Common Lisp is to remove the verbose lambda notation from programs. This usage is mostly not necessary in Smalltalk as the lower overhead block notation is used instead. Indeed, there have been attempts to bring a variation of the Smalltalk block notation into various Lisp versions.

Also discarded is any notion of hygienic macros as is commonly found in Scheme or Dylan. While potentially useful, it add complexity to the implementation that I don;'t want in a first effort.

• Inline Literals
One of the points made in response to my declaration of intent was to point out the inline literal facility that was built into VisualAge Smalltalk. This enables the developer to have a chunk of Smalltalk code evaluated at compile time, the result of which is inserted into the bytecode as a literal. Some found it very useful, others found it not at all useful.

This should be an easy use case for any general macro facility to implement.

• DSL
The first thing I thought that ST macros would be useful for is the creation of better DSLs for Smalltalk, as this is one of the important uses claimed for macros by the Common Lisp believers.

Martin Fowler has talked about DSLs at :
http://martinfowler.com/bliki/DomainSpecificLanguage.html
and shown example of implementing a DSL in both C#/XML and Ruby:
http://martinfowler.com/articles/languageWorkbench.html.
In response Rainer Joswig demoed Martin's example in Common Lisp at
http://lispm.dyndns.org/news?ID=NEWS-2005-07-08-1
using significantly less code. I ended up writing a couple VW Smalltalk versions of the example just to see how good or bad it would be and posted the result to the LASTUG list, receiving a thunderous silence in return.

• Proxy
More recently, I have been writing a game in Croquet. One of the implementation choices in Croquet is the syntax for sending a message to an object that resides in a different object space than the sender. This is essentially the classic proxy pattern. A proxy can be considered to have two faces, that of itself and that of the object it is fronting for. Distinguishing which aspect a message send is to go to without ugly hard to read code has been a problem in ST for quite some time.

The usual mechanism for sending to the remote object is typically some variation of a #perform:withArgs: method. However this can be hard to read the intent of and rather verbose. In GemStone this verbosity was resolved by using the DNU handler and a common selector prefix. So to get the class of the local proxy object you would write [gsProxy class] and to get the class of the remote object you would write [gsProxy gsclass]. This method has the problem of accidental name collisions. What if there is a method that happens to start with the chosen prefix? There is also a performance issue since the DNU handler will have to parse out the desired selector from the sent selector.

In Croquet this issue is resolved by sending the message #future to the proxy then sending the message desired to the result. So in Croquet getting the class of the local proxy object would be [croqProxy class] and the class of the remote object would be [croqProxy future class]. But all is not as it seems. The compiler in Croquet, when it sees this future construct, is compiling [croqProxy future class] to [croqProxy futureDo: #class withArgs: {}] which is in essence a special purpose macro facility. After seeing this I realized that this mechanism could profitably be replaced by something more general, based on a macro system. The #future mechanism as implemented has a number of problems particularly in it's poor integration with the Squeak development tools. A proper macro system that resolves these issues would make it easier to build ..... in the future.

• Var args
Standard Smalltalk syntax is quite rigid with regard to arguments and has no provision for what are referred to in other languages as varargs or optional arguments. A standardizing an alternate syntax would be a problem, however enhancing the syntax via a standardized macro facility could end run these issues.

• Compiler Hints
Proxy can be regarded as a special case of this,

•
▼ Design and Implementation
• Collection Constructors
[10 . 100 . 10 squareRoot] `array
• Influences
The bulk of the current design and implementation owes a great deal to Jonathan Bachrach and Brian Rice. Dr. Bachrach did a great del of work of adding macros to various programming languages (Java, Dylan, etc) and Mr. Rice implemented a macro system in his language Slate, which uses a Smalltalk-like syntax.

Most specifically, the Slate macro calling syntax and the Slate approach of making macro sends into message sends on parse trees that return a parse tree that is substituted into the method parse tree at the macro send point were lifted conceptually pretty directly.

• Macro Syntax
To indicate a macro send in a chunk of code, a backquote is prepended onto the message selector. For example, to insert a compile time literal empty dictionary into the method the code would look like [ (Dictionary new) `literal ]

Another example returns the parse tree of the macro call receiver: [(self foo: (3 + 4)) `quote] .

Note that this diverges from the CL approach of making macro calls indistinguishable from normal functional calls. CL does it's binding earlier than ST, so it can tell macros from functions more easily.

The precedence of a macro call is currently the same as a regular message send of the same type. This may change as experience is gained.

• Implementation
The implementation is of macros ass an additional class, MacroMessageSend and an extra step in the compilation process. Macro calls are compiled into MacroMessageSend instances and after the parsing step is done, these calls are expanded. Expansion is done from the root to the leaves.

• Implementing the Literal example

ParseNode>>literal
^LiteralNode on: self asCodeString execute

• Implementing the proxy example
3 changes: ProvyMessageSendNode, Add ValueNode>>proxy, refactor compiler to ask receriver on how to compile send

ValueNode>>proxy
^ProxyMessageSendNode on: self

statementNode>>compileSendTo: messageSend
messsgeSend normalthiingy: self.

ProvyMessageSendNode >>compileSendTo: messageSend
^messsageSend proxyThingy: self

▼ Problems
• Devtool Integration
Code inside macro may not be found/recognized by tools.
Recompilation issue
Keep the parse trees around? Or recompile and examine as necessary?
Keep a list of symbols in the final method?

• ST syntax not well suited to this
Need to make Skeleton Syntax Trees for Smalltalk
`increment example
Special place of message in the code, unlike the lisp function.

• Macro methods cluttering up the node classes
macros are methods on parse tree nodes
not terribly modular.

• ST compilers/parse trees are not standardized
To make it work across any supporting ST would require standardizing aspects of the compiler
Define a standardizable SST which is then built into a platform specific AST?

•
•
•
•
•
▼ References
▼ General Macros
• Macros as multi-stage computations: type-safe, generative, binding macros in MacroML
• Macros that work
• Programmabe Syntax Macros
▼ Jonathan Bachrach
• D-Expressions - Macros for Dylan
• Java Syntactic Extender - Macros for Java
• Java Syntactic Extender at Sourceforge
▼ Slate Smalltalk and Brian Rice
• Slate Smalltalk Home Page
• The Slate Smallltalk Blog
• Slate Language Tutorial
• Slate Syntax
• "Slate - Brian Rice" - Blog Post by James Robertson
• Brian Rice's Home Page
• Brian Rice's del.icio.us linkblog
• "Development Direction"
▼ Olin Shivers
• The anatomy of a loop: a story of scope and control
• Static analysis for syntax objects
▼ Alt. Syntax
• Literal
<10 literal>
<(Float pi * 2) literal>

〈10 literal〉
〖(Set with: 10 ) literal〗
〖literal (Float pi * 2)〗
〖literal (Set with: 10 )〗
〖(Float pi * 2) literal〗
〖literal (Set with: 10 )〗
〖literal (Set with: 10 )〗
〖(Set with: 〖(Set with: 10 ) unquote: 10〗 ) quote: 10〗
〖(Set with: (Set with: 10 ) `unquote: 10) quote: 10〗
〖literal (Set with: 10 )〗
〖literal (Set with: 10 )〗
〖literal (Set with: 10 )〗



() `literal

• Array constructor
〈[10 . 20 . 10 squareRoot . x * 2] array〉
〈array [10 . 20 . 10 squareRoot . x * 2] 〉


• clde Quotes


〖(a b: c) quote〗
〖(a b: c) quote〗


•
•
•
•
•