Improvement to syntax extensions


#1

Continuing the discussion from Rust Vs Dlang,I want more experienced:

I’d really want to hear more of your thoughts on syntax extensions. I recall you having some pretty cool ideas about that on internals forum. I’m also interested what Dlang and other languages do in this area.


#2

I’m not sure what exactly you’re asking about; you’re muddling terminology a bit.

If you’re asking about stable procedural macros (i.e. syntax extensions/compiler plugins that are invoked with macro syntax, but are implemented in external Rust code and not macro_rules!), I don’t have much to say other than: it’s a great idea, I can’t wait. macro_rules! is very limited, and just being able to have loops and branches and arbitrary parsers will help enormously. That will net Rust a portion of what D’s capable of. Heck, being able to do FFI means it will be able to do something D can’t.

If you’re asking about macro_rules! macros, I don’t think there’s much point in talking too much about it given the (currently rather vague) plans for the new macro system.

If you’re asking about the new macro system, I could produce a long list of wanted features and changes, but again, I don’t know that there’s much point until it’s clear what the new design will involve.

As for what D can do that Rust can’t… again, I’m not sure it’s really worth talking about extensively, but I can try to summarise: CTFE (compile-time function evaluation) allows you to write complex processing that can operate on basic types, strings, arrays, even tree or graph structures. If the output of CTFE is a string, that can be injected back into the program source via a string mixin. This is roughly similar to procedural macros, except that CTFE invocations in D can operate on arbitrary compile-time expressions; procedural macros cannot. So, even with stable procedural macros, Rust won’t be quite as powerful as D in this respect.

Continuing, said string mixins can contain template invocations that can do operations on types (like extracting the fields from a structure, or map types to values). The expansion of these template invocations can include CTFE calls whose results are consumed by a string mixin… thus letting you pass information back between the two.

And that is something I doubt Rust is going to see any time soon, if ever. Quite aside from how it would interact with type inference and generics, it would require the compiler to be able to cycle back and forth between syntactic and semantic analysis, which are currently entirely distinct phases, and do so an arbitrary number of times. I don’t know the compiler internals well at all, but my impression is that it’d have to be significantly redesigned for that to be tractible.

Now, to be fair, the last I used D, what I’ve described above was not exactly well implemented. It wasn’t tremendously difficult to get the compiler so confused by mixins that symbols would just disappear from some contexts. I don’t want to give the impression that all this is necessarily easy and that Rust is just slacking off.


#3

Yes, this terminology gets me slightly confused. In essence what kind of meta programming capabilities you want to see in Rust?

Personally, for me it was some way to reflect/retrieve type information from a variable outside the macro context.


#4

Personally, for me it was some way to reflect/retrieve type information from a variable outside the macro context.

Note that typeof is a keyword in Rust, presumably reserved for this purpose. Currently it results in E0516 “typeof is a reserved keyword but unimplemented”.