Here is how to avoid being broken by Syntex updates

Introduction to Syntex

Skip this section if you know what Syntex is.

Libraries that provide a compiler plugin to do code generation (among them Serde, Quasi, Indoc, and many others) sometimes use a hack called Syntex to allow the code generation to work on the stable and beta release channels of Rust, which do not support compiler plugins. Syntex is basically a copy of the syntax package of the nightly Rust compiler, so by using Syntex these libraries are using the nightly compiler (with its plugin support) even on the stable and beta channels. The codegen code can be written to use the compiler's libsyntax on nightly, and Syntex on stable and beta.

The problem

Problems arise when the nightly compiler's libsyntax diverges from Syntex. Codegen code that is written to work with either one will fail to compile on nightly. Whenever this happens, the Syntex team (really @erickt) releases a new version of Syntex that once again aligns with the libsyntax in the nightly.

Until all of your dependencies adopt the newly released Syntex version, your crate may fail to compile if it has a transitive dependency on more than one version of Syntex. For example your crate may depend on Serde which is on Syntex 0.21 and Quasi which is on Syntex 0.20. Typical problematic code looks like this:

// register plugins
let mut registry = syntex::Registry::new();
serde_codegen::register(&mut registry);
quasi_codegen::register(&mut registry);

// run the plugins on your code
let src = Path::new("main.rs.in");
let dst = Path::new(&out_dir).join("main.rs");
registry.expand("", &src, &dst).unwrap();

The solution

Based on a productive discussion on issue serde#356, the Serde team has decided to provide an alternative way of using Serde codegen without exposing our dependency on Syntex. The new code would look like this:

let src = Path::new("main.rs.in");
let dst = Path::new(&out_dir).join("main.rs");

// run plugins directly
serde_codegen::expand(&src, &dst).unwrap();
quasi_codegen::expand(&dst, &dst).unwrap();

This allows each codegen dependency to use its own version of Syntex without needing to register into a shared registry. We think this will avoid a good deal of Syntex-related breakage in the future and we would like to encourage other codegen crates to take a similar approach.

6 Likes

That's great news! Lately I've just been implementing serialisation manually in other libraries to get around this issue entirely. It'd be good to be able to depend on serde+syntex.