TT-muncher macro causing "variable not in scope" error

help
1

I'm making a simple macro for CLI (something much simpler than clap). A simple version of it already works:

cli!(param1, param2, param3, "help text");

Parses 3 arguments into param1..3. If there are < 3 args, shows help text and exits.

I tried to add optional type annotation that will make it parse() the args into the named type. Expected usage:

cli!(param1, param2:u32, param3:f64, "help text");

(param1 becomes String by default.)

This requires repetitions of tt's, which can be parsed with TT Munchers.

Here's what I wrote. The compiler complains that in a nested pattern (@phase2 ...) => { __ here __ } a var is not defined. Even though the fully expanded macro makes a seemingly valid code (the var is mentioned where it is already defined).

Down in this macro book, I read that I can make invalid token trees that will be validated only after the entire macro finished, but I don't see if it can fix the undefined var problem.

The code:

use std::{ffi::OsString, str::FromStr}; // importing for clarity of code in the example

macro_rules! cli {
    // last bit -- the help text
	(@phase2 $help_text:literal) => {
		let help_text = $help_text;
	};
	// argument with type annotation
	(@phase2 $var_name:ident: $var_type:ty, $($token2:tt)*) => {
		let $var_name:$var_type = __parse_var(stringify!($var_name), &mut args, help_text).unwrap();
		cli!(@phase2 $($token2)*)
	};
	// argument without type annotation, string by default
	(@phase2 $var_name:ident, $($token2:tt)+) => {
		let $var_name: String = __parse_var(stringify!($var_name), &mut args, help_text).unwrap();
		cli!(@phase2 $($token2)+)
	};

    // ENTRY pattern
	($($var_name:tt)*) => {
		fn __exit(help_text: &str) {
			println!("{}", help_text);
			std::process::exit(64)
		}

		let mut args: Vec<OsString> = vec!["value1".into(), "value2".into()]; // std::env::args_os().collect();
		args.reverse();
		args.pop();
		
		fn __parse_var<T: FromStr>(name: &str, args: &mut Vec<OsString>, help_text: &str) -> Option<T> {
			let Some(tmp_var) = args.pop() else {
				__exit(help_text);
			};
			let Ok(result) = tmp_var.to_str().unwrap().parse::<T>() else {
				__exit(help_text);
			};
			Some(result)
		}

		cli!(@phase2 $($var_name)*)
	};
}

fn main() {
    cli!(param1, param2, "Help text");
}

Error:

error[E0425]: cannot find value `args` in this scope
  --> src/main.rs:15:67
   |
15 |         let $var_name: String = __parse_var(stringify!($var_name), &mut args, help_text).unwrap();
   |                                                                         ^^^^ not found in this scope
2

The problem here is identifier hygiene.

TLDR: identifiers are invisibly "tagged" with a unique ID for every macro expansion. It doesn't work because you define args in one expansion, but use it in different ones.

The solution is to pass names like args and help_text down to successive macro invocations.

Edit: okay, here is a modified version of your code.

This is still incomplete, because you're using help_text before you define it, and that's not going to work. You could switch to using an Option<&str>, or possibly re-jig how the expansion is done.

I also had to change __exit to actually diverge (try removing -> ! and see what happens), and there are some other errors, but the macro issue is addressed, at least.

2 Likes
3

Thanks, I edited it too, put a stub for help_text and managed to get args working.

4

I don't know if you will get notified of the edit I did since you've already seen my reply, so just in case, I'll annoy you a second time.

5

although the macros match on token trees, they expands to AST, and they are hygienic, so you can't expands macros to "piece" of code and expect they combined to work as valid code. for instance, you can't do this:

macro_rules! define_foo {
	 () => {
		  let foo = 42;
	 };
}
macro_rules! use_foo {
	 () => {
		 println!("value of foo is {}", foo);
	 };
}
fn test() {
	 define_foo!();
	 use_foo!();
	 println!("this is also error: {}", foo);
}

you'll have to pass the identifier to the scope of the macros, like this:

macro_rules! define_variable {
	 ($var:ident) => {
		  let $var = 42;
	 };
}
macro_rules! use_variable {
	 ($var:ident) => {
		  println!("value of variable is {}", $var);
	 };
}
fn test() {
	 define_variable!(foo);
	 use_variable!(foo);
	 println!("this is ok: {}", foo);
}
1 Like
6

actually, you don't need tt munchers for the use case shown in your example code. here's what I came up with (note: personally I like to group macro arguments in parenthesis instead of using comma to separate)

macro_rules! cli {
	($($var:ident $(:$type:ty)?),+ , $help:literal) => {
		let mut args = std::env::args();
		// variable names are hygienic, functions are not.
		// so here I use a closure to avoid leaking function definition
		let fail = || -> ! {
			eprintln!($help);
			::std::process::exit(1)
		};
		cli!(@define_variable (args fail) $(($var $($type)?))+)
	};
	// base case, finish
	(@define_variable ($args:ident $fail:ident)) => { };
	// no type, use String
	(@define_variable ($args:ident $fail:ident) ($var:ident) $($rest:tt)*) => {
		cli!(@define_variable ($args $fail) ($var String) $($rest)*);
	};
	// explict type
	(@define_variable ($args:ident $fail:ident) ($var:ident $type:ty) $($rest:tt)*) => {
		let s = $args.next().unwrap_or_else(|| $fail());
		let $var: $type  = FromStr::from_str(&s).unwrap_or_else(|_err| $fail());
		cli!(@define_variable ($args $fail) $($rest)*);
	};
}
1 Like
7

Others have already addressed the issue. My additional recommendation would be to simply write a procedural macro instead of trying to make the declarative method work using increasingly dirty (no pun intended) tricks. Proc-macros aren't hygenic, so they'll match your expectations more closely.

1 Like