`file_exists!` Compile time check macro would be useful

community
1

When writing unit tests which loads some "fixture" from a file, e.g. a .json file it is incredible helpful to use include_str! to assert at compile time that the file exists. However, it increases the size of the test binary, right? so maybe that is not wanted.

I want a macro which checks for the presence of a file at compile time, but I want to load the actual contents of the file at runtime. I think no such macro exists to day?

And I want to to this inline in my unit test, i.e. no build script.

So I wanted to check if other would find this useful? Maybe called file_exists

Actually it would be neat with another macro - read_file! which would use file_exists! at compile time and use fs::read_to_string at runtime:

  1. at compile time checks for the existence of the file
  2. at runtime loads it contents
#[test]
fn test_deserialize_user()  {
    // This verifies the existince of the file at compile time, and reads its contents at runtime.
    let fixture = read_file!("user.json"); 
    let deserialization_result = serde_json::from_str(fixture)
    assert!(deserialization_result.is_ok());
}

is that possible - if written part of std? Would anyone else beside me find it useful?

2

it doesn't make much sense to me.

if the file does not exists, do you want a compile error or runtime error?

  • if the former, use include_str!() or include_bytes!();
  • if the latter, use std::fs::read("/path/to/file").unwrap() (or .expect("reason")).

also, there's a chance the file exists at compile time, but doesn't exist when you run it. how do you deal with that?

5 Likes
3

If the file does not exist at compile time => compilation error
If it did exist at compile time, but later at runtime it does not exist => runtime error.

Think of it as a guard rail helping me with getting spelling of file name and relative path right. It is just a faster feedback loop for incorrect filename/path errors!

1 Like
4

oh, this part does make sense.

such a simple macro should be trivial to implement. however, without syn, the current proc_macro is painful to work with, for example, a task as simple as extracting a literal string is not easy! but on the other hand, syn is a heavy dependency, pulling in syn for such a simple task feels overkill.

anyways, here's an incomplete prototype of it, fill the TODOs if you need the functionality:

#[proc_macro]
pub fn file_exists(ts: TokenStream) -> TokenStream {
	let tokens: Vec<_> = ts.into_iter().collect();
	if let &[TokenTree::Literal(ref lit)] = &tokens[..] {
		let lit = lit.to_string();
		// TODO: also accept raw string literals?
		if lit.starts_with('"') {
			let path = unescape(&lit);
			if std::path::Path::new(&*path).exists() {
				return TokenStream::from_str("true").unwrap();
			} else {
				return TokenStream::from_str("false").unwrap();
			}
		}
	}
	TokenStream::from_str(
		r#"compile_error!("file_exists!() only accepts a single literal string")"#,
	)
	.unwrap()
}

fn unescape(s: &str) -> Cow<'_, str> {
	if !s.contains("\\") {
		Cow::Borrowed(s.strip_prefix('"').unwrap().strip_suffix('"').unwrap())
	} else {
		todo!("properly unescape the string")
	}
}
3 Likes
5

That won't cause cargo to rebuild the crate if the file changed between present and not present. There is currently no stable way for proc macros to tell rustc to add things to the dep-info file. Nor is there any way at all (even on nightly) to tell cargo that a crate should be rebuilt when a file that previously didn't exist now exists without also rebuilding if the file still doesn't exist.

6

For this specific case I think something like the following would work:

macro_rules! read_file {
    ($file:literal) => {{
        let _ = include_str!($file);
        std::fs::read($file).unwrap()
    }}
}

Just be warned that the changing the fixture will still cause a rebuild and the size of the fixture will count towards the 4GB total source size across the entire dependency graph of the test executable (the sourcemap uses 32bit indices).

Honestly I wouldn't care much about the size of test binaries unless the fixtures are like hundreds of megabytes or even larger.

1 Like
7

Wouldn't

let _ = const { include_str!($file).len() };

guide compiler not to include the whole fixture but only its length, better for source size? Modification causing a rebuild is inevitable, though.

8

Neither my nor your code include it in the final executable, but both our code includes the full file contents in the source map. It is the expansion of include_str!() itself that causes the file to be added to the source map, long before the compiler notices the const block and len call.