Syn: parsing file with shebang

I'm trying to process Rust source files using syn. The problem appeared with parsing the shebang line. Specifically, this code errors:

use proc_macro2::TokenStream;
use syn::{File, parse2};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let code = "#!/bin/rust";
    let stream: TokenStream = code.parse().unwrap();
    let _: File = parse2(stream)?;
    Ok(())
}

Playground

Am I doing something wrong? Shebang seems to be supported by Syn, according to its docs.

Probably because shebang is not TokenStream but a rather special thing which can be appeared only at the start of File.
https://docs.rs/syn/1.0.5/src/syn/file.rs.html#83 It seems like parsing is not supported because it cannot be represented as ParseStream either.

I think it only supports parsing inner attribute. It parse this successfully
(It is not that :D, they are different things)

In other words, shebang doesn't obey Rust's lexical rules. Thus, it cannot be decomposed to Rust tokens i.e. TokenStream.
Such special header has to be stripped from str representation before transformed to TokenStream representation.

From syn's doc:

Struct syn::File
A complete file of Rust source code.

I guess syn::File::shebang is just a placeholder for future use?? Currently it is not a valid Rust syntax?

If you want to parse:

Playground: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=562da8025f070b122127a5840b54ec91

This approach may appear to work in practice but it is not correct because it doesn't work when shebang isn't a invalid token stream e.g. unbalanced parentheses or unused character such as backtick or control characters.

1 Like

There is: ::syn::parse_file

use ::syn::{File, parse_file};

fn main ()
  -> Result<
        (),
        Box<dyn ::std::error::Error>,
    >
{
    let code = "#!/bin/rust";
    let _: File = parse_file(code)?;
    Ok(())
}
3 Likes

Thanks! That's the missing piece.