How to specify a spanned map as return type?

help
1

To follow DRY principles I'm trying to reuse boilerplate as a function call in Rust:


 
use logos::Logos;
use std::fmt;
use std::ops::Range;

#[derive(Logos, Debug, PartialEq, Copy, Clone)]
pub enum Token<'a> {
    #[regex("[ \r\n\t]+", logos::skip)]
    #[error]
    Error,
}

// use Logos lexer in Lalrpop, from: https://github.com/ratmice/json-pop/blob/master/src/lex.rs#L54
impl<'input> TheToken<'input> {
  pub fn to_lalr_triple((t, r): (TheToken<'input>, Range<usize>)) -> Result<(usize, TheToken, usize), CompilationError> {
    if t == TheToken::Error {
      Err(CompilationError::LexicalError { range: r })
    } else {
      Ok((r.start, t, r.end))
    }
  }
}


#[derive(Debug)]
pub enum CompilationError {
  LexicalError { range: Range<usize> },
  NumericalError { range: Range<usize> },
  UnterminatedStringLiteral { range: Range<usize> },
}


// here is the failure:
fn to_lexer<'input>(bytes: &str) -> Box<dyn Iterator<Item=TheToken<'input>>> {
  TheToken::lexer(bytes).spanned().map(TheToken::to_lalr_triple)
}

and from what I gather, map should return as an iter based on:

However, I am using spanned first, and that changes the return type dramatically. If I paste in what the compiler says it's expecting, there are loads of syntax errors. So how can I indicate the return type properly?:

error[E0308]: mismatched types
--> src/parser/lex.rs:110:3
|
109 | fn lexed<'input>(bytes: &str) -> Box<dyn Iterator<Item=TheToken<'input>>> {
| ---------------------------------------- expected Box<(dyn std::iter::Iterator<Item = TheToken<'input>> + 'static)> because of return type
110 | TheToken::lexer(bytes).spanned().map(TheToken::to_lalr_triple)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected struct Box, found struct std::iter::Map
|
= note: expected struct Box<(dyn std::iter::Iterator<Item = TheToken<'input>> + 'static)>
found struct std::iter::Map<SpannedIter<'_, TheToken<'_>>, fn((TheToken<'_>, std::ops::Range<usize>)) -> Result<(usize, TheToken<'_>, usize), CompilationError> {TheToken::<'_>::to_lalr_triple}>

2

Apologies in advance... Due to Lalrpop having generated code I do not have an easy way to show the parse function.

3

That's a type error, not a syntax error. I've only looked at the code, and haven't tried to run it, but three things stand out to me.

Your return type is

Box<dyn Iterator<Item=TheToken<'input>>>

However,

  • You're not returning a Box
  • Your to_lalr_triple returns Result<_, _>, so after the map, you have an iterator over Result<_, _> and not over TheToken<'input>
  • There's probably a somewhat subtle lifetime issue, as there was in that Stackoverflow

So if you just wanted to get to_lexer working, my suggestion would be something like:

fn to_lexer<'input>(bytes: &str)
-> Box<
    dyn Iterator<
        Item=Result<(usize, TheToken<'input>, usize), CompilationError>
    > 
    + 'input
>
{
    Box::new(
        TheToken::lexer(bytes).spanned().map(TheToken::to_lalr_triple)
    )
}

Where I have

  • Boxed up the iterator in the function body
  • Updated the Item= to match to_lalr_triple in the return type
  • Added a + 'input bound inside the Box in the return type

(Untested.)

4

Incredible, I was so close! Good call you nailed it.