Composing generic closures together, with lifetimes

help
1

Hey y'all, I'm running into an issue when composing closures together. My issues started when I began allowing the closures to return a generic type instead of &[u8] in bind_handler_to_parser(). If I restore the commented out lines (and remove their counterparts) everything compiles fine.

fn main() {
    let buf = &mut [0u8; 4096];
    let mut buffer = Buffer { buf };

    bind_handler_to_parser(create_parser(), handler)(&mut buffer);
}

fn handler(_: &[u8]) {}

pub struct Buffer<'a> {
    buf: &'a mut [u8],
}

impl<'a> Buffer<'a> {
    pub fn bytes(&self) -> &[u8] {
        self.buf
    }
    pub fn update_state(&mut self) {}
}

pub fn bind_handler_to_parser<'a, P, O, H>(parser: P, handler: H) -> impl Fn(&'a mut Buffer)
// pub fn bind_handler_to_parser<P, H>(parser: P, handler: H) -> impl Fn(&mut Buffer)
where
    P: Fn(&'a [u8]) -> O,
    // P: Fn(&[u8]) -> &[u8],
    H: Fn(O),
    // H: Fn(&[u8]),
{
    move |buffer: &mut Buffer| {
        let bytes = buffer.bytes();
        handler(parser(bytes));
        buffer.update_state();
    }
}

pub fn create_parser() -> impl Fn(&[u8]) -> &[u8] {
    move |bytes| bytes
}

I think it comes down to my bad use of lifetimes, but I can't figure out how to annotate this code to get it compiling again. Making the return value of parser generic in bind_handler_to_parser() required that I add lifetime parameters to the signature, but I feel like the lifetimes I supplied are maybe overly restrictive. This is the error I am currently getting:

error[E0502]: cannot borrow `*buffer` as mutable because it is also borrowed as immutable
  --> src/main.rs:32:9
   |
21 | pub fn bind_handler_to_parser<'a, P, O, H>(parser: P, handler: H) -> impl Fn(&'a mut Buffer)
   |                               -- lifetime `'a` defined here
...
30 |         let bytes = buffer.bytes();
   |                     ------ immutable borrow occurs here
31 |         handler(parser(bytes));
   |                 ------------- argument requires that `*buffer` is borrowed for `'a`
32 |         buffer.update_state();
   |         ^^^^^^^^^^^^^^^^^^^^^ mutable borrow occurs here

error: aborting due to previous error

I understand what it's saying, but I don't know how to address it. Any help would be greatly appreciated. Thanks!

2

Here's a couple options to work around it. Not sure exactly what you have in mind for the intermediate type O, so these may not solve your problem.


pub fn bind_handler_to_parser<P, O, H>(parser: P, handler: H) -> impl Fn(&mut Buffer)
where
    O: ?Sized,
    P: Fn(&[u8]) -> &O,
    H: Fn(&O),
{
    move |buffer: &mut Buffer| {
        let bytes = buffer.bytes();
        handler(parser(bytes));
        buffer.update_state();
    }
}

Playground 1

fn main() {
    let buf = &mut [0u8; 4096];
    let mut buffer = Buffer { buf };

    bind_handler_to_parser(|b| handler(create_parser()(b)))(&mut buffer);
}

pub fn bind_handler_to_parser<H>(handler: H) -> impl Fn(&mut Buffer)
where
    H: Fn(&[u8]),
{
    move |buffer: &mut Buffer| {
        let bytes = buffer.bytes();
        handler(bytes);
        buffer.update_state();
    }
}

Playground 2

3

As far as the why it’s happening, I think you might be running into this issue LINK where the compiler currently can’t figure out what you’re doing. The first solution explicitly tells it that the intermediate result borrows from the input. Whereas the second solution basically doesn’t make the intermediate type generic at the function level which is easier for the compiler to deal with.

4

Thanks so much for the quick response, and the explanations! The intermediate type is a generic enum in my actual code, and the different variants are handled in bind_handler_to_parser() so I think your first solution works better with what I currently have. It got me moving forward :slight_smile:

What I'm ultimately trying to do is combine parsers like this (if you're familliar with the nom crate then this might seem familiar)

pub fn both<P1, O1, P2, O2>(parser1: P1, parser2: P2) -> impl Fn(&[u8]) -> (O1, O2)
where
    P1: Fn(&[u8]) -> O1,
    P2: Fn(&[u8]) -> O2,
{
    move |bytes| (parser1(bytes), parser2(bytes))
}

So I can start doing things like this

bind_handler_to_parser(both(create_parser(), create_parser()), |(a,b)| {})(&mut buffer);

But that won't compile (Playground). I'm not sure the same technique can be used here, as returning a reference to a tuple creates a dangling reference. Any ideas?

Thanks for the link to the GitHub issue, I'm reading through it and there do seem to be some similar issues other people are having.

5

I played around with this for a bit. Honestly I'm not sure what the best way to do that is. My only thought was to make bind_handler_to_parser a macro which gets around the issue with needing to specify the intermediate type. Here's what I did. I rearranged a couple things to get it to compile, so it still might not work for you.

Playground

6

Thanks so much for helping me look into this, I really appreciate it. You taught me about ?Sized and your macro solution looks like it will work for me!

I'm still new to Rust, so I still have a lot to learn. Do you think that I just ran into a tricky case for the compiler to understand, or am I doing something that fundamentally isn't a good fit for Rust?

7

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.