Passing stack allocated structs containing references into a closure

[gatesn]

I've created a repro using notional concepts, so it does seem a little bit contrived! However I essentially have a Rust FlatBuffer that wraps a reference to a slice of bytes.

I want to parse these bytes into a new slice (by reference), wrap them up in a struct that implements some dyn trait, then pass this struct into a closure in order to avoid any heap allocations.

However, I'm struggling to annotate the correct lifetime for "SomeMetadataThingy" since it is bound by both the lifetime of the original slice of bytes, as well as the lifetime of the stack-allocated "first" and "second" objects.

Here is the playground: https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=da18c6fb6d7fad843a5e1e1671b3f48b

struct Bytes<'a> {
    // The raw bytes of the message.
    bytes: &'a [u8],
}

trait WithMetadata<M> {
    // Invoke the closure with a parsed metadata object.
    fn with_metadata<F, T>(&self, f: F) -> T
    where
        F: FnOnce(&M) -> T;
}

trait SomeDynTrait {}

impl SomeDynTrait for Bytes<'_> {}

// Some parsed metadata object
struct SomeMetadataThingy<'a> {
    first: &'a dyn SomeDynTrait,
    second: &'a dyn SomeDynTrait,
}

impl<'meta, 'bytes> WithMetadata<SomeMetadataThingy<'meta>> for Bytes<'bytes> where 'bytes: 'meta, Self: 'meta {
    fn with_metadata<F, T>(&self, f: F) -> T
    where
        F: FnOnce(&SomeMetadataThingy<'meta>) -> T,
    {
        let first = Bytes {
            bytes: &self.bytes[0..2],
        };
        let second = Bytes {
            bytes: &self.bytes[2..],
        };
        let parsed = SomeMetadataThingy {
            first: &first,
            second: &second,
        };
        f(&parsed)
    }
}

[vague]

Well, to understand the lifetime error here, let's put the trait impl aside and experiment on inherent impl:

Your trait impl is like this if for inherent impl:

impl<'bytes> Bytes<'bytes> {
    fn with_metadata<'meta, F, T>(&self, f: F) -> T
    where
        F: FnOnce(&SomeMetadataThingy<'meta>) -> T,
    {
        let first = Bytes {
            bytes: &self.bytes[0..2],
        };
        let second = Bytes {
            bytes: &self.bytes[2..],
        };
        let parsed = SomeMetadataThingy {
            first: &first,
            second: &second,
        };
        f(&parsed)
    }
}

so same error as yours.

Note there is an outer lifetime 'meta defined on method or impl block, but for the real usage f(&parsed), the value parsed comes from inside the function, which conflicts and thus causes the error. To express the trait bound of f(&parsed), F: FnOnce(&SomeMetadataThingy<'meta>) -> T, shoud become F: FnOnce(&SomeMetadataThingy<'_>) -> T, or more explicit F: for<'meta> FnOnce(&SomeMetadataThingy<'meta>) -> T, in HRTB syntax.

Then let's try it: Rust Playground

impl<'bytes> Bytes<'bytes> {
    fn with_metadata<'meta, F, T>(&self, f: F) -> T
    where
        F: FnOnce(&SomeMetadataThingy<'_>) -> T,
        // '_ is the intersection of &first and &second
        // which comes from the inner of the function,
        // thus you can't write it on the function or impl definition
        // where outer lifetime is defined.
    {
        let first = Bytes {
            bytes: &self.bytes[0..2],
        };
        let second = Bytes {
            bytes: &self.bytes[2..],
        };
        let parsed = SomeMetadataThingy {
            first: &first,
            second: &second,
        };
        f(&parsed)
    }
}

It compiles! And apply it to trait impl:

impl<'bytes> WithMetadata<SomeMetadataThingy<'_>> for Bytes<'bytes> {
    fn with_metadata<F, T>(&self, f: F) -> T
    where
        F: FnOnce(&SomeMetadataThingy<'_>) -> T,
    {
        let first = Bytes {
            bytes: &self.bytes[0..2],
        };
        let second = Bytes {
            bytes: &self.bytes[2..],
        };
        let parsed = SomeMetadataThingy {
            first: &first,
            second: &second,
        };
        f(&parsed)
    }
}


error[E0276]: impl has stricter requirements than trait
  --> src/lib.rs:68:12
   |
8  | /     fn with_metadata<F, T>(&self, f: F) -> T
9  | |     where
10 | |         F: FnOnce(&M) -> T;
   | |___________________________- definition of `with_metadata` from trait
...
68 |           F: FnOnce(&SomeMetadataThingy<'_>) -> T,
   |              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ impl has extra requirement `for<'a, 'b> F: FnOnce(&'a SomeMetadataThingy<'b>)`

Unfortunately, another error. It says the current trait bound conflicts with that from the defining trait.

So you have to change the definition of the trait to say I want a possible lifetime or maybe a lifetime placeholder to let me write it in bounds.

trait WithMetadata {
    type M<'maybe_lifetime>;
    // Invoke the closure with a parsed metadata object.
    fn with_metadata<F, T>(&self, f: F) -> T
    where
        F: for<'any_maybe_lifetime> FnOnce(&Self::M<'any_maybe_lifetime>) -> T;
}

impl<'bytes> WithMetadata for Bytes<'bytes> {
    type M<'a> = SomeMetadataThingy<'a>;
    fn with_metadata<F, T>(&self, f: F) -> T
    where
        F: FnOnce(&SomeMetadataThingy<'_>) -> T,
    {
        let first = Bytes {
            bytes: &self.bytes[0..2],
        };
        let second = Bytes {
            bytes: &self.bytes[2..],
        };
        let parsed = SomeMetadataThingy {
            first: &first,
            second: &second,
        };
        f(&parsed)
    }
}

It works.

[gatesn]

Ah that makes a lot of sense, thank you. I was struggling to find the HRTB syntax that would let me do that.

Unfortunately I can't now figure out how to invoke it! https://play.rust-lang.org/?version=stable&mode=debug&edition=2018&gist=299d11f3e5fe2d86fc600696cb8bfef9

--- Edit

Actually, I've realised that this works:

fn foo<D: for<'a> WithMetadata<Metadata<'a> = SomeMetadataThingy<'a>>>(d: D) -> usize {
    d.with_metadata(|m| m.first.len())
}

But this doesn't:

fn foo<'a, D: WithMetadata<Metadata<'a> = SomeMetadataThingy<'a>>>(d: D) -> usize {
    d.with_metadata(|m| m.first.len())
}

[vague]

-fn foo<'a, D: WithMetadata<Metadata<'a> = SomeMetadataThingy<'a>>>(d: D) -> usize {
+fn foo<D: for<'a> WithMetadata<Metadata<'a> = SomeMetadataThingy<'a>>>(d: D) -> usize {

Yeah, you want an inner lifetime, not an outer one!