How to use HRTBs to specify lifetime of return value

When return value is a simple reference, it works fine:

struct Closure<F> {
    data: Vec<u8>,
    func: F
}

impl<F> Closure<F>
where
    F: for<'a> Fn(&'a [u8]) -> &'a [u8]
{
    fn call(&self) -> &[u8] {
        (self.func)(&self.data)
    }
}

However when the return type is a custom type, it comes out an error:

trait Packet<'a> {
    fn new(data: &'a [u8]) -> Self;
    /* some other functions */
}

struct TcpPacket<'a> {
    data: &'a [u8],
}

impl<'a> Packet<'a> for TcpPacket<'a> {
    fn new(data: &'a [u8]) -> Self {
        Self { data }
    }
}

struct Closure<F> {
    data: Vec<u8>,
    func: F
}

impl<F, P> Closure<F>
where
    F: for<'a> Fn(&'a [u8]) -> P,
    P: for<'a> Packet<'a>,
{
    fn call(&self) -> P {
        (self.func)(&self.data)
    }
}

fn do_it<F, P>(data: Vec<u8>, func: F)
where
    F: for<'a> Fn(&'a [u8]) -> P,
    P: for<'a> Packet<'a>,
{
    let closure = Closure { data, func };
    closure.call();
}

fn main() {
    do_it(vec![1, 2, 3], |buf| TcpPacket::new(buf));
}

The error message is "error: implementation of Packet is not general enough".

This is because I cannot specify the lifetime of Packet is the same as &'a [u8]. But I don't know how to do this.

Generic types (like your P) can only represent a single type, and types that differ by lifetime (even if they only differ by lifetime) are still distinct types. That's the source of the error; for any given concrete input lifetime, the type of closures you want output different types.

And unfortunately, the Fn traits make you name the output on stable.

Rust doesn't have generic type constructors so far, which would let you do something like

// Made up syntax
impl<F, P<*>> Closure<F>
where
    F: for<'a> Fn(&'a [u8]) -> P<'a>,
    P: for<'a> Packet<'a>,

Instead, you can mirror the Fn trait with a trait that doesn't force you to name output (and apply any other bounds you would like).

trait PacketFn<'a, _LifetimeGuard = &'a Self>: Fn(&'a [u8]) -> Self::Out {
    type Out: Packet<'a>;
}

impl<'a, F, Out> PacketFn<'a> for F
where
    F: Fn(&'a [u8]) -> Out,
    Out: Packet<'a>,
{
    type Out = Out;
}
impl<F> Closure<F>
where
    F: for<'a> PacketFn<'a>,
{
    fn call(&self) -> <F as PacketFn<'_>>::Out {
        (self.func)(&self.data)
    }
}

fn do_it<F>(data: Vec<u8>, func: F)
where
    F: for<'a> PacketFn<'a>,
{ /* ... */ }

Unfortunately, using another trait like this interferes with the bounds ability to influence the compiler's interpretation of the closure, and instead the compiler's poor inference around higher-ranked closures kicks in. That's what this additional bit I added is about:

fn nudge_closure<F: Fn(&[u8]) -> TcpPacket<'_>>(f: F) -> F { f }

Without it, the compiler fails to figure out that it should accept any input lifetime and that it should propagate that lifetime to the return type. There are other ways around the first problem, but this is the best general and stable way around the second problem that I'm aware of.

5 Likes

Thanks a lot. I came out a new problem in a simpler scene:

trait Packet<'a> {
    fn new(data: &'a [u8]) -> Self;
    fn print(&self);
    /* some other functions */
}

struct TcpPacket<'a> {
    data: &'a [u8],
}

impl<'a> Packet<'a> for TcpPacket<'a> {
    fn new(data: &'a [u8]) -> Self {
        Self { data }
    }
    fn print(&self) {
        println!("{:?}", self.data);
    }
}

fn do_it<P>(len: usize)
where
    P: for<'a> Packet<'a>
{
    let data = vec![0; len];
    let packet = P::new(&data);
    packet.print();
}

fn main() {
    do_it::<TcpPacket>(10);
}

The compiler still raise en error "implementation of Packet is not general enough" this time. I found a similar issue in https://github.com/rust-lang/rust/issues/70263, but that issue is just about closure. Is there any workaround for my code?

I don't know why do_it::<TcpPacket>(10); doesn't work.

But there is another way by moving the lifetime parameter on Packet to GAT instead:

trait Packet {
    type Out<'a>: Packet;
    fn new(data: &[u8]) -> Self::Out<'_>;
    fn print(&self);
}

struct TcpPacket<'a> {
    data: &'a [u8],
}

impl Packet for TcpPacket<'_> {
    type Out<'a> = TcpPacket<'a>;
    fn new(data: &[u8]) -> Self::Out<'_> {
        TcpPacket { data }
    }
    fn print(&self) {
        println!("{:?}", self.data);
    }
}

fn do_it<P>(len: usize)
where
    P: Packet,
{
    let data = vec![0; len];
    let packet = P::new(&data);
    packet.print();
}

fn main() {
    do_it::<TcpPacket>(10);
}

And for your OP, based on @quinedot 's solution, you can keep on

    trait Callback<'a>: Fn(&'a [u8]) -> Self::P {
        type P: Packet<Out<'a> = Self::P>;
    }
    impl<'a, P, F> Callback<'a> for F
    where
        P: Packet<Out<'a> = P>,
        F: Fn(&'a [u8]) -> P,
    {
        type P = P;
    }

Rust Playground

2 Likes

To explain the error without a GAT...

That's more explicitly

fn main() {
    do_it::<TcpPacket<'_>>(10);
}

Where the compiler will infer an appropriate (single) lifetime, if possible.

And there is no lifetime for the '_ which can satisfy the bound. Like generic type parameters, the thing in the turbofish must be a single type, not a type constructor.

4 Likes

FWIW, there are way to express this in somewhat convoluted ways (as showcased in your own snippet), and with helper crates such as:

there is even a somewhat acceptable level of sugar:

fn do_it<F, P : ForLt>(data: Vec<u8>, func: F)
where
    F : Fn(&'_ [u8]) -> P::Of<'_>,
    for<'a>
        P::Of<'a> : Packet<'a>
    ,
{
    let closure = Closure { data, func };
    closure.call::<P>();
}

fn main()
{
    //   main ergonomic convenience of the crate
    //         vvvvvvvvvvvvvvvvvvvv
    do_it::<_, ForLt!(TcpPacket<'_>)>( // Note: when doing these things, you "lose" type inference and need to be explicit.
        vec![1, 2, 3],
        |buf| TcpPacket::new(buf),
    );
}

Aside

Thanks @vague and @jofas for the mention of my documentation link problem, I'll try to update it when I have the time (I'm not answering in a standalone post to avoid further diverting this thread)

4 Likes

@Yandros Hi, the inline link on ForLifetime is broken,
https://docs.rs/higher-kinded-types/0.2.0-rc1/higher-kinded-types/trait.ForLifetime.html redirects to
https://docs.rs/higher-kinded-types/0.2.0-rc1/higher_kinded_types/?search=trait.ForLifetime.html

-https://docs.rs/higher-kinded-types/0.2.0-rc1/higher-kinded-types/trait.ForLifetime.html
should be
+https://docs.rs/higher-kinded-types/0.2.0-rc1/higher_kinded_types/trait.ForLifetime.html
or
+https://docs.rs/higher-kinded-types/latest/higher_kinded_types/trait.ForLifetime.html
2 Likes

Took me a while to find which link you've meant. Funnily enough, if you call the link from docs.rs it works just fine, but if you call it from the README.md, it doesn't work.

1 Like

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.