Why an type MyType<'a> = Arc<dyn Fn(&'a u8)> needs to have a lifetime parameter

help
1 
use std::sync::Arc;

pub struct EncodedPacket<'a>{
    packet: &'a[u8]
}

pub type OnConsume<'a> = Arc<dyn Fn(&'a u8) -> Option<EncodedPacket<'a>> + Send + Sync>;

pub struct HoldsOnConsume<'a> {
    on_consume: OnConsume<'a>
}

fn main() {
    
}

As you see here, OnConsume is a function that produces an EncodedPacket with certain lifetime. However, the lifetime is not intrinsic to the object that holds the function. The closure that is then coerced into dyn Fn(&'a u8) -> Option<EncodedPacket<'a>> + Send + Sync doesn't necessairly store the &'a u8 it receives. However, I'm forced to put <'a> on the type OnConsume, which then pollutes the HoldsOnConsume<'a> with a lifetime parameter, making it not implementing Any, which is needed for downcasting, which is what I'm trying to do.

Is there a way to create an OnConsume that does not have a lifetime, since it does not store any reference, it just processes a reference?

2

This is the primary use of Higher-rank trait bounds (HRTBs). You can use the clause for<'a> to introduce a new lifetime name 'a which refers to a lifetime that will only be known at the callsite:

use std::sync::Arc;

pub struct EncodedPacket<'a>{
    packet: &'a[u8]
}

pub type OnConsume = Arc<dyn for<'a> Fn(&'a u8) -> Option<EncodedPacket<'a>> + Send + Sync>;

pub struct HoldsOnConsume {
    on_consume: OnConsume
}

fn main() {
    
}
2 Likes
3

What if I want a function that produces an EncodedPacket either by creating one inside of it or returning a moved one?

For example, this:

use std::sync::Arc;

pub struct EncodedPacket<'a>{
    packet: &'a[u8]
}

pub type OnConsume = Arc<dyn for<'a> Fn() -> Option<EncodedPacket<'a>> + Send + Sync>;

pub struct HoldsOnConsume {
    on_consume: OnConsume
}

fn main() {
    
}

won't work.

error[E0582]: binding for associated type `Output` references lifetime `'a`, which does not appear in the trait input types
 --> src/main.rs:7:46
  |
7 | pub type OnConsume = Arc<dyn for<'a> Fn() -> Option<EncodedPacket<'a>> + Send + Sync>;
  |                                              ^^^^^^^^^^^^^^^^^^^^^^^^^

The idea was to have a function like this:

Arc::new(move ||{
    EncodedPacket{}
})

or one that produces an EncodedPacket that was moved to its inside:

Arc::new(move ||{
    encoded_packet
})
4

For this you should use 'static, since that's the only kind of EncodedPacket it can return without an input parameter to borrow from.

pub type OnConsume = Arc<dyn Fn() -> Option<EncodedPacket<'static>> + Send + Sync>;

This cannot work with Fn. From the std docs:

Instances of Fn can be called repeatedly without mutating state.

But you cannot repeatedly move a non-Copy value:

struct NonCopyStruct;

fn make_fn() -> impl Fn() -> NonCopyStruct {
    let s = NonCopyStruct;
    move || s
}

error[E0507]: cannot move out of `s`, a captured variable in an `Fn` closure
 --> src/main.rs:5:13
  |
4 |     let s = NonCopyStruct;
  |         - captured outer variable
5 |     move || s
  |             ^ move occurs because `s` has type `NonCopyStruct`, which does not implement the `Copy` trait

In the simplified example above, changing Fn to FnOnce works, since there's no problem if the closure is only called once. I'm not sure there's any way to apply that to OnConsume though.

1 Like
5

unfortunately my EncodedPacket example is not complete, it can be cloned. This is a more realistic example:

#[derive(Clone)]
pub enum EncodedPacket<'a> {
    Owned(Vec<u8>),
    Ref(Rc<&'a [u8]>),
    RefMut(Rc<RefCell<&'a mut [u8]>>)
}

the clone here is just so I can reuse it in here:

Arc::new(move ||{
    encoded_packet.clone()
})

So the question is, can I make a type OnConsume that returns an EncodedPacket with references, such that OnConsume has no lifetime parameters?

Something close to

pub type OnConsume = Arc<dyn for<'a> Fn() -> Option<EncodedPacket<'a>> + Send + Sync>;
6

here's one attempt I did at modifying the Arc so at least I can continue

use std::sync::Arc;

pub struct EncodedPacket<'a>{
    packet: &'a[u8]
}

pub type OnConsume = Arc<dyn for<'a> Fn(&'a mut Option<EncodedPacket<'a>>) + Send + Sync>;

fn main() {
    let packet = vec![1,2,3];
    let on_consume = Arc::new(move |e: &mut Option<EncodedPacket>|{
        *e = Some(EncodedPacket{
            packet: packet.as_slice()
        });
    });
    let mut encoded_packet = None;
    on_consume(&mut encoded_packet);
}

Playground

but this also cannot be done

error[E0495]: cannot infer an appropriate lifetime for autoref due to conflicting requirements
  --> src/main.rs:13:28
   |
13 |             packet: packet.as_slice()
   |                            ^^^^^^^^
   |
note: first, the lifetime cannot outlive the lifetime `'_` as defined on the body at 11:31...
  --> src/main.rs:11:31
   |
11 |     let on_consume = Arc::new(move |e: &mut Option<EncodedPacket>|{
   |                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
note: ...so that closure can access `packet`
  --> src/main.rs:13:21
   |
13 |             packet: packet.as_slice()
   |                     ^^^^^^
note: but, the lifetime must be valid for the anonymous lifetime #2 defined on the body at 11:31...
  --> src/main.rs:11:31
   |
11 |       let on_consume = Arc::new(move |e: &mut Option<EncodedPacket>|{
   |  _______________________________^
12 | |         *e = Some(EncodedPacket{
13 | |             packet: packet.as_slice()
14 | |         });
15 | |     });
   | |_____^
note: ...so that the expression is assignable
  --> src/main.rs:12:14
   |
12 |           *e = Some(EncodedPacket{
   |  ______________^
13 | |             packet: packet.as_slice()
14 | |         });
   | |__________^
   = note: expected `Option<EncodedPacket<'_>>`
              found `Option<EncodedPacket<'_>>`
7

Still, even with the cloning case, the only possible lifetime this function can use is 'static.

Lifetimes are breadcrumbs for answering the question of "where this has been borrowed from". If a function doesn't have access to any data to borrow via its arguments, then there's nothing to trace borrows back to, so the only possible lifetime remaining is 'static, which means leaked memory or compile-time constants that can be borrowed from globals.

9

are you talking about my latest try or the original one? Because on the latest, the function has access to a lifetime argument, so it should know that the &mut Option<EncodedPacket> passed to it has a short lifetime

10

Oops, the earlier.

11

what do you think of my new approach? I guess there's stil lthe issue of the captured packet though

12

First, sharing any lifetime with &mut is going to be super problematic, because &mut is "invariant", which means it won't be shortened to match another lifetime. It's maximally inflexible when matching other borrow lifetimes (it has to for safety in less obvious cases), so often putting the same mut lifetime in multiple places makes impossible chicken-egg problems. So I've split for<'a, 'b> to use different lifetimes for the mutable Option and shared packet.

But the real problem here is:

note: closure implements Fn, so references to captured variables can't escape the closure

I've only made the compiler say it explicitly by adding #![feature(nll)] (I thought we'd be at #![feature(polonius)] by now, but apparently not).

Here's the minimal version:

fn main() {
    let packet = vec![1,2,3];
    let packet_closure = move |leak: &mut &[i32]| { *leak = packet.as_slice(); };
}

I know it's actually unsafe for FnMut closures, because a reference leaked in one function call could be invalidated by a second function call. I don't know why it's forbidden for Fn though.

13

this is very nice, but I was trying to avoid a lifetime parameter on OnConsume because then this lifetime will polute the struct that holds OnConsume, making it not implement Any, which I need for downcasting :woozy_face:

14

Can you make it a trait with a method that uses &self? Then a call to obj.get_packet() would borrow the object for an easy-to-define lifetime. I think the problem may be that calling Fn() doesn't explicitly borrow it, so you can't express that lifetime of the packet depends on the non-borrowed lifetime of the closure.

15

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.