Lifetimes and function

    /// Stores the `signal` inside the internal timed-queue for `implicated_cid`, and then sends `packet` to `target_cid`.
    /// After `timeout`, the closure `on_timeout` is executed
    #[inline]
    pub fn route_primary<'a, 'b: 'a, 'c: 'b>(&'c mut self, implicated_cid: u64, target_cid: u64, ticket: Ticket, signal: PeerSignal, packet: impl Fn(&'b Rc<PostQuantumContainer>, &'b Drill) -> Bytes + 'a, timeout: Duration, on_timeout: impl Fn(PeerSignal) + 'static) -> Result<(), String> {
        if self.account_manager.hyperlan_cid_is_registered(target_cid) {
            // get the target cid's session
            if let Some(sess) = self.sessions.get(&target_cid) {
                if self.hypernode_peer_layer.insert_tracked_posting(implicated_cid, timeout, ticket, signal, on_timeout) {
                    let sess_ref = sess.inner.borrow();
                    let target_pqc = sess_ref.post_quantum.as_ref().unwrap();
                    let peer_sender = sess_ref.to_primary_stream.as_ref().unwrap();
                    let peer_cnac = sess_ref.cnac.as_ref().unwrap();

                    if let Some(ref peer_latest_drill) = peer_cnac.get_drill_blocking(None) {
                        log::info!("Routing packet through primary stream ({} -> {})", implicated_cid, target_cid);
                        let packet = packet(target_pqc, peer_latest_drill );
                        let _drill = peer_latest_drill;
                        peer_sender.unbounded_send(packet).map_err(|err| err.to_string());
                    } else {
                        return Err(format!("Unable to acquire peer drill for {}", target_cid))
                    }

                    return Ok(())
                } else {
                    Err(format!("Unable to insert tracked posting for {}", implicated_cid))
                }
            } else {
                // session is not active, but user is registered (thus offline). Deliver to mailbox
                if self.hypernode_peer_layer.try_add_mailbox(true, target_cid, signal) {
                    Ok(())
                } else {
                    Err(format!("Peer {} is offline. Furthermore, that peer's mailbox is not accepting signals at this time", target_cid))
                }
            }
        } else {
            Err(format!("CID {} is not registered locally", target_cid))
        }

    }

Error:

error[E0597]: `sess_ref` does not live long enough
   --> hyxe_net/src/hdp/hdp_session_manager.rs:449:38
    |
443 |     pub fn route_primary<'a, 'b: 'a, 'c: 'b>(&'c mut self, implicated_cid: u64, target_cid: u64, ticket: Ticket, signal: PeerSignal, packet: impl Fn(&'b Rc<PostQuantumContainer>, &'b Drill) -> Bytes + 'a, timeout: Duration, on_timeout: impl Fn(PeerSignal) + 'static) -> Result<(), String> {
    |                              -- lifetime `'b` defined here
...
449 |                     let target_pqc = sess_ref.post_quantum.as_ref().unwrap();
    |                                      ^^^^^^^^ borrowed value does not live long enough
...
455 |                         let packet = packet(target_pqc, peer_latest_drill );
    |                                      -------------------------------------- argument requires that `sess_ref` is borrowed for `'b`
...
463 |                 } else {
    |                 - `sess_ref` dropped here while still borrowed

error[E0716]: temporary value dropped while borrowed
   --> hyxe_net/src/hdp/hdp_session_manager.rs:453:58
    |
443 |     pub fn route_primary<'a, 'b: 'a, 'c: 'b>(&'c mut self, implicated_cid: u64, target_cid: u64, ticket: Ticket, signal: PeerSignal, packet: impl Fn(&'b Rc<PostQuantumContainer>, &'b Drill) -> Bytes + 'a, timeout: Duration, on_timeout: impl Fn(PeerSignal) + 'static) -> Result<(), String> {
    |                              -- lifetime `'b` defined here
...
453 |                     if let Some(ref peer_latest_drill) = peer_cnac.get_drill_blocking(None) {
    |                                                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ creates a temporary which is freed while still in use
454 |                         log::info!("Routing packet through primary stream ({} -> {})", implicated_cid, target_cid);
455 |                         let packet = packet(target_pqc, peer_latest_drill );
    |                                      -------------------------------------- argument requires that borrow lasts for `'b`
...
460 |                     }
    |                     - temporary value is freed at the end of this statement

I want the closure packet to be called in order to obtain a Bytes object, and from there, send it through the unbounded sender. However, when I try this I get the lifetime error shown above

My thinking (please correct me):

I set Bytes to lifetime 'a because it is the shortest-lived. It gets sent through send_unbounded and is thus memcopied through the channel mechanism (thus preserving its existence after the end of closure with lifetime 'b and 'c).

I set the packet closure to lifetime 'b because it must last longer than 'a, but last shorter than 'c.

I set self to lifetime 'c because that has the longest lifetime.

Writing this out, I do see a developing problem in this logic: packet gets pushed into the closure's stack frame, and thus may be correlated with lifetime 'c (it is created in the same closure as self). Still not sure how to resolve the problem though

What crate are you using that provides send_unbounded? Mainly what’s the function signature for that call?

Actually I wonder if one of the errors looks like it may be because you’ve got the same lifetime on both arguments to packet requiring them to be the same. What happens if you just remove all the trait bounds from the signature and let the compiler put them in?

For example, this raises the same error
Playground Link

Whereas this compiles with lifetimes removed
Playground Link

Obviously I made up a bunch of types just to get something to compile, so its possible that they are incorrect in a way that invalidates the solution.

1 Like

I could be way off base here, but it seems to me that your reasoning is flawed because you are thinking about the lifetimes of objects rather than the lifetimes of borrows. Lifetime parameters have to do with how long something is borrowed for, not how long it lives altogether.

The + 'a lifetime is applied to impl Fn..., not Bytes. Perhaps it would be more clear to write it the other way around: packet: impl 'a + Fn(&'b Rc<PostQuantumContainer>, &'b Drill) -> Bytes. (I'm not sure what Bytes is but if it's a struct with a lifetime parameter, maybe you meant Bytes<'a>.)

Hmm. 'b is how long the packet closure borrows its arguments for. So whatever you pass to packet has to be derived from a reference of lifetime at least 'b, where the actual lifetime 'b is chosen by the caller. But it looks to me like the actual arguments of packet, target_pqc and peer_latest_drill, are actually borrowed from sess, which is a local variable. It can't have a lifetime of 'b because 'b is a lifetime in the caller, and sess only lives inside route_primary itself. The errors you see are consistent with that theory.

It doesn't matter that the Rcs being referenced may be long-lived after you're done with them. What matters is how long packet borrows them for, and in this case (I assume) it only borrows them for a portion of route_primary. So maybe what you want is to say that packet can borrow a thing for any lifetime, something like impl for<'b> Fn(&'b Rc<PostQuantumContainer>, &'b Drill) -> Bytes (which is actually just the same as impl Fn(&Rc<PostQuantumContainer, &Drill) -> Bytes, unless I'm overlooking something).

It's also not necessarily the case that because self is a long-lived object that it should also be borrowed for the longest lifetime. I can't tell from looking at the code you've posted but I'd also try to start without 'c and add it back only if it solves a problem.

2 Likes

Interesting. I thought I would need HKTB for my solution. But then I removed the lifetime annotations and it worked. This would explain why. Though it also means my reasoning is probably wrong.

Though just for my understanding, wouldn’t it expand to something more like for <‘b1,’b2> Fn(&’b1 arg1, &’b2 arg2) -> res

Thanks guys for your replies. So, I went ahead and removed the lifetime parameters. It compiles, but when I add-in the calling-closure below:

                    let packet = |peer_pqc, peer_drill| {
                        // Here, we route the packet to the target_cid
                        hdp_packet_crafter::peer_cmd::craft_peer_signal(peer_pqc, peer_drill, PeerSignal::PostRegister(peer_conn_type), ticket, timestamp)
                    };

                    // Give the target_cid 10 seconds to respond
                    let res = sess_mgr.route_primary(implicated_cid, target_cid, ticket, signal, packet, Duration::from_secs(10), move |signal| {
                        // on timeout, run this
                        log::warn!("Running timeout closure. Sending error message to {}", implicated_cid);
                        let error_packet = hdp_packet_crafter::peer_cmd::craft_peer_signal(&pqc2, &drill2, signal, ticket, timestamp);
                        let _ = to_primary_stream.unbounded_send(error_packet);
                    });

I get this compile error:

error[E0631]: type mismatch in closure arguments
  --> hyxe_net/src/hdp/hdp_packet_processor/peer_cmd_packet.rs:60:40
   |
54 |                     let packet = |peer_pqc, peer_drill| {
   |                                  ---------------------- found signature of `fn(&std::rc::Rc<ez_pqcrypto::PostQuantumContainer>, &hyxe_crypt::drill::Drill) -> _`
...
60 |                     let res = sess_mgr.route_primary(implicated_cid, target_cid, ticket, signal, packet, Duration::from_secs(10), move |signal| {
   |                                        ^^^^^^^^^^^^^ expected signature of `for<'r, 's> fn(&'r std::rc::Rc<ez_pqcrypto::PostQuantumContainer>, &'s hyxe_crypt::drill::Drill) -> _`

error[E0271]: type mismatch resolving `for<'r, 's> <[closure@hyxe_net/src/hdp/hdp_packet_processor/peer_cmd_packet.rs:54:34: 57:22 peer_conn_type:_, ticket:_, timestamp:_] as std::ops::FnOnce<(&'r std::rc::Rc<ez_pqcrypto::PostQuantumContainer>, &'s hyxe_crypt::drill::Drill)>>::Output == bytes::bytes::Bytes`
  --> hyxe_net/src/hdp/hdp_packet_processor/peer_cmd_packet.rs:60:40
   |
60 |                     let res = sess_mgr.route_primary(implicated_cid, target_cid, ticket, signal, packet, Duration::from_secs(10), move |signal| {
   |                                        ^^^^^^^^^^^^^ expected bound lifetime parameter, found concrete lifetime

error[E0631]: type mismatch in closure arguments
   --> hyxe_net/src/hdp/hdp_packet_processor/peer_cmd_packet.rs:100:40
    |
94  |                     let packet = |peer_pqc, peer_lastest_drill| {
    |                                  ------------------------------ found signature of `fn(&std::rc::Rc<ez_pqcrypto::PostQuantumContainer>, &hyxe_crypt::drill::Drill) -> _`
...
100 |                     let res = sess_mgr.route_primary(implicated_cid, target_cid, ticket, PeerSignal::PostConnect(peer_conn_type, None), packet, Duration::from_secs(10), move |peer_signal| {
    |                                        ^^^^^^^^^^^^^ expected signature of `for<'r, 's> fn(&'r std::rc::Rc<ez_pqcrypto::PostQuantumContainer>, &'s hyxe_crypt::drill::Drill) -> _`

error[E0271]: type mismatch resolving `for<'r, 's> <[closure@hyxe_net/src/hdp/hdp_packet_processor/peer_cmd_packet.rs:94:34: 97:22 peer_conn_type:_, ticket:_, timestamp:_] as std::ops::FnOnce<(&'r std::rc::Rc<ez_pqcrypto::PostQuantumContainer>, &'s hyxe_crypt::drill::Drill)>>::Output == bytes::bytes::Bytes`
   --> hyxe_net/src/hdp/hdp_packet_processor/peer_cmd_packet.rs:100:40
    |
100 |                     let res = sess_mgr.route_primary(implicated_cid, target_cid, ticket, PeerSignal::PostConnect(peer_conn_type, None), packet, Duration::from_secs(10), move |peer_signal| {
    |                                        ^^^^^^^^^^^^^ expected bound lifetime parameter, found concrete lifetime

error: aborting due to 4 previous errors

Hmm, this works, but it requires heap-allocation:

  1. change function signature's packet type to:
packet: Box<dyn Fn(&Rc<PostQuantumContainer>, &Drill) -> Bytes>
  1. Declare the closure which gets input as packet as such:
                    let packet: Box<dyn Fn(&Rc<PostQuantumContainer>, &Drill) -> Bytes> = Box::new(move |peer_pqc, peer_lastest_drill| {
                        // Here, we route the packet to the target_cid
                        hdp_packet_crafter::peer_cmd::craft_peer_signal(peer_pqc, peer_lastest_drill, PeerSignal::PostConnect(peer_conn_type, None), ticket, timestamp)
                    });

However, heap-allocation is usually undesirable if it isn't necessary. Surely there's a better way?

Maybe try FnOnce instead of Fn for the argument type. It looks like you’re only calling the closure once and then the lifetimes of everything the closure captures don’t need to extend past that one call. Just a guess though.

1 Like

That's a good call. It is, indeed, only called once. I'll leave that change in there. The error persists:

error[E0631]: type mismatch in closure arguments
  --> hyxe_net/src/hdp/hdp_packet_processor/peer_cmd_packet.rs:60:40
   |
54 |                     let packet = move |peer_pqc, peer_drill| {
   |                                  --------------------------- found signature of `fn(&std::rc::Rc<ez_pqcrypto::PostQuantumContainer>, &hyxe_crypt::drill::Drill) -> _`
...
60 |                     let res = sess_mgr.route_primary(implicated_cid, target_cid, ticket, signal, packet, Duration::from_secs(10), move |signal| {
   |                                        ^^^^^^^^^^^^^ expected signature of `for<'r, 's> fn(&'r std::rc::Rc<ez_pqcrypto::PostQuantumContainer>, &'s hyxe_crypt::drill::Drill) -> _`

error[E0271]: type mismatch resolving `for<'r, 's> <[closure@hyxe_net/src/hdp/hdp_packet_processor/peer_cmd_packet.rs:54:34: 57:22 peer_conn_type:_, ticket:_, timestamp:_] as std::ops::FnOnce<(&'r std::rc::Rc<ez_pqcrypto::PostQuantumContainer>, &'s hyxe_crypt::drill::Drill)>>::Output == bytes::bytes::Bytes`
  --> hyxe_net/src/hdp/hdp_packet_processor/peer_cmd_packet.rs:60:40
   |
60 |                     let res = sess_mgr.route_primary(implicated_cid, target_cid, ticket, signal, packet, Duration::from_secs(10), move |signal| {
   |                                        ^^^^^^^^^^^^^ expected bound lifetime parameter, found concrete lifetime

error[E0631]: type mismatch in closure arguments
   --> hyxe_net/src/hdp/hdp_packet_processor/peer_cmd_packet.rs:100:40
    |
94  |                     let packet = move |peer_pqc, peer_lastest_drill| {
    |                                  ----------------------------------- found signature of `fn(&std::rc::Rc<ez_pqcrypto::PostQuantumContainer>, &hyxe_crypt::drill::Drill) -> _`
...
100 |                     let res = sess_mgr.route_primary(implicated_cid, target_cid, ticket, PeerSignal::PostConnect(peer_conn_type, None), packet, Duration::from_secs(10), move |peer_signal| {
    |                                        ^^^^^^^^^^^^^ expected signature of `for<'r, 's> fn(&'r std::rc::Rc<ez_pqcrypto::PostQuantumContainer>, &'s hyxe_crypt::drill::Drill) -> _`

error[E0271]: type mismatch resolving `for<'r, 's> <[closure@hyxe_net/src/hdp/hdp_packet_processor/peer_cmd_packet.rs:94:34: 97:22 peer_conn_type:_, ticket:_, timestamp:_] as std::ops::FnOnce<(&'r std::rc::Rc<ez_pqcrypto::PostQuantumContainer>, &'s hyxe_crypt::drill::Drill)>>::Output == bytes::bytes::Bytes`
   --> hyxe_net/src/hdp/hdp_packet_processor/peer_cmd_packet.rs:100:40
    |
100 |                     let res = sess_mgr.route_primary(implicated_cid, target_cid, ticket, PeerSignal::PostConnect(peer_conn_type, None), packet, Duration::from_secs(10), move |peer_signal| {
    |                                        ^^^^^^^^^^^^^ expected bound lifetime parameter, found concrete lifetime

error: aborting due to 4 previous errors

For more information about this error, try `rustc --explain E0271`.
error: could not compile `hyxe_net`.

K. Think I figured it out. Don't store the closure in a variable. Pass it directly to the function call. From Issue 41078

In the comment from @eddyb

Generally you should avoid putting closures in variables, if you're only going to pass them to a function, because placing the closure expression nested in the call expression lets the compiler deduce a better signature.

Though I would read his whole comment with the one before it since it explains what looks like the same situation in more detail.

4 Likes

Wow. 6 hours of pulling my hair out (almost bald now), and you saved me from losing my prom date entirely. Thank you!

If res does not contain any lifetimes, that's equivalent to the one-lifetime version:

for<'b> Fn(&'b arg1, &'b arg2) -> res

because if the caller can choose lifetimes 'b1 and 'b2 that are both valid at the time of calling the closure, then it can simply choose the single lifetime 'b to be the intersection of 'b1 and 'b2.

If res is a type with lifetimes in it, you might need to be more specific because lifetime parameters tell the compiler the relationships between input and output lifetimes. So these three all mean the same thing:

Fn(&i32, &i32) -> i32                    // no lifetime
for<'a> Fn(&'a i32, &'a i32) -> i32      // one lifetime
for<'a, 'b> Fn(&'a i32, &'b i32) -> i32  // two lifetimes

But these three all mean different things:

for<'a> Fn(&'a i32, &'a i32) -> &'a i32      // the output borrows from both inputs
for<'a, 'b> Fn(&'a i32, &'b i32) -> &'a i32  // the output borrows only from the first input
for<'a, 'b> Fn(&'a i32, &'b i32) -> &'b i32  // the output borrows only from the second input
2 Likes

Thanks @trentj. That makes perfect sense.

Glad I could help. That’s not often the case with lifetimes and closures.

1 Like

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.