Lifetimes and borrowing between closure and parent struct

help
1

Can someone help me please with to shift my mindset around rust borrowing concept?
I have a scenario in which I create a service and this service have a router which accepts closures. I can pass closure or a function without problems but I wanna be able to access parent service from that closure (read only for now). Example below

impl Service<Router> {
pub fn new() -> Self {
let mut service= Self {
router: Router::new().boxed(),
worker: Arc::new(worker) // I actually need to be able to access worker but being able to access parent service might be even better
};
...
let w1 = Arc::clone(&service.worker);
...
// Here, I would get an error that w1 would not leave long enough, but the service would be static and live through entire application. How can I do such thing proper "rust way"? Any article advice?
let router = router.route("/1",handler::get(move || w1.handler("/1")));
service
}
}

Thanks!

2

It will be easier for people to help you if they understand the code in question: it is thus highly advisable to write it within code blocks for the forum to keep the whitespace and highlight the code:

How to write it: 
```
impl Service<Router> {
    pub fn new() -> Self {
        let mut service = Self {
            router: Router::new().boxed(),
            worker: Arc::new(worker), // I actually need to be able to access
                                      // worker but being able to access parent
                                      // service might be even better
        };
        /* ... */
        let w1 = Arc::clone(&service.worker);
        /* ... */
        // Here, I would get an error that w1 would not leave long enough, but
        // the service would be static and live through entire application. How
        // can I do such thing proper "rust way"? Any article advice?
        let router = router.route("/1", handler::get(move || w1.handler("/1")));
        service
    }
}
```

▼ How the forum renders it:

impl Service<Router> {
    pub fn new() -> Self {
        let mut service = Self {
            router: Router::new().boxed(),
            worker: Arc::new(worker), // I actually need to be able to access
                                      // worker but being able to access parent
                                      // service might be even better
        };
        /* ... */
        let w1 = Arc::clone(&service.worker);
        /* ... */
        // Here, I would get an error that w1 would not leave long enough, but
        // the service would be static and live through entire application. How
        // can I do such thing proper "rust way"? Any article advice?
        let router = router.route("/1", handler::get(move || w1.handler("/1")));
        service
    }
}
4 Likes
3

Now, regarding:

That is a bit surprising, since you are giving ownership of that owned Arc clone, w1, to the closure given to handler::get. And contrary to the usual short-lived borrow &'borrow Referee, which can be used, at most, within the 'borrow region, a Arc<Referee> can, at most, be used within the 'static / ever-lasting region, provided Referee itself be (usable-for-)'static as well.

So I suspect the worker itself is a variable which holds its own &'borrow … which makes it short-lived: once something holds a short-lived borrow, it gets "infected" with that (short) lifetime (parameter), and no matter how much you wrap it, it will remain so. So you may also need to hold interior Arcs (or sync::Weaks to avoid cycles) all the way down to be able not to lose the (usable-for-)'static property.

2 Likes
4

Oh, sorry about that. I copy/pasted forgetting to format it. Thanks for adding formatted version.

5

The worker is empty right now:

pub struct Worker {
    wasm_modules: Arc<CHashMap<String, WasmModule>>
}


impl Worker {
    pub fn new() -> Self {
        let wasm_modules: Arc<CHashMap<String, WasmModule>> = Arc::new(CHashMap::new());

        Self {
            wasm_modules
        }
    }

    pub fn add_module(&mut self, path: &str, module: WasmModule) {
        self.wasm_modules.insert(path.to_string(), module);
    }

    pub fn handler(&self, path: &str) -> BoxFuture<Html<Vec<u8>>> {
        let path = path.to_owned();
        Box::pin(async move {
            Html(format!("<h1>Worker::path - {}</h1>", path).as_bytes().to_owned())
        })
    }
}

And exact error:

lifetime may not live long enough
closure implements `Fn`, so references to captured variables can't escape the closurerustc
worker_service.rs(36, 53): lifetime `'1` represents this closure's body
worker_service.rs(36, 59): return type of closure is Pin<Box<(dyn futures::Future<Output = Html<Vec<u8>>> + std::marker::Send + '2)>>
1 Like
6

@Yandros thanks for your help! I actually resolved one of the error and it had nothing to do with lifetime. I am still really confused with lengthy rust error reports. One of the issues is that I had to use Box::pin for async block inside of the method (It seems async methods are not supported yet)

So, I needed to turn w1.handle("/1") into || { w1.handle("/1").await }

1 Like
7

Indeed. Btw, you can use

to have a macro do it for you.

Regarding the error and the workaround, I missed the fact that handler::get was expecting:

  1. not only a (usable-for-)'static closure (Fn…() -> …)
  2. but also a closure that would return a (usable-for-)'static Future when called.

And that's the part that was failing on your code:

unsugared, it gives:

fn handler<'borrowed_worker> (
    self: &'borrowed_worker Worker,
    path: &'_ str,
) -> BoxFuture<'borrowed_worker, Html<Vec<u8>>> {
    let path = path.to_owned(); // `path: String`, so `path: impl 'static`
    Box::pin(async move /* path */ {
          //       ^^^^^^^^^^^^^^^-------<------+
        … // mentions `path` but not `self`: >--+
    }) // `: Pin<Box<impl Future + 'static>>`
    // coerced as:
    //    `: Pin<Box<dyn Future + 'borrowed_worker>> == BoxFuture<'borrowed_worker, …>`
}

So all that to say that:

  • the implementation of handler was indeed 'static since it was only capturing that owned "copy" / .to_owned() / strdup of path, and doing nothing about self.

  • but the signature the contract was conservatively restricting the API to returning a Future that could be capturing self / borrowing from *self, since those were the semantics of lifetime elision. Indeed, the BoxFuture<Ret> type alias is actually hiding a hidden lifetime parameter; it's actually a BoxFuture<'_, Ret>, and that elided '_ lifetime parameter unifies with that of the self parameter, hence the unsugaring.

    • If you find that these "hidden" lifetime parameters that may be lurking in type aliases are error-prone / footgunny, I very much agree with you! There is a way to opt into a lint that will warn against those by adding a #![warn(elided_lifetimes_in_paths)] at the root of the src/{lib,main}.rs file(s).

A final question now may be to understand

why, if w1.handler() is a non-'static future, is async move { w1.handler().await } a (usable-for-)'static future?

The answer is regarding the order of operations: a(n impl) Future represents suspended / lazy / not-yet-evaluated code.

  • when doing w1.handler(), you are first eagerly borrowing w1, and then yielding a Future which (may) borrow w1 / which captures that borrow of w1, which makes the returned Future not be 'static.

  • when doing async move { w1.handler().await }, you are first creating an async / future suspension point, which captures w1 in an owned and thus 'static (because w1 is 'static) fashion. That Future is thus 'static. Inside the suspended code of the Future, that owned w1 will be borrowed to yield that internally borrowed sub-future (the w1.handler()), and then that sub-future shall be .awaited. But the body of a Future does not (directly) affect whether it's 'static or not, only its captures / upvars.

    • (Implementation-wise, the first future was (potentially / API-wise) referencing an "outer" upvar, w1, which made it non-'static, but on the plus side it made the returned future be non-self-referential / made it Unpin. Whereas the second future's trick is to perform the borrow internally, in a self-referential manner (non-Unpin), which is how it manages to be usable for 'static without danger).
3 Likes
8

Oh, thanks! I did not understand quite a few stuff from your explanation, but some awesome hints!

  1. By using 'borrowed_worker lifetime I was able to remove path.to_owned() by using same lifetime.
  2. It was not intuitively clear when I was reading in docs to use 'a as a lifetime but your example ''borrowed_worker' is awesome! It tells me that whatever I am using inside of the method it is coupled with worker lifetime. Hard to explain the difference between rust docs but much more intuitive. Thanks!

I will take a look into async_trait

9

Exactly. That's one of the important things that need to "click" in, since it's really the core mechanism of lifetimes at the API level: connecting a borrow with the return value (when appropriate), while keeping short-lived borrows disconnected from the return value to let Rust understand these things at the call sites.

There is then also the aspect of "lifetime as an area / region of owned-usability", which can be a bit more subtle. I've just written a post somewhere where I've made a special effort to try and go over things from scratch: With `static input: Borrowed value does not live long enough - #2 by Yandros

10

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.