With `static input: Borrowed value does not live long enough

2

The 'static annotations are out place here; I suspect you might have been tempted or even driven into adding those because of other requirements :slightly_smiling_face: (maybe the stream! macro? actually it's the Uri::from_static which lead you down that path)

Let's go over these things, shall we?

A "short" course / some remainders about lifetimes

  1. I'll often be referring to the lifetimes as regions, spans of code or of control-flow. Indeed, the key word in lifetime is "time", as a span of time / code execution; the "life" part is rather misleading.

  2. These regions are mainly focused or identified by the point at which they end.

    In that regard, 'static is a special lifetime which represents the never-ending region of code.

On top of that, lifetimes, in Rust, are used to express two related and yet different notions:

  • on the one hand, they can represent an area of code during which something, the referee or borrowee, is borrowed. Since 'static represents the never-ending region, if you happend to have a &'static Something, it means that some instance of type Something is being borrowed —in a shared fashion, since it's not &mut— forever / can remain borrowed for as long as the program goes.

    And while something is borrowed, it cannot be moved or deallocated, so a &'static borrow is always synonym of non-deallocated storage:

    • either because it was embedded-within-the-binary / global storage, historically called static storage (hence the name of the lifetime),

    • or because some runtime-allocated storage, such as some heap memory, is never getting freed (the memory is then said to have been leaked)

    Needless to say, the latter is extremely rare, and the former can only work for hard-coded data, in practice, things such as string literals.

    • In that regard, it is a rather acceptable approximation, at least when starting with Rust, to consider that &'static str is a special type referring to (hard-coded) string literals. More on that below.

    Be it as it may, once we have a &'borrow Thing, one of the great things about Rust is that it will know that you shouldn't be using such as reference beyond the end of (span of the) 'borrow. Now, this is easy to say and express "informally" about the type &'borrow Thing, but what about more complex types which carry such a reference? Say, for instance, an Option<&'borrow Thing>, a Vec<&'borrow Thing>, or a pair of such borrows, that is, a (&'borrow Thing, &'borrow Thing). Neither of the (instances of) these types can safely be used beyond the end of 'borrow. And while being able to talk about such a property informally is not that problematic when dealing with concrete types, things get a bit more hairy when dealing with polymorphism, that is, with generic types or with dyn Trait type erasure. In either case, some APIs may require that they be given stuff that can be used within regions of code where they might be using this stuff.

  • And that's the second aspect of lifetimes / spans of code: in generic (or dyn) contexts, they can be used to express that some generic (or dyn-type-erased) type can safely be used within certain regions of code. This is when you'll see things like T : 'region or impl 'region + … (or dyn 'region + …).

    A typical example of this, and a very recurring one, is when dealing with multi-threaded executions, such as thread::spawn, or the async executors which will poll futures potentially in background threads. In that case, compile-time / static analysis, cannot know whether one thread necessarily stops and .join()s over another thread's execution: the threads are conservatively viewed as detached. And this has an impact w.r.t. what I was saying above:

    some APIs may require that they be given stuff that can be used within regions of code where they might be using this stuff

    In this instance, these multi-threaded APIs, such as thread::spawn or most things that have to deal with spawn()ing Futures, will require that they be given stuff that will be accessed by a background thread that may be executed arbitrarily late: if they are given some reference with an expiry date, then, if the code in that thread is run past that expiry date, they would be using stale / dangling data, which is Very Bad. So, to avoid Very Bad stuff from happening, the only conservative solution is to be given things without an expiry date, that is, stuff that can be used within a never-ending region of code: the 'static region / duration. Such APIs will thus come with Param : 'static requirements, or, equivalently, with impl 'static requirements.

    And indeed, if you look at ::std::thread::spawn, for instance, you'll see that it features bounds such as:

    F : Send + 'static

    And with Futures and Streams this is something similar: ::tokio::spawn, for instance, requires that it be given a (top-level) Future (i.e., a task) that has to uphold that T : Future + 'static, i.e., T must be usable within the whole 'static region, that none of the captured environment may dangle at any point.

    And that's why you may encounter such requirements here and there.

And these two aspects, duration of a borrow, area of (owned-)usability of some object are related: if an object holds a borrow, then it cannot be used beyond the duration of the borrow.

If we go back to one of your functions and look at its signature, we observe a very interesting "textbook" example of this phenomenon:

What you are saying here is that thed function is taking a Client<String> which happens to be borrowed for some region / "duration" called 'a, and you are saying that what you are returning, an impl Stream + 'a is thus something, let's call it Ret, which verifies that Ret : Stream + 'a, i.e., Ret : Stream (the returned thing implements the Stream trait / interface / contract), and Ret : 'a (the returned thing, if owned, can be used within the region 'a without anything dangling).

  • Renaming lifetimes for increased readability

    Aside: I personally like to rename the lifetime parameter so that its name matches that of the function parameter it is used on: this often improves the readability of the code / makes it clearer to know what is going on:

    fn make_stream<'client> (
    client: &'client Client<String>,
) -> impl futures::Stream + 'client

Imagining that the returned Future is indeed a state machine that captures that client: &'a Client<String>, we indeed know that client can be used during the region 'a. Funnily enough, that's the maximal region within which client can be used: with the borrow checker at their side, rustaceans can live on the edge!

  • Anecdotically, one can even start expressing more complex signatures which are still right. Indeed, let's consider a second region, let's call it 'b, which happens to be contained within the region 'a / a region 'b which is known not to end after 'a ends. Mathematically, we could write this as 'a ⊇ 'b, or 'a ≥ 'b. In Rust, this is written as 'a : 'b.

    With such a region, we can say that the return value is guaranteed not to dangle within 'b:

    fn make_stream<'a, 'b> (
    client: &'a Client<String>,
) -> impl Stream + 'b
where
    'a : 'b,

    And this is correct / compiles (modulo the implementation / the function body): since &'a Client<String> can be used within the region 'a, and since the region 'b is smaller than (or equal to) the region 'a, then &'a Client<String> can safely be used within the region 'b as well.

Application: let's solve your issue

As I mentioned in the previous part, types such as &'static Something are almost never seen, with the most notable exception of &'static str, which is "only" (almost always) seen for hard-coded string literals.

If we look at your code, you are being given a 'a-limited borrow to a Client<String>, that is, a client which seems to be hold runtime / heap-allocated strings ≠ string literals. And given the outer &'a borrow on the client, you're only gonna be having &'a access to those Strings at best: &'a String.

You start off client.addresses.iter(), that is, you (re)borrow &client.addresses for some region that I shall call 'iter (a region which cannot span beyond 'a, i.e., a region which is upper-bounded by 'a, i.e., we have 'a : 'iter (similar to the 'b example above)), and call .iter() on it, to get an iterator which shall yield references over its elements which are thus borrowed for the duration 'iter. You then .cycle() over the iterator, and .take() at most client.count elements, nothing of which affects the type or the lifetime of the yielded elements, &'iter String.

But, in that last step you call the .cloned(), which shall call Clone::clone on each and every element of the iterator. The signature of Clone is:

impl Clone for String {
    fn clone (&self) -> String;
// i.e.
    fn clone (self: &String) -> String;
// i.e.
    fn clone (self: &'_ String) -> String;
// i.e.
    fn clone<'clone> (self: &'clone String) -> String;
    // for a duration `'clone` which can be as short as that of
    // the `.clone()` call, since `'clone` does not appear in the
    // return type.
}

Be it as it may, we thus end up with it which is a (lazy) iterator —that is borrowing from the client and which is therefore bound by the lifetime 'a— which will be able to produce String elements (from cloning the &'iter String ones).

You then go and eagerly collect such Strings into a Vec:

(so that, at this point, you already have a Vec<String>),

and then, quite surprisingly, you go and call .to_vec(), which is an operation which is conceptually like kind of a Clone for slices. It is not incorrect per se (the Vec<String> you had just collected will be borrowed for a short-time, duplicated as per .to_vec() semantics, and then, since nobody cares about this intermediate value (called a temporary), it will be thrown away, while you keep the (redundant) copy into vec.

From there, you seem to try to construct a v: Vec<&'static str> out of your vec: Vec<String>. I hope that you realize how:

  • that is impossible (except if leaking memory), since vec contains runtime / heap-allocated Strings, and v is expected to hold "string literals".

  • not necessary for whatever happens later on: even if the Streams or whatever asks you to have : 'static / impl 'static captures, it just so happens that vec : Vec<String> does meet that requirement: indeed, if you own a Vec<String> (which is a growable array of heap-allocated elements, where each element is itself a String, that is, a growable array of heap-string contents (String is kind of like a Vec<char>, but with a more efficient / compact representation (UTF-8 encoded Vec<u8>)), then you'll never have any dangling whatsoever: you own the Vec buffer, and you own each and every String involved. If you'll never have anything dangling whatsoever, then that's the very definition of being (owned-)usable within the never-ending 'static region / duration. Another way of seeing this is that Vec<String> features no &'smth borrows, nor Foo<'smth> generic structs (both of which would be upper-bounded by 'smth). And with no upper-bounds, sky 'static is the limit!

Now, the only limiting thing is the ::hyper::Uri::from_static operation you have there: indeed, it's a rather niche API which requires it be given an ever-lasting borrow over string contents, in practice it expects to be given a string literal. And, again, that's not what you have.

What you'll need here is to, instead, use another one of Uri constructors. In this instance, the proper constructor for this use case is a bit hidden: it's the FromStr trait, which is the one which provides .parse() capabilities: impl FromStr for Uri.

    let it = client.addresses.iter().cycle().take(client.count).cloned();
    let vec = it.collect::<Vec<String>>().to_vec();
-   let v: Vec<&'static str> = vec.iter().map(std::ops::Deref::deref).collect::<Vec<&'static str>>().to_vec();
    let strm = async_stream::stream! {
        for i in 0..client.count {
            let query_start = tokio::time::Instant::now();
-           let response = client.session.get(hyper::Uri::from_static(v[i]).unwrap()).await.expect("Hyper response");
+           let response = client.session.get(v[i].parse::<hyper::Uri>().unwrap()).await.expect("Hyper response");

Finally, once you get your code to compile and work in the "happy path" (e.g., correctly parseable urls), you'll want to refactor a bit the code and use try_stream! instead to be using ? to propagate failures upwards in a controlled manner, and at some point have a caller handle the failure to inform the user about it, in a graceful manner :slightly_smiling_face:

4 Likes