The 'static
annotations are out place here; I suspect you might have been tempted or even driven into adding those because of other requirements
(maybe the stream!
macro? actually it's the Uri::from_static
which lead you down that path)
Let's go over these things, shall we?
A "short" course / some remainders about lifetimes
-
I'll often be referring to the lifetimes as regions, spans of code or of control-flow. Indeed, the key word in lifetime is "time", as a span of time / code execution; the "life" part is rather misleading.
-
These regions are mainly focused or identified by the point at which they end.
In that regard, 'static
is a special lifetime which represents the never-ending region of code.
On top of that, lifetimes, in Rust, are used to express two related and yet different notions:
-
on the one hand, they can represent an area of code during which something, the referee or borrowee, is borrowed. Since 'static
represents the never-ending region, if you happend to have a &'static Something
, it means that some instance of type Something
is being borrowed βin a shared fashion, since it's not &mut
β forever / can remain borrowed for as long as the program goes.
And while something is borrowed, it cannot be moved or deallocated, so a &'static
borrow is always synonym of non-deallocated storage:
-
either because it was embedded-within-the-binary / global storage, historically called static storage (hence the name of the lifetime),
-
or because some runtime-allocated storage, such as some heap memory, is never getting freed (the memory is then said to have been leaked)
Needless to say, the latter is extremely rare, and the former can only work for hard-coded data, in practice, things such as str
ing literals.
- In that regard, it is a rather acceptable approximation, at least when starting with Rust, to consider that
&'static str
is a special type referring to (hard-coded) string literals. More on that below.
Be it as it may, once we have a &'borrow Thing
, one of the great things about Rust is that it will know that you shouldn't be using such as reference beyond the end of (span of the) 'borrow
. Now, this is easy to say and express "informally" about the type &'borrow Thing
, but what about more complex types which carry such a reference? Say, for instance, an Option<&'borrow Thing>
, a Vec<&'borrow Thing>
, or a pair of such borrows, that is, a (&'borrow Thing, &'borrow Thing)
. Neither of the (instances of) these types can safely be used beyond the end of 'borrow
. And while being able to talk about such a property informally is not that problematic when dealing with concrete types, things get a bit more hairy when dealing with polymorphism, that is, with generic types or with dyn Trait
type erasure. In either case, some APIs may require that they be given stuff that can be used within regions of code where they might be using this stuff.
-
And that's the second aspect of lifetimes / spans of code: in generic (or dyn
) contexts, they can be used to express that some generic (or dyn
-type-erased) type can safely be used within certain regions of code. This is when you'll see things like T : 'region
or impl 'region + β¦
(or dyn 'region + β¦
).
A typical example of this, and a very recurring one, is when dealing with multi-threaded executions, such as thread::spawn
, or the async
executors which will poll futures potentially in background threads. In that case, compile-time / static analysis, cannot know whether one thread necessarily stops and .join()
s over another thread's execution: the threads are conservatively viewed as detached. And this has an impact w.r.t. what I was saying above:
some APIs may require that they be given stuff that can be used within regions of code where they might be using this stuff
In this instance, these multi-threaded APIs, such as thread::spawn
or most things that have to deal with spawn()
ing Future
s, will require that they be given stuff that will be accessed by a background thread that may be executed arbitrarily late: if they are given some reference with an expiry date, then, if the code in that thread is run past that expiry date, they would be using stale / dangling data, which is Very Bad. So, to avoid Very Bad stuff from happening, the only conservative solution is to be given things without an expiry date, that is, stuff that can be used within a never-ending region of code: the 'static
region / duration. Such APIs will thus come with Param : 'static
requirements, or, equivalently, with impl 'static
requirements.
And indeed, if you look at ::std::thread::spawn
, for instance, you'll see that it features bounds such as:
F : Send + 'static
And with Future
s and Stream
s this is something similar: ::tokio::spawn
, for instance, requires that it be given a (top-level) Future
(i.e., a task) that has to uphold that T : Future + 'static
, i.e., T
must be usable within the whole 'static
region, that none of the captured environment may dangle at any point.
And that's why you may encounter such requirements here and there.
And these two aspects, duration of a borrow, area of (owned-)usability of some object are related: if an object holds a borrow, then it cannot be used beyond the duration of the borrow.
If we go back to one of your functions and look at its signature, we observe a very interesting "textbook" example of this phenomenon:
What you are saying here is that thed function is taking a Client<String>
which happens to be borrowed for some region / "duration" called 'a
, and you are saying that what you are returning, an impl Stream + 'a
is thus something, let's call it Ret
, which verifies that Ret : Stream + 'a
, i.e., Ret : Stream
(the returned thing implements the Stream
trait / interface / contract), and Ret : 'a
(the returned thing, if owned, can be used within the region 'a
without anything dangling).
-
Renaming lifetimes for increased readability
Aside: I personally like to rename the lifetime parameter so that its name matches that of the function parameter it is used on: this often improves the readability of the code / makes it clearer to know what is going on:
fn make_stream<'client> (
client: &'client Client<String>,
) -> impl futures::Stream + 'client
Imagining that the returned Future
is indeed a state machine that captures that client: &'a Client<String>
, we indeed know that client
can be used during the region 'a
. Funnily enough, that's the maximal region within which client
can be used: with the borrow checker at their side, rustaceans can live on the edge!
-
Anecdotically, one can even start expressing more complex signatures which are still right. Indeed, let's consider a second region, let's call it 'b
, which happens to be contained within the region 'a
/ a region 'b
which is known not to end after 'a
ends. Mathematically, we could write this as 'a β 'b
, or 'a β₯ 'b
. In Rust, this is written as 'a : 'b
.
With such a region, we can say that the return value is guaranteed not to dangle within 'b
:
fn make_stream<'a, 'b> (
client: &'a Client<String>,
) -> impl Stream + 'b
where
'a : 'b,
And this is correct / compiles (modulo the implementation / the function body): since &'a Client<String>
can be used within the region 'a
, and since the region 'b
is smaller than (or equal to) the region 'a
, then &'a Client<String>
can safely be used within the region 'b
as well.
Application: let's solve your issue
As I mentioned in the previous part, types such as &'static Something
are almost never seen, with the most notable exception of &'static str
, which is "only" (almost always) seen for hard-coded string literals.
If we look at your code, you are being given a 'a
-limited borrow to a Client<String>
, that is, a client which seems to be holding runtime / heap-allocated strings β string literals. And given the outer &'a
borrow on the client
, you're only gonna be having &'a
access to those String
s at best: &'a String
.
You start off client.addresses.iter()
, that is, you (re)borrow &client.addresses
for some region that I shall call 'iter
(a region which cannot span beyond 'a
, i.e., a region which is upper-bounded by 'a
, i.e., we have 'a : 'iter
(similar to the 'b
example above)), and call .iter()
on it, to get an iterator which shall yield references over its elements which are thus borrowed for the duration 'iter
. You then .cycle()
over the iterator, and .take()
at most client.count
elements, nothing of which affects the type or the lifetime of the yielded elements, &'iter String
.
But, in that last step you call the .cloned()
, which shall call Clone::clone
on each and every element of the iterator. The signature of Clone
is:
impl Clone for String {
fn clone (&self) -> String;
// i.e.
fn clone (self: &String) -> String;
// i.e.
fn clone (self: &'_ String) -> String;
// i.e.
fn clone<'clone> (self: &'clone String) -> String;
// for a duration `'clone` which can be as short as that of
// the `.clone()` call, since `'clone` does not appear in the
// return type.
}
Be it as it may, we thus end up with it
which is a (lazy) iterator βthat is borrowing from the client
and which is therefore bound by the lifetime 'a
β which will be able to produce String
elements (from cloning the &'iter String
ones).
You then go and eagerly collect such String
s into a Vec
:
(so that, at this point, you already have a Vec<String>
),
and then, quite surprisingly, you go and call .to_vec()
, which is an operation which is conceptually like kind of a Clone
for slices. It is not incorrect per se (the Vec<String>
you had just collected will be borrowed for a short-time, duplicated as per .to_vec()
semantics, and then, since nobody cares about this intermediate value (called a temporary), it will be thrown away, while you keep the (redundant) copy into vec
.
From there, you seem to try to construct a v: Vec<&'static str>
out of your vec: Vec<String>
. I hope that you realize how:
-
that is impossible (except if leaking memory), since vec
contains runtime / heap-allocated String
s, and v
is expected to hold "string literals".
-
not necessary for whatever happens later on: even if the Stream
s or whatever asks you to have : 'static
/ impl 'static
captures, it just so happens that vec : Vec<String>
does meet that requirement: indeed, if you own a Vec<String>
(which is a growable array of heap-allocated elements, where each element is itself a String
, that is, a growable array of heap-str
ing contents (String
is kind of like a Vec<char>
, but with a more efficient / compact representation (UTF-8 encoded Vec<u8>
)), then you'll never have any dangling whatsoever: you own the Vec
buffer, and you own each and every String
involved. If you'll never have anything dangling whatsoever, then that's the very definition of being (owned-)usable within the never-ending 'static
region / duration. Another way of seeing this is that Vec<String>
features no &'smth
borrows, nor Foo<'smth>
generic structs (both of which would be upper-bounded by 'smth
). And with no upper-bounds, sky 'static
is the limit!
Now, the only limiting thing is the ::hyper::Uri::from_static
operation you have there: indeed, it's a rather niche API which requires it be given an ever-lasting borrow over string contents, in practice it expects to be given a string literal. And, again, that's not what you have.
What you'll need here is to, instead, use another one of Uri
constructors. In this instance, the proper constructor for this use case is a bit hidden: it's the FromStr
trait, which is the one which provides .parse()
capabilities: impl FromStr for Uri
.
let it = client.addresses.iter().cycle().take(client.count).cloned();
let vec = it.collect::<Vec<String>>().to_vec();
- let v: Vec<&'static str> = vec.iter().map(std::ops::Deref::deref).collect::<Vec<&'static str>>().to_vec();
let strm = async_stream::stream! {
for i in 0..client.count {
let query_start = tokio::time::Instant::now();
- let response = client.session.get(hyper::Uri::from_static(v[i]).unwrap()).await.expect("Hyper response");
+ let response = client.session.get(v[i].parse::<hyper::Uri>().unwrap()).await.expect("Hyper response");
Finally, once you get your code to compile and work in the "happy path" (e.g., correctly parseable urls), you'll want to refactor a bit the code and use try_stream!
instead to be using ?
to propagate failures upwards in a controlled manner, and at some point have a caller handle the failure to inform the user about it, in a graceful manner 