For several years I've been writing code in Rust. I thought I had the understanding of how the hardest part of Rust works - lifetimes. But developing complex projects with a large number of files and modules, sometimes I was faced with unresolvable compilation errors. The reason for these errors was lifetimes checking. At such moments, I thought deeply and decided that I didn't understand this topic well enough. Well, can't it be that complicated? I'll give a specific example. I tried to simplify my real case as much as possible and leave the gist.
I want to make a generic loader that loads json files and caches parsed structs in self
so as not to download the same files again. Since this is already responsible for loading, why not store a buffer inside it, into which files will be read? So the struct
will be
struct Loader<A> {
buf: String,
vec: Vec<A>,
}
Since I'm going to read and parse files, I need the error type
#[derive(Debug)]
enum Error {
Io(std::io::Error),
Serde(serde_json::Error),
}
I'll write the load
method. It has no data caching yet, just downloading
impl<A> Loader<A> {
fn load<'a, L, T>(&mut self, name: &str, loader: L) -> Result<A, Error>
where
// High level function to abstract some conversion logic `T` -> `A`
L: FnOnce(T) -> Result<A, Error>,
T: Deserialize<'a>,
// Remark:
// I do not write this explicitly, but I am sure that
A: 'static,
// This is important because I need to get the `Result` without any borrowing
{
let buf = &mut self.buf; // <- Compile error here
buf.clear();
// Build path and read json from file here
buf.push_str(r#"{ "x": 1.0, "y": 2.0 }"#);
serde_json::from_str(buf.as_str())
.map_err(Error::Serde)
.and_then(loader)
}
}
Compilation results in an error as the compiler cannot infer an appropriate lifetimes. And it's pretty easy to understand the reason if you look at the serde_json::from_str
definition
pub fn from_str<'a, T>(s: &'a str) -> Result<T>
where
T: de::Deserialize<'a>,
// ..
Ok. It looks like I just need to specify my buffer lifetime. But the buffer is in self
, which means I need to specify the lifetime on the self
reference? I'll try
fn load<'a, L, T>(&'a mut self, name: &str, loader: L) -> Result<A, Error>
// ..
Now it works! For now..
Yes, this code works great, but it can deadlock whole next development. What if I want to write a method that calls load
internally? For example now I'll use the vec
fn load_and_save<'a, L, T>(&'a mut self, name: &str, loader: L) -> Result<(), Error>
where
L: FnOnce(T) -> Result<A, Error>,
T: Deserialize<'a>,
{
let a = self.load(name, loader)?;
self.vec.push(a);
Ok(())
}
Then I get the message "cannot borrow self.vec
as mutable more than once at a time".
(Playground)
But why? I just used another method, which returns a structure with static lifetime. For some reason, after calling this method, I can no longer use self
. My intuition about lifetimes has failed me
If I think deeply about this code, it seems that I'm missing something. Back to the load
method, why bother to specify a lifetime to its input arguments? In fact, I need to get any lifetimes at the input, because the output is only a structure with a static lifetime. Let's say if you simplify this signature, you get something like
fn to_string(s: &str) -> String
Something that borrows data and returns data by value. But the problem is I have to tell this to compiler somehow. But I can't just omit the 'a
parameter in
T: Deserialize<'a>,
For example like this
T: Deserialize<'_>,
// or
T: Deserialize,
In this case, the compiler swears and advises me to consider introducing a named lifetime parameter. If only it could be possible to create a temporary local parameter that would only apply to Deserialize
bound. Although it looks like I've seen something like this
T: for<'a> Deserialize<'a>,
Then the load
signature will be simply
fn load<L, T>(&mut self, name: &str, loader: L) -> Result<A, Error>
where
L: FnOnce(T) -> Result<A, Error>,
T: for<'a> Deserialize<'a>,
// ..
And it works great! It would solve all my problems if I knew about it earlier. And I think that not only mine. I'm sure that many people will find this feature useful, especially those who like to program in a functional style. So happened that I didn't know about it, only rarely saw it in random code snippets and didn't understand what it was and why it exists. Only in the Rustonomicon I found a short description about this "Higher-Rank Trait Bounds". But I haven't read Rustonomicon yet. I planned to read it only after I finally understand the safe Rust. Why is HRTB in the Rustonomicon if it's a regular code, without any unsafe line? Is it really some kind of secret knowledge that only those who know the true essence of Rust can learn? This is an invaluable useful thing that puts a lot of things in their places. Answers many questions and solves many everyday tasks. It's much better than writing &'a mut self
and trapping yourself without understanding what's going on. Perhaps it should be taught in beginner books and pay more attention to it? Or is it possible to teach compiler to advise this solution?
I admit I could easily have missed something. If there is a more suitable solution for this problem, then I would be glad to see it. I really understand those who find Rust is difficult. After all, lifetimes are really the hardest and most important topic in the language studying and HRTB will help clarify a lot.