use std::sync::Arc;
use tokio::task;
pub trait Object<'a>: 'a {
fn to_bytes(&self) -> Vec<u8>;
fn from_bytes(bytes: &'a [u8]) -> Self;
}
struct MyKey<'a> {
data: &'a [u8],
}
impl<'a> Object<'a> for MyKey<'a> {
fn to_bytes(&self) -> Vec<u8> {
self.data.to_vec()
}
fn from_bytes(bytes: &'a [u8]) -> MyKey<'a> {
MyKey { data: bytes }
}
}
struct MyValue<'b> {
data: &'b [u8],
}
impl<'b> Object<'b> for MyValue<'b> {
fn to_bytes(&self) -> Vec<u8> {
self.data.to_vec()
}
fn from_bytes(bytes: &'b [u8]) -> MyValue<'b> {
MyValue { data: bytes }
}
}
#[tokio::main]
async fn main() {
let key_data: Vec<u8> = vec![107, 101, 121]; // "key" in bytes
let value_data: Vec<u8> = vec![118, 97, 108, 117, 101]; // "value" in bytes
let key_data_arc = Arc::from(key_data.into_boxed_slice());
let value_data_arc = Arc::from(value_data.into_boxed_slice());
let key = Arc::new(MyKey::from_bytes(&key_data_arc));
let value = Arc::new(MyValue::from_bytes(&value_data_arc));
let handle = {
let key = key.clone();
let value = value.clone();
let key_data_arc = key_data_arc.clone();
let value_data_arc = value_data_arc.clone();
task::spawn(async move {
println!("{:?}", key.to_bytes());
println!("{:?}", value.to_bytes());
println!("{:?}", key_data_arc);
println!("{:?}", value_data_arc);
})
};
handle.await.unwrap();
}
error[E0597]: `key_data_arc` does not live long enough
--> examples/test.rs:45:42
|
42 | let key_data_arc = Arc::from(key_data.into_boxed_slice());
| ------------ binding `key_data_arc` declared here
...
45 | let key = Arc::new(MyKey::from_bytes(&key_data_arc));
| ------------------^^^^^^^^^^^^^-
| | |
| | borrowed value does not live long enough
| argument requires that `key_data_arc` is borrowed for `'static`
...
61 | }
| - `key_data_arc` dropped here while still borrowed
Context:
The code is very simplified to demonstrate the error. I understand that I can move the arc of the data and call from_bytes inside async move. But I need this kind of pattern in a real project (passing byte slices to the async function). Tried chatgpt-4o and it kept returning the same code using arc to prolong the data lifetime, which failed to compile.
Questions:
No matter how I prolong the lifetime of the underlying data (key_data and value_data), the compiler asked for the 'static lifetime of the argument of from_bytes. I did not find any safety concern here. I want to understand the reason mechanically.
How to fix this? I do not want to leak the underlying data since I may use the data and its viewing struct in a short lifetime frequently. I need to deallocate the data.
You can clone the data into something not borrowed, or you could seek some async solution that doesn't require 'static. (Other people on this forum are better equipped to address potential tokio-specific alternatives than I.)
as far as I know, tokio doesn't support this use cases, at least not out of the box. if you are ok to use other async runtimes, smol can support scoped async tasks by using an explicit Executor::spwan:
just FYI: I have read good words about moro but I never used it myself. you may try it if you cannot use smol for your application.
if I understand it correctly, moro, unlike smol or async-std, is not an async runtime itself, and allows to combine non-'static futures in a scoped/structured manner into large futures/tasks, and it is compatible to different async eosystems, inlcuding tokio.
Passing a &[u8] (or any kind of borrowed slice) to tokio::spawn is likely to never work.
Always pass owned data. In your small example this can be done by calling to_bytes() before the tokio::spawn, thus moving an owned Vec<u8> into the task as opposed to a MyKey<'a>.
Arc doesn't prolong lifetimes. It allows to share data between multiple threads without using lifetimes, but to do that you have to replace the use of references with Arc and not wrap them in an Arc (like you were trying to do). That is, replace MyKey's &[u8] field with an Arc<[u8]>/Arc<Vec<u8>>/bytes::Bytes
It cannot be guaranteed that you'll call .await on the handle, and if you don't do that then the task could execute after the main function has returned, when the slice references will become invalid due to the Vecs being dropped.
Root of your problem may be in misuse of temporary references in structs. Your struct definition means that MyValue is forbidden from storing the data. There's no data in this struct, and the struct itself becomes a temporary view into some other place, like a variable inside a function, that actually holds the data. This is a very restrictive situation, and these restriction cannot be undone. You can't fix it with Arc or anything else, and you can't extend these lifetimes. There is no workaround for them, by design. They are meant to reliably infect and restrict everything that touches them, to prevent use-after-free statically.
Such restrictive design is sometimes useful and in rare cases necessary, but in 99% of cases it's a design mistake and an anti-pattern, caused by mistaking Rust's references with using data "by reference" in other languages. Rust's references are not for storing data "by reference", and their primary purpose isn't even avoiding copying. They're for not owning, which has very specific, and quite restrictive meaning in Rust. Those references (loans) are not even the only reference type in Rust — there's Box, Rc, Arc, and most containers like Vec store their data by reference too, without any & in sight. In structs almost always other reference types are more appropriate.
You're more likely to be correct if you never ever put any references in any structs, ever, than if you try to make all those lifetimes annotations work.
Passing a &[u8] (or any kind of borrowed slice) to tokio::spawn is likely to never work.
It is not true. The code works correctly if I leak key_data_arc and value_data_arc (Box::leak(key_data.into_boxed_slice())), which means the borrowed slice could be used in async threads. Runnable code:
Arc doesn't prolong lifetimes. It allows to share data between multiple threads without using lifetimes, but to do that you have to replace the use of references with Arc and not wrap them in an Arc (like you were trying to do). That is, replace MyKey 's &[u8] field with an Arc<[u8]> /Arc<Vec<u8>> /bytes::Bytes
It cannot be guaranteed that you'll call .await on the handle, and if you don't do that then the task could execute after the main function has returned, when the slice references will become invalid due to the Vec s being dropped.
Even if the main exits first, the async thread still owns the vector (key_data and value_data) via key_data_arc and value_data_arc which outlives key and value here.
Technically, it is 100% avoidable to pass such kind of view struct across async-io/multithreads (just pass the arc of the owned data). However, using such kind of view struct could improve the code readability and neatness by minimizing the data visibility in the interfaces. Imo, using non-static reference in struct should not be discouraged to ease the coding pattern.