Hi,
I am having a fun problem involving lifetimes, iterators and Arc, hope you can help me as I've tried every possible solution I could fine online. I am implementing a program that has support for pluggable key value stores, so I wrote a trait Database that looks something like this:
pub trait Database
where
Self: Sized + Send + Sync,
{
type Iterator: Iterator<Item = (Box<[u8]>, Box<[u8]>)>;
fn iterator(
&self,
start: Vec<u8>,
) -> Result<Self::Iterator>;
}
Then the main program wraps implementations of Database in an Arc and calls Database::iterator from other threads, for example:
pub struct MyProgram<T> {
pub db: Arc<T>
}
impl<T> MyProgram <T>
where
T: Store + 'static,
{
pub fn test_iterator(&self) {
let db = self.db.clone();
tokio::task::spawn_blocking(move || {
for (key, value) in db.iterator("test".to_vec()).unwrap() {
println!("{:?} {:?}", key, value);
}
});
}
}
So far everything looks good. The problem came when implementing RocksDB which requires a lifetime in the iterator:
impl Database for RocksDB {
type Iterator = DBIteratorWithThreadMode<'a, DBWithThreadMode<MultiThreaded>>;
fn iterator(
&self,
start: Vec<u8>,
) -> Result<Self::Iterator> {
// code goes here
}
}
Which of course does not compile as the lifetime 'a is not defined. So I tried multiple things:
- First I tried using PhantomData:
pub struct RocksDB<'a> {
db: DBWithThreadMode<MultiThreaded>,
_lifetime: PhantomData<&'a ()>,
}
impl<'a> Database for RocksDB<'a> {
type Iterator = DBIteratorWithThreadMode<'a, DBWithThreadMode<MultiThreaded>>;
fn iterator(
&self,
cf: store::ColumnFamily,
start: Vec<u8>,
direction: store::Direction,
) -> Result<DBIteratorWithThreadMode<'a, DBWithThreadMode<MultiThreaded>>> {
// Implementations goes here
}
}
Which does not work as the are conflicting lifetimes:
cannot infer an appropriate lifetime for autoref due to conflicting requirements
expected `std::result::Result<rocksdb::DBIteratorWithThreadMode<'a, _>, _>`
found `std::result::Result<rocksdb::DBIteratorWithThreadMode<'_, _>, _>`
- Then I tried implementing the trait on references to the struct:
impl<'a> Database for &'a RocksDB {
type Iterator = DBIteratorWithThreadMode<'a, DBWithThreadMode<MultiThreaded>>;
// implementations goes here
}
Which looks promising but then it is not possible to allocate MyProgram as it expects implementations of Database and in this case the trait is implemented on references. Not sure if there is a way to tell the compiler that Database is implemented on &T and not T.
- I also tried adding a lifetime parameter to self on the iterator function, which is not ideal as it infects all other functions and also causes other errors.
pub trait Database<'a>
where
Self: Sized + Send + Sync,
{
type Iterator: Iterator<Item = (Box<[u8]>, Box<[u8]>)>;
fn iterator(
&'a self,
start: Vec<u8>,
) -> Result<Self::Iterator>;
}
But this causes a different problem when calling db inside the spawned task:
let db = self.db.clone();
tokio::task::spawn_blocking(move || {
for (key, value) in db.iterator("test".to_vec()).unwrap() {
println!("{:?} {:?}", key, value);
}
});
Indicating that the reference inside Arc is dropped while still borrowed:
`db` does not live long enough
borrowed value does not live long enough rustc E0597
bitmap.rs(334, 9): `db` dropped here while still borrowed
bitmap.rs(210, 6): lifetime `'a` defined here
bitmap.rs(283, 29): argument requires that `db` is borrowed for `'a`
Do you have any ideas of how to make this work without relying on GAT?
Thank you!