Requesting help with saving data into Redb (lifetime problem)

help
1

With jofas's help
I solved reading data from Redb problem

I got another challenge: saving data into redb

this is my saving function
(I have to set V and KEY with the same lifetime, because table.insert needs the same lifetime args

pub(crate) fn write<'c, K, V, KEY, D>(table: TableDefinition<K, V>, key: KEY, value: &'c D) -> Result<()>
where
    K: redb::RedbKey,
    for<'a> V: redb::RedbValue<SelfType<'a> = &'a [u8]>,
    for<'a> KEY: Borrow<&'a str> + std::borrow::Borrow<<K as redb::RedbValue>::SelfType<'a>>,
    D: serde::Serialize,
{
    match serde_json::to_vec(value) {
        Ok(r) => {
            let write_txn = DB.begin_write()?;
            {
                let mut table = write_txn.open_table(table)?;
                table.insert(key.borrow(), r.as_slice())?;
            }
            write_txn.commit()?;
            Ok(())
        }
        Err(e) => Err(Error::ErrorWithMessage(format!("{:?}", e))),
    }
}

when I call this function

const SETTINGS_KEY: &str = "settings";
pub(crate) fn init() -> Result<()> {
    let settings = Settings::default();
    db::write(TABLE, SETTINGS_KEY, &settings)
}

I got this error

error: implementation of `Borrow` is not general enough
  --> src\man\settings.rs:39:5
   |
39 |     db::write(TABLE, SETTINGS_KEY, &settings)
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ implementation of `Borrow` is not general enough
   |
   = note: `&'2 str` must implement `Borrow<&'1 str>`, for any lifetime `'1`...
   = note: ...but it actually implements `Borrow<&'2 str>`, for some specific lifetime `'2`

I also tried this

pub(crate) fn write<'a, K, V, KEY, D>(table: TableDefinition<K, V>, key: KEY, value: &D) -> Result<()>
where
    K: redb::RedbKey,
    V: redb::RedbValue<SelfType<'a> = &'a [u8]>,
    KEY: Borrow<&'a str> + std::borrow::Borrow<<K as redb::RedbValue>::SelfType<'a>>,
    D: serde::Serialize,
{
    let d = serde_json::to_vec(value)?;
    let vv = d.as_slice();
    let write_txn = DB.begin_write()?;
    {
        let mut table = write_txn.open_table(table)?;
        table.insert(key, vv)?;
    }
    write_txn.commit()?;
    Ok(())
}

Still failed with

error[E0597]: `d` does not live long enough
   --> src\db\mod.rs:248:14
    |
239 | pub(crate) fn write<'a, K, V, KEY, D>(table: TableDefinition<K, V>, key: KEY, value: &D) -> Result<()>
    |                     -- lifetime `'a` defined here
...
247 |     let d = serde_json::to_vec(value)?;
    |         - binding `d` declared here
248 |     let vv = d.as_slice();
    |              ^^^^^^^^^^^^ borrowed value does not live long enough
...
253 |         table.insert(key, vv)?;
    |         --------------------- argument requires that `d` is borrowed for `'a`
...
264 | }
    | - `d` dropped here while still borrowed

I got what compiler said, the 'a will last whole process call, but d dropped at the end of the function
but I didn't understand that this line table.insert(key, vv)?;, vv had finished its job, why needs to be last as long as 'a

Thanks in advance

2

I tried without table: TableDefinition<K, V> parameter, the following code compiled

pub(crate) fn save<'a, D>(key: impl Borrow<&'a str>, value: &D) -> Result<()>
where
    D: serde::Serialize,
{
    match serde_json::to_vec(value) {
        Ok(r) => {
            // let db = Database::open(TABLE_FILE_NAME)?;
            let write_txn = DB.begin_write()?;
            {
                let mut table = write_txn.open_table(TABLE)?;
                table.insert(key.borrow(), r.as_slice())?;
            }
            write_txn.commit()?;
            Ok(())
        }
        Err(e) => Err(Error::ErrorWithMessage(format!("{:?}", e))),
    }
}

table.insert(key.borrow(), r.as_slice())?;
here needs key.borrown() instead of key, otherwise compiler says not the same lifetime

3

Please share full minimal code when asking for help.

If K or V is not generic, it won't be hard to write a working function:

pub(crate) fn write_non_generic<D>(
    table: TableDefinition<&str, &[u8]>,
    key: &str,
    value: &D,
    db: &Database,
) -> Result<()>
where
    D: serde::Serialize,

// or
pub(crate) fn write_key_fixed<V, D>(
    table: TableDefinition<&str, V>,
    key: &str,
    value: &D,
    db: &Database,
) -> Result<()>
where
    V: for<'a> RedbValue<SelfType<'a> = &'a [u8]>,
    D: serde::Serialize,

But it'd be hard to write a generic one you're wanting, because you have to think about borrows and ownership all the time. The hassle comes from this line

table.insert(key.borrow(), r.as_slice())?

// the sig of insert
// impl<'db, 'txn, K: RedbKey + 'static, V: RedbValue + 'static> Table<'db, 'txn, K, V>
pub fn insert<'a>(
    &mut self,
    key: impl Borrow<K::SelfType<'a>>,
    value: impl Borrow<V::SelfType<'a>>
) -> Result<Option<AccessGuard<'_, V>>, StorageError>
where
    K: 'a,
    V: 'a,

it just means you have to make both key and value valid at insert point, which is not an issue in non-generic code due to the borrow checker.

But in generic code, you have to write lifetime constrains by yourself which seems a part of the job of borrowck as in non-generic code. Consider your current code:

  • key is an argument from the fn signature, so it comes from the outer scope
  • r is an owned Vec<u8> generated in the function, and r.as_slice() surely comes from the inner scope
  • to meet the requirement from .insert, your write signature just expresses key and r can generate any reference but doesn't convey both the references can live at the same point: this is the missing bound and the info the error is showing

I don't come up with a good solution for now. But if you want to avoid the bound, owned type can help:

write_generic(TableDefinition::<&str, Bytes>::new(""), ...)?;
write_generic(TableDefinition::<(), Bytes>::new(""), (), &"", &db)?;

#[derive(Debug)]
struct Bytes(Vec<u8>); // tackle orphan rule
impl RedbValue for Bytes { ... }

pub(crate) fn write_generic<K, V, D>(
    table: TableDefinition<K, V>,
    key: K::SelfType<'_>,
    value: &D,
    db: &Database,
) -> Result<()>
where
    K: RedbKey,
    V: for<'a> RedbValue<SelfType<'a> = Vec<u8>>, // this line!
    D: serde::Serialize, {
    ...
    table.insert(key, r)?; // and this line!
    ...
}

rustexplorer

1 Like
4

Really appreciate your detailed response!

I have one question about this funtion:

pub(crate) fn write_key_fixed<V, D>(
    table: TableDefinition<&str, V>,
    key: &str,
    value: &D,
    db: &Database,
) -> Result<()>
where
    V: for<'a> RedbValue<SelfType<'a> = &'a [u8]>,
    D: serde::Serialize,
{
    match serde_json::to_vec(value) {
        Ok(r) => {
            let write_txn = db.begin_write()?;
            {
                let mut table = write_txn.open_table(table)?;
                table.insert(key, r.as_slice())?;
            }
            write_txn.commit()?;
            Ok(())
        }
        Err(e) => Err(anyhow::anyhow!("{e:?}")),
    }
}

Although defined table as TableDefinition<&str, V>,
but this line:V: for<'a> RedbValue<SelfType<'a> = &'a [u8]>, making V(the value of table) must be &[u8], right?

If I'm misunderstanding, please correct me

5

Yes, if you don't have a &[u8] wrapper type. So it's kinda useless in this case since that type bound comes from the function inside thus is fixed.

And if curious about the write_value_fixed case:

pub(crate) fn write_value_fixed<K, D>(
    table: TableDefinition<K, &[u8]>,
    key: impl for<'a> Borrow<K::SelfType<'a>>,
    value: &D,
    db: &Database,
) -> Result<()>
where
    K: RedbKey,
    D: serde::Serialize,

it works for an owned key, but causes the same problem in OP for borrowed key:

    // error: implementation of `Borrow` is not general enough
    write_value_fixed(TableDefinition::<&str, _>::new(""), "", &"", &db)?;
    
    // ok for owned type
    write_value_fixed(TableDefinition::<(), _>::new(""), (), &"", &db)?;

error: implementation of `Borrow` is not general enough
  --> src/main.rs:15:5
   |
15 |     write_value_fixed(TableDefinition::<&str, _>::new(""), "", &"", &db)?;
   |     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ implementation of `Borrow` is not general enough
   |
   = note: `&'2 str` must implement `Borrow<&'1 str>`, for any lifetime `'1`...
   = note: ...but it actually implements `Borrow<&'2 str>`, for some specific lifetime `'2`
6

Thanks!

I guess in my case, it's very hard or impossible to write a general version of TableDefinition<K, V> that accept TableDefinition<&str, &str> or TableDefinition<&str, &[u8]> or TableDefinition<&[u8], &[u8]> etc.

I just saying, never mind.

7

Well, as said above, making V generic is useless, because it's almost determined by the implementation inside the function: &[u8] is stored in table.insert(key, r.as_slice())?.

But K can be made generic as write_value_fixed shows. The downside is K should satisfy 'static.

To accept K as a referenced type, here's a way:

  • the .insert method should be changed to have different lifetimes: it matters when writng generic code
-    pub fn insert<'a>(
+    pub fn insert<'k, 'v>(
         &mut self,
-        key: impl Borrow<K::SelfType<'a>>,
-        value: impl Borrow<V::SelfType<'a>>,
+        key: impl Borrow<K::SelfType<'k>>,
+        value: impl Borrow<V::SelfType<'v>>,
     ) -> Result<Option<AccessGuard<V>>>
     where
-        K: 'a,
-        V: 'a,
+        K: 'k,
+        V: 'v,
  • your function should replace key: impl for<'a> Borrow<K::SelfType<'a>> with key: &K::SelfType<'_>
pub(crate) fn write_key_generic<K, D>(
    table: TableDefinition<K, &[u8]>,
    key: &K::SelfType<'_>, // or `K::SelfType<'_>` this line!
    value: &D,
    db: &Database,
) -> Result<()>
where
    K: RedbKey,
    D: serde::Serialize,
{
    match serde_json::to_vec(value) {
        Ok(r) => {
            let write_txn = db.begin_write()?;
            {
                let mut table = write_txn.open_table(table)?;
                table.insert(key, r.as_slice())?;
            }
            write_txn.commit()?;
            Ok(())
        }
        Err(e) => Err(anyhow::anyhow!("{e:?}")),
    }
}

use anyhow::Result;
use redb::*;
fn main() -> Result<()> {
    let db = Database::open("a.db")?;
    // both are ok, but note the defined type of key affects the type of second argument
    write_key_generic(TableDefinition::<&str, _>::new(""), &"" /* or "" */, &"", &db)?;
    write_key_generic(TableDefinition::<(), _>::new(""), &() /* or () */, &"", &db)?;
    Ok(())
}
8

Yes, you're right.

I guess I was over-abstract.
I'm a Java coder, and I sometimes applied some Java design patterns to Rust

9

I've made a PR and it's megered: fix(insert sig lifetime): key and value should have different lifetimes by zjp-CN · Pull Request #659 · cberner/redb · GitHub

1 Like
10

Cool! I saw that. Looking forward new release of Redb