How to work with local variables when there are a lot of Futures+Closures

help
1

I'm trying to implement a method which looks for dictionary value in local HashMap and if it doesn't find than goes to the database. I use async mysql with tokio and futures. And it's so hard to implement it because I'm receiving many compiler errors about lifetime :frowning:

Here's my current code:

#[derive(Debug)]
pub struct Dictionary {
    pub name: String,
    pub values: HashMap<String, usize>,
}

#[derive(Debug)]
pub struct Dictionaries {
    values: HashMap<String, Dictionary>,
    pool: Option<mysql_async::Pool>,
}

impl Dictionaries {
...
pub fn get_id(&self, dict_name: String, key_name: String)
                  -> Either<impl Future<Item=usize, Error=DictionaryError>, impl Future<Item=usize, Error=DictionaryError>> {
        //look for id in the local cache
        let id_in_cache = self.values.get(&dict_name)
            .and_then(|dict| dict.values.get(&key_name))
            .map(|id| *id);

        match id_in_cache {
            Some(id) => Either::A(future::ok(id)),
            None => {
                match &self.pool {
                    Some(p) => {
                        info!("Key: {:?} in dictionary {:?} not found. trying to get from DB", &dict_name, &key_name);
                        let res = p.get_conn()
                            .and_then(move |conn| {
                                let insert_sql = format!("INSERT IGNORE INTO {} (NAME, created_date) VALUES('{}', now())", &dict_name, &key_name);
                                conn.drop_exec(insert_sql, ()).map(|c|(c, dict_name, key_name))
                            })
                            .and_then(|(conn, d, k)| {
                                let select_sql = format!("SELECT id FROM {} WHERE name = '{}'", &d, &k);
                                conn.prep_exec(select_sql, ())
                                    .and_then(move |result| {
                                        match result.is_empty() {
                                            true => panic!("Id should be there"),
                                            false => {
                                                result.collect::<usize>()
                                                    .map(move |(_, v)| v[0])
                                                    .map(move |new_id| {
                                                        info!("Creating for dict {:?} key {:?} with id: {:?}", &d, &k, &new_id);
                                                        //let dict_opt = &self.values.get(&d);
                                                        new_id
                                                    })
                                            }
                                        }
                                    })
                            })
                            .map_err(|e| {
                                panic!("Can not use dictionaries: {:?}", e);
                            });

                        return Either::B(res);
                        //future::ok(10)
                    }
                    None => Either::A(future::err(DictionaryError::PoolIsNotExisted))
                }
            }
        }
    }
}

First of all, I can not use refs in method params like dict_name: &String, key_name: &String
Because I receive that lifetime of these references is too short to be used in and_then futures.

Next, if I have and_then(#1)...and_then(#2) and I need to use dict_name in both of them then I should make clone or pass them as params for the next future:

.and_then(move |conn| {
    ...
    conn.drop_exec(insert_sql, ()).map(|c| (c, dict_name, key_name))
})

How should I work in Rust way for such use cases?

And the last thing I couldn't win is &self reference.
After I inserted the new value to database I have to put in self->HashMap
But if I try to make let dict_opt = &self.values.get(&d); inside my future then I receive lifetime error :frowning:
Is there a solution?

It's quite hard to understand the right way in Rust after Java or JS where you can easily use local variables in closures.

Thanks :slight_smile:

2

The standard way to get around lifetime issues is to use reference counting with Rc/Arc (use Arc if you need thread safety). The problem is that because these are shared they prevent mutability. To regain mutability you can use Cell/RefCell for Rc and Mutex/RwLock for Arc. Note: Cell is only really useful for Copy and Default types. With reference counting you get a really cheap clone so you can pass of a clone for each closure to use.

can't this be written as an if-else instead of the match, or even

assert!(!result.is_empty(), "Id should be there");
result.collect::<usize>()
    .map(move |(_, v)| v[0])
    .map(move |new_id| {
        info!("Creating for dict {:?} key {:?} with id: {:?}", &d, &k, &new_id);
        //let dict_opt = &self.values.get(&d);
        new_id
    })

If you don't mind using nightly Rust you can use async-await syntax to reduce the boiler plate and maybe ease the lifetime issues.

tracking issue
RFC

edit based off of @Hyeonu's comment

1 Like
3

I tried to use Arc:

pub fn get_id(&self, dict_name: String, key_name: String)
                  -> Either<impl Future<Item=usize, Error=DictionaryError>, impl Future<Item=usize, Error=DictionaryError>> {
        let this = Arc::new(self);
        let this1 = this.clone();
        ...
let res = p.get_conn()
                            .and_then(move |conn| {
                               ...
                            })
                            .and_then(move |(conn, d, k), | {
                                ...
                                conn.prep_exec(select_sql, ())
                                    .and_then(move |result| {
                                        assert!(!result.is_empty(), "Id should be there");
                                        result.collect::<usize>()
                                            .map(move |(_, v)| v[0])
                                            .map(move |new_id| {
                                                info!("Creating for dict {:?} key {:?} with id: {:?}", &d, &k, &new_id);
                                                let dict_opt = this1.values.get(&d);
                                                new_id
                                            })
                                    })
                            })
                            .map_err(|e| {
                                panic!("Can not use dictionaries: {:?}", e);
                            });
...

But it doesn't work :frowning:

error[E0495]: cannot infer an appropriate lifetime due to conflicting requirements
  --> src/dictionaries.rs:82:29
   |
82 |         let this = Arc::new(self);
   |                             ^^^^
4

You are putting self, which has type &'a Self, into the Arc, but you probably need something that's 'static. Try putting the HashMap into an Arc and clone it when necessary.

1 Like
5

Cell is also useful for Default types! I prefer Cell<Option<T>> over RefCell<Option<T> as refcell_option.borrow_mut().take() has some runtime overhead while cell_option.take() has not, and it's even shorter.

1 Like