Defeated by the borrow checker extending a vec in parallel with rayon

help
1

Here's a simplified version of what I'm trying to do:

type Items = Vec<Item>;

fn read_items(rec_ids: &[i32]) -> Result<Items> {
    let mut db = Database::open()?;
    let mut items = vec![];
    items.par_extend(rec_ids.par_iter().filter_map(|rec_id| {
	if db.set_record(*rec_id) == true {
	    Some(db.read_items(*rec_id)) // Vec<Item>
	} else {
	    None
	}
    }));
    Ok(items.into_par_iter().flatten().collect())
}

The borrow checker is telling me "cannot borrow db as mutable, as it is a captured variable in a Fn closure" and that I should use FnMut. I need db to be mutable because set_record changes an internal database cursor. But I don't know how to change this to FnMut.

Also I doubt that the way I'm doing this is the best way. I'm trying to iterate over a selected bunch of records and for each record I gather a vector of items, the aim being to end up with a single flat vector of all the items from all the records in the rec_ids list.

2

Your Database can't be updated concurrently by multiple threads as you said it actually changes an internal database cursor. .par_iter() makes parallel iterator that runs its modifiers(like the argument for filter-map() in this case) concurrently by multiple threads.

1 Like
3

To make it "work" requires going from FnMut closure to Fn. Which is achieved using internal mutability. For multithreaded this is done by wrapping db in Mutex.

It comes with performance penalty so quite possible single threaded or alternative approach is needed.

1 Like
4

OK, thanks, I get it now.

5

I just want to point out that this is a great example of Rust strengths: your code was racy, and Rust's type system (Fn vs FnMut closures in this case) was able to spot it and forbid compilation :slight_smile:

3 Likes
6

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.