Spawning multiple threads with rayon and wait until all threads finish

help
21 
    let test: Mutex<Vec<String>> = Mutex::new(Vec::new());
    pool.scope(|scope| {
        // Borrow the mutex, so the reference can be moved
        let test = &test;
        for entry in list {
            // `move` so the closure captures the entry instead of borrowing.
            // You might not need that if the real work uses more than &entry.
            scope.spawn(move |_| {
                // do the bulk of work *before* taking the lock
                let hash = create_hash(&entry);
                test.lock().unwrap().push(hash);
            });
        }
    });
22

I think I can identify your last message as a solution but when you
wrote par_iter I wonder if par_iter will not be slow when I work with
a vector that has 3000 filesf only

2022-05-24 0:18 GMT+02:00, Josh Stone via The Rust Programming
Language Forum notifications@rust-lang.discoursemail.com:

23

Why do you think par_iter() would be slow?

24

I found that par_iter is fast especially if there is something complex
wrapped in it and then it pays off but I would like to ask you if
par_iter can maintain the order of the vector it was processing

2022-05-24 1:52 GMT+02:00, Josh Stone via The Rust Programming
Language Forum notifications@rust-lang.discoursemail.com:

25

Yes -- par_iter will execute in some arbitrary parallel "order", but the original order is maintained for things like collect or fold+reduce.

26

but if i make .par_iter().collect then order is ok or i must make sure
in other places?

2022-05-24 18:44 GMT+02:00, Josh Stone via The Rust Programming
Language Forum notifications@rust-lang.discoursemail.com:

27

.par_iter().collect() will keep the same order.

28

i think we can end this resolved topic with just one question at the end.
is there any steps when you using parallel iterators how you can tell
then to use max threads and you set how much?

2022-05-24 19:52 GMT+02:00, Josh Stone via The Rust Programming
Language Forum notifications@rust-lang.discoursemail.com:

29

If you set the number of threads in the overall pool, then that's your maximum.

If you want to limit the number of threads used by one particular task, leaving the pool larger, that's harder because the internal work-stealing has no idea about that kind of grouping. What you can do is prevent your iterator from splitting "too much" in the first place, for example using with_min_len.

30

and if i use rayon::spawn and then i want to drop all threads that is possible?

2022-05-27 20:45 GMT+02:00, Josh Stone via The Rust Programming
Language Forum notifications@rust-lang.discoursemail.com:

31

You can drop the ThreadPool, but there's currently no way to wait for all the threads to exit, nor any loose spawns. Use a scope and its spawn if you want to synchronize their end.

32

if i don't need to wait until all threadpools stuff finish then where
i can find docs or example how to terminate everything?

2022-05-27 23:51 GMT+02:00, Josh Stone via The Rust Programming
Language Forum notifications@rust-lang.discoursemail.com:

33

I don't understand what you're asking for. If you need job results, or confirmed completion, then you should synchronize that. Otherwise, the threads will eventually exit after the ThreadPool is dropped, or if the main thread exits then the OS will terminate other threads.

34

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.