Accept Rayon ParallelIterator or standard Iterator?

help
#1

I'm working on a small library that has functions that take iterators as arguments. The functions are "embarrassingly parallel", so I decided to use Rayon to speed them up. However, I would like to accept either a Rayon ParallelIterator or a std::iter::Iterator, but there doesn't seem to be a way to accept both.

I can take an impl ParallelIterator, which will work with Rayon iterators but not sequential iterators, or I can take an impl Iterator, which will work with sequential iterators but not Rayon iterators.

I only use methods that are available on both Rayon and sequential iterators (only map and collect). Is there a way I can accept both?

#2

I don't think there's a good way to do it in one method, because in theory a type could implement both traits, and you have to choose how to use it.

One possibility is to use different entry points, and internally make it call your shared code. I have an experimental rayon-cond crate which you could use for the common case.

#3

I'm new to using Rayon, so I'm wondering what people tend to do for these situations? It seems like a fairly common occurrence that you want your API to work both sequentially and in parallel.

#4

I think people usually just duplicate it. But if you're really only using map and collect, you can also just have two methods that share the map, implemented foo.into_iter().map(shared_call).collect() and the same with into_par_iter().

2 Likes
#5

I like that idea! I think I'll try that and maybe some other things too. Thanks for your help :slight_smile: