Distributed Computing


#1

Hi!

I am currently using simple_parallel for parallel computations (Genetic Algorithm). I’d like to improve performance by distributing the problems to multiple computational nodes.
Is there a crate that is easy to use that does exactly that? A
distributed computing pool? Any suggestions what library to use? If I
were to hack it myself, I’d probably start building it on top of
nanomsg.

Thanks!

Michael


#2

You can take a peek at timely dataflow. It may be a bit more exotic than you need, but it does make it pretty easy (for me) to stand up processes with multiple workers on different machines, and ship data between them.

There are a few blog posts introducing it through fairly simple examples. Here is part 1.


#3

Thanks! I will try it!