Async or threads for many process::Command?

help
#1

Hi! I have 255 process::Command instances I want to run with the output being processed after. I want these done as fast as possibe. Right now I made a thread pool of four threads each doing a task when available until the 255 Commands are run and output processed.
Is this the fastest possible way? Or is the fastest way to implement an async library to spawn 255 tasks? Perhaps just creating 255 OS threads is the fastest?

Any help is appreciated!

#2

You probably wont beat spawning 255 OS threads.

1 Like
#3

Oh ok, I was not sure if I was able to spawn this many OS threads even. Thanks!

#4

The limit is around a few thousand threads.

1 Like
#5

There's a lot of variables in play.

Do the tasks use the same resources? (accessing file system, accessing network, computing information on CPU)? Do the tasks wait for their own IO (e.g. accessing remote web resources that might take some time)? Do some tasks depend on results from other tasks?

Default stack size in Rust when spawning a thread is 2MB so when you fire off your 255 threads you will consume 610MB memory just for the stacks. So if you are just coordinating work then async can work better.

#6

Note this is platform specific. On some platforms threads simply reserve a range of virtual memory and doesn't actually commit resources until a virtual page is written to.

1 Like
#7

Even then 610 MB is not a big problem on a modern computer.

#8

On Windows, spawning 256 threads (I said 255 but meant 256 :stuck_out_tongue:) is fast as hell and my application, which is a websocket server among other things, uses ~16MB of memory at most spawning 256 OS threads.
Screenshot 2020-10-27 125658

1 Like
#9

To answer your question ehiggs, I am spawning 256 ping commands with a max wait time of 1.5 seconds. When output is ready, it is determined if ping was successful or not.

#10

Threads are for doing real work. Heavy calculation can be spread over as many cores as you have.

Async is for waiting. With it's lightweight tasks it can wait on thousands of network connections, or whatever whilst using less memory and resources in context switching.

If you are spawning thousands of pings then you are doing a lot of waiting. I would use async.

Edit: Of course async systems like Tokio also maintain a thread pool and can spread async tasks around cores. Which is even mo' better.

#11

Would you recommend asynd-std or tokio? As I understand it, when implementing async in my contex I want a "green thread" kind of setup, where all my tasks are spawned instead of thread::spawn. Are both these libraries capable of that?

Edit: Does tokio maintain a thread pool automatically, or do I need to enable it in some way? Is it just to call tokio::spawn instead of thread::spawn?

#12

I would recommend Tokio of the two. Async is certainly usable in your scenario, but so are threads and threads seem simpler in this case.

1 Like
#13

To elaborate a bit, your scenario has the following features:

  1. All the tasks are created at the start rather than continuously like on a web server.
  2. You care about minimizing the total running time rather than the running time of each individual task. Or in other words, you don't care if a few tasks are much slower than others, as long as the total is low.
  3. The work is IO bound (in this case on talking to other processes)

When you are in this case, it does not matter that much whether you use threads or async. Both will do the job just fine.

Note that the third bullet point is why I recommend spawning 255 threads rather than one for each core. If the work is CPU bound, you definitely want threads, but you also want to limit the number of threads.

1 Like