I'm using rayon for some expensive background tasks in my app.
Sometimes the user will perform some action in the app that generates new work that I want to run though rayon. I would like this work to take priority over the background work.
If I use the single global rayon thread pool for everything it creates problems. The background work must all be finished before the user work gets processed. To work around this I'm currently using two rayon thread pools, one for background tasks and one for user initiated tasks. This works pretty well, but I'm wondering if there's a better way to do it. A more general way of adding priority to tasks/scopes in rayon?
I'm not super familiar with rayon; I see that it has a ScopeFifo type, but FIFO is the opposite of what you want. You want something that is LIFO. I see that there's also a Scope type, but it doesn't mention LIFO or FIFO at all, and since you're asking this question, it probably isn't LIFO.
You could check some "is user initiated stuff running" value shared using an Arc<AtomicBool> and sleep in the background processes. This would give the other threads more CPU time.
Another possible approach would be to code up your expensive operations as state machines. A state machine is basically a way to pause something and continue later. This would allow you to keep the state of a partially finished operation in a variable, and continue the task when you want to. Now you can run them on a custom thread pool and just pause the unimportant operation when a user-initiated one comes in.
The state machine solution is basically the future ecosystem, and you could do it using futures, but unfortunately futures aren't really built for expensive computations.
This is probably the best you can do right now. We don't have any sort of prioritization in rayon yet, though we've thought about it -- I haven't come up with anything that doesn't approach "reinvent the OS scheduler". It might be a good idea for you to adjust the thread priority of that background pool though, e.g. by setting the unix nice level in a start_handler.
The normal Scope is kind of a hybrid in that respect -- locally LIFO, but stealers are still FIFO. This is the default mode for rayon's work queues in general. See the scope scheduling RFC for more.