Actor systems using async-await?

help
1

I'm a big fan of the actor model (in particular, the actix crate) and was thinking of using it as the basis for a motion controller.

For my use case, almost everything can be modeled as "poll me every X µs" or "wake me up again when Y happens" which fits in really well with the futures model. I've also seen the source code for several motion controller projects in the past, and needing to deal with asynchronicity (e.g. pause execution of the current task until a condition is satisfied some time in the future) and concurrency (make progress on several tasks at a time) lead to a lot of spaghetti code and complex state machines which use global variables to communicate between tasks. I was hoping async-await would be able to simplify the spaghetti and actors could help with the communication and decoupling.

While I can dive into the source code for actix, it's quite a large system and what I'm really looking for is the high level understanding and architecture.

Can anyone point me towards articles about implementing actor systems (either in Rust or other languages) or async primitives (e.g. a runtime and executor)?

2 Likes
2

I'm developing an actor library, inspired by actix, but ten times smaller code base. Feel free to ask any specific questions you have. You can also have a look at the code at:

Unfortunately it isn't quite polished for publication. I estimate another 2 months before it's published.

You can surely handroll an actor model and an executor, but you can also just use existing ones to implement your motion controller and save yourself a lot of time. As far as I can tell the requirements you have are pretty trivial for existing solutions.

For implementing an executor, have a look at async-task.

As for actor model in rust, there are other alternatives that popped up over the last year: riker, actori, stakker, kay.
So there's plenty to choose from.

3 Likes
3

Funny, I was just going to ask a question about this last week, but couldn't figure out how to boil it down to something simple. Basically it is this...

I have a small, old C++ actor library called "cooper", which was meant for managing resources on embedded Linux systems... Actually, coincidentally, for motor controllers! (And some other sensors).

It's strictly in-process and tries to mimic the Elixir/Erlang GenServer in which you generally hide the actor within a module (class) and just use normal function calls to interact with it. Internally it uses "cast" and "call" to make requests to the actor, where cast is fire-and-forget, but call blocks until the actor returns a value.

So essentially, you don't deal with discrete messages - the public functions/methods define the interactions with the internal actor. Internally every call creates a closure that is queued for the actor to run serially, where cast just returns, but call blocks on a future and returns its result.

It works great. The problem is that it's thread-per-actor, so doesn't scale.

But it certainly doesn't need to be so. Thousands of actors could share a thread pool, provided that each had a serial executor. Each task/function in a single actor could execute on any thread, but we just need to be sure that, for a specific actor, the function runs to completion before starting the next one so that there is no chance for a data race.

Anyway, I thought this would be great to port to Rust, using async/await, if for no other reason than as a learning experience. But the problem remains... If all the functions in a struct are async, is there an existing executor that sits atop a thread pool, but schedules its tasks to run in serial? Or is there a better way to think about it?

1 Like
4

thespis does just this. Basically the mailbox of the actor is a simple while loop that just calls envl.handle( &mut actor ).await;, where envl is an envelope type to be able to have a heterogeneous list of messages (eg. the actor can receive messages of different types at the cost of boxing them).

As you can see here, we pass a &mut actor, so the rust compiler guarantees us that this is the unique access to this object, so it can freely access it's internal state mutably, and when it's done, we take the next message and repeat.

This loop is a future, so you can spawn it on any executor. It will yield to let other tasks make progress when it's waiting for something, letting many actors make progress at the same time, and on the same thread(pool).

The point is that here we require an actor to implement Handler<MessageType>. So it's not so much that all methods on the struct are async, but the Handler<T>::handle is async and returns a future so that it can yield when it can't make progress. The mailbox task will then be suspended midway through processing the message, but it will only start processing the next one when the actor has finished with the current one.

Does that clarify things?

1 Like
5

Perhaps crate crate axiom (github repo) may provide some inspiration (...and perhaps more than that...).

6

Yeah, axiom looks pretty cool; I'll give that a try.

The thing I was describing, though, is meant to hide the implementation details inside a struct, So you interact with it as a normal struct and don't deal with messages or normal actor stuff. So maybe it can be built on top of another actor library, but I'm not sure it needs one.

For example, with a motor controller, you might have the API:

struct Motor { ... }

impl Motor {
    // ...
    pub fn start_moving(&self, distance: f64)  { ... }
    pub fn position(&self) -> f64 { ... }
    pub fn wait(&self) -> Result<()> { ... }
}

So, start_moving() would queue the start task in the serial executor and return immediately. The position() function would queue a query task to the same serial executor and block (await) for it to complete. Ditto for wait().

So the Motor struct would need a reference to a shared thread pool executor to submit tasks to run one at a time. Each public function would need to be able to create a future, submit it to the pool, and then decide whether to wait for it to complete or not.

Does that make any sense?

7

"...any sense?"

Not quite... Why do you need a thread pool c.q. actor system if your Motor control is purely sequential?

8

Well they're not really sequential. They're asynchronous. You start a motion, and then some time later you get a callback telling you the motion completed. So it maps nicely to futures. Some motors need to be polled for completion. That sounds familiar, right? And on a large robotic system, you might be dealing with lots of independent motors, so having them run in a thread pool makes it more scalable.

But another thing is that each motor needs to be callable from multiple threads. Perhaps you have a real-time thread monitoring an Emergency Stop button.

So you need to serialize access to the device. You could use a Mutex, but queuing requests has some advantages.

  1. There's "fairness". Each request is queued in the order received, and no one client thread can hog the Mutex.
  2. Client threads don't block for asynchronous fire-and-forget commands. They just put the request in the queue and continue on.
  3. The internal coding is so much simpler when you don't need to worry about locking!
9

I see. I think axiom will serve you well. I'm using it and am very impressed with the clean architecture. Concerning your case I'd suggest one actor/process for each motor, and async/.await concurrency for local motor control. At any time you can send messages to motors/actors from any place. (The asynchronous rust doc gives clear join examples for controlled execution).

There exist also some crates providing shared memory data manipulation. I'm not using them yet.

Perhaps also relevant: At the moment I'm trying to figure out how to have the actor state stored in the heap, or alternatively, have the actor state on the stack with a field value pointing to a heap object (is an open issue on this forum).

10

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.