Feedback on brining Erlang's patterns to Rust

[bkolobara]

Hi everyone!

I'm working on lunatic, a vm that brings some ideas from the Erlang world (lightweight process concurrency, message passing, supervisors, etc.) to other programming languages through WebAssembly. The vm is written in Rust, but here I would like to talk about the other side of things. Compiling your Rust code to Wasm and taking advantage of Erlang's programming model.

Erlang is a dynamically typed language and some of the patterns inside of it don't translate nicely to Rust. Currently there is a rust library (https://crates.io/crates/lunatic) that exposes some of lunatic's principles and packages them in a Rusty way. It's fairly simple, it allows you to spawn processes from functions. The parent gets a handle and can send messages to it, and the child gets a mailbox that can be used to wait on messages and handle them. You can check out some examples in the docs: lunatic - Rust.

Now I would like to extend this simple process model with some higher level constructs (Erlang's GenServers and Supervisors). I have been working on a PR here: Add high level process types by bkolobara · Pull Request #15 · lunatic-solutions/rust-lib · GitHub.

At the moment I would love to get some feedback from the broader Rust community. I guess the most important questions are:

1. How do these programming patterns and APIs feel to a Rust developer?
2. Are the ideas clearly communicated through the code?
3. Could you imagine yourself using lunatic? Or are the benefits of this programming model not so obvious?

[alice]

So, the reason this is able to preempt is because it's running the rust code in a WASM box and it is able to control the WASM box somehow? It seems like an interesting alternative to async/await.

I have also had thoughts on actors in Rust, which I wrote a blog post on here. This is based on spawning async tasks rather than using some sort of process separation though.

As for whether I could imagine myself using this, well, it's cool, but I don't think I have any use-cases where it makes sense to use this over using async/await with Tokio.

A question for you: Why use smol for scheduling as opposed to Tokio?

[bkolobara]

Yes, by introducing a wasm bytecode layer we can insert preemption points during the JIT compilation to native code. You get some of the benefits of async/await without needing to color functions.

I used actually Tokio before (and may switch back to it). If I remember correctly Tokio's TcpStream wasn't clone and I didn't want to expose the splitting mechanism to guest wasm code when I was adding the networking part.

Thank you for the blog post! I will check it out.

[alice]

Huh, I'm very surprised that smol's TcpStream is clone - IO resources like that usually are not. It seems that smol accomplishes this by wrapping the inner IO resource in an Arc. You could do the same with Tokio, though you would need to use the readable+try_read and writable+try_write methods to actually do IO if you did that.

I do think that inserting preemption points like this is a very cool idea.

[bjorn3]

&std::net::TcpStream implements Read¹ and Write² so you can read and write from/to a TcpStream without it having to contain an Arc. If you need you can write Arc<TcpStream> yourself. Smol could do the same to avoid the Arc (and associated allocation cost) for those not using this capability while still allowing reads and writes from multiple threads.

[alice]

@bjorn3 Unlike for the std types, implementing the asynchronous IO traits on immutable references to IO resources is generally problematic, and there are good reasons that neither Tokio nor smol does this. (if you want to discuss why, we should open a new thread for that)