I am writing an OSI networking stack (based on the ITU-T X.200-series recommendations) in Rust here (Sorry to plug a personal project, if that is considered rude, but this is relevant.). Even though it deals with networking I/O, each layer of the protocol stack is modeled in the specifications as a finite state machine, so I figured I could make it synchronous, then define a generic I/O interface (currently called
NSProvider) which would provide non-blocking socket functions using the abstract functions defined for the OSI networking layer. This means that it would not wait for a response to system calls, essentially. It just fires and forgets, and the code that "owns" the state machine would have to synchronously apply events to the stack (e.g. It would not wait for a successful TCP connection. The owning code would do that, then fire the
.dispatch_N_CONNECT_confirm() method on the
NSProvider to synchronously input an event into the protocol stack. To add to this, the relations between objects at the network and transport layers is many-to-many. So imagine modeling a many-to-many relation that shares identifier namespaces and I/O resources, and you can't use async code. Not the easiest problem I have ever solved for sure.
(Sorry, I'm getting to my point.)
I tried to do this with sync code and without using any Tokio-specific primitives, such as Tokio's async Mutex, because I want this library to be as re-usable as possible. But increasingly, I think I need to make it async. I have already started this change, and it has surfaced a bunch of other changes that I would need to make to my code for this to work.
I know going from sync to async is a big deal, but in several cases throughout my Rust experience over the past year, I have had a more broad categorization of this problem crop up over and over again: I write a lot of code, then find out it will not work thousands of lines in. Even when I go to solve whatever problem crops up, I do not know in advance that the solution itself is not yet another problem to be solved.
I don't even think my point is really about async alone. I'm not even sure I know what my question is either. Maybe one or more of the following:
- Do you feel like you are productive in Rust? I know the guys that wrote Bun (the NodeJS alternative) tried it in Rust, then switched to Zig because they were "productive almost immediately," which scares me a little.
- Does this problem get better as you level up in Rust? I have been programming in Rust for over a year now, and I would consider myself to be a very talented software engineer overall, but maybe I still have a ways to go with Rust.
- How do you know how to construct a complicated library or application upfront so you don't get 10,000+ lines of code in before you get the dreadful red squiggly that requires you to redo the whole thing?
- Was there something obviously wrong about my approach to the above project where I would have known not to do it that way upfront? Or is this approach still salvageable?
Thanks in advance