are the contributors ever going make async into a complete, robust core feature for threading without tokio and remove the third party dependency on Rust?
The question you would have to answer for this is "why?"
Rust put a lot of effort into ensuring that the "core feature" did not, in fact, tie you to any specific execution implementation, which means you can do things like zero-cost abstract FFI to other async languages like JS, or implement a request execution quota tracking system, or a million other things that are really neat.
Even if you do just only use tokio, it can have more and riskier changes by being a separately installed crate, and the downsides are mostly just that you need to write "cargo add tokio -F full" once
Rust wouldn't be able to make async execution "invisible" like in other languages regardless, as the whole point of it is to not add extra runtime stuff you didn't ask for. None of the current pain points of async have much, if anything, to do with the executor at all, really.
I know this exists for stabilizing a lot of sync rust features into async rust - Just add async - Rust Project Goals
As for an async runtime, idk if an async runtime should exists in core rust. Once you can write async rust in a runtime agnostic fashion (like Zig's IO implementation(s)) you will be free to choose the runtime yourself.
No. Letting the user decide on an async implementation is actually really powerful. It's how we have features like async on embedded, where you couldn't use the same runtime as on a desktop PC.
As for bundling one that users can optionally use, it would be a huge API surface to stabilize in an ecosystem that isn't stable. We aren't yet sure what the ideal async API is, if there even is one, so we can't commit to any one API. That's a common philosophy with Rust's development; it's frustrating at times, but better than becoming bloated with outdated features.
What I might like to see is some surface level API, like filesystem traits or whatnot, to break down inter-ecosystem conflicts.
Language being executor-agnostic is a very good thing. The problem lies elsewhere:
-
Cargo's optional dependencies/feature flags are unsuitable for reliably selecting one of multiple alternative dependencies. Workarounds for that are clunky and fragile enough that very few crates bother to support more than one executor, creating a winner-takes-all situation where it's easiest for everyone to just depend on
tokio. -
The standard library doesn't have interfaces that could abstract away async executors. It's a tricky design problem, because any such API would be the lowest common denominator for async functionality, so it has to be sophisticated enough to be useful, but also not promise too much to avoid excluding alternative executors that may be browser JS, GUI toolkit event loops, or minimal runtimes for embedded systems.
Tyvm for the explanation. Do you think they takeover incorporate tokio in the standard core library?
When you say that, you probably mean it should be officially blessed, easily accessible, and have the quality and trust of the rest of the Rust language.
However, Rust's standard library has only one version, which must stay backwards compatible forever. It needs to be ported to platforms that Rust supports. Its maintenance and evolution is bottlenecked by capacity of the libs team which also maintains everything else in libstd. The blessed status makes it even harder for any alternative better implementation to emerge (and when it does, it fragments the ecosystem, and you're back to depending on a 3rd party).
So putting large complex projects into the standard has terrible costs that may ruin the projects. It would be stuck with the existing API and couldn't improve it without accumulating deprecated and legacy code. It would make it harder to port Rust to new systems. It would throw more work at the libs team, and tie its development to Rust's processes, rather than letting maintainers passionate about the specific niche to freely iterate on it.
So I hope tokio will never get burdened by being chained to the inflexibility and responsibility of the standard library. Instead, the standard library and/or Cargo should make it easier to choose executors other than tokio (tokio is very good, but there are environments where order runtimes are more appropriate).
I agree with all that, I wonder if the problem is sometimes that it can be hard to know how dependable a crate is, so people look for some kind of "official" status. How would I know (for example) that I can depend on tokio?
That's a very good point. I think crates.io could have a section with something like https://blessed.rs or Most popular Rust libraries — list of Rust libraries/crates // Lib.rs
There's also an option of rust-lang org adopting crates under its org, without changing them to be distributed inside libstd. That would avoid most severe limitations while providing "officialness".
The work you have done on lib.rs does a lot to close the gap between official and de facto standards by offering an expressive mixture of quality signals that crates.io lacks. I appreciate the effort and the result.
Being able to swap in the runtime of my choice is a feature, not a bug, but I rely on the metrics in lib.rs to make informed choices.
When I was learning rust, this is what was thought to me, that Rust team intends to not tie us to specific implementations and let the community provide what they think would "work best" for their use case afaik. Which is why we have Tokio as a popular choice but not the sole async crate to exist.
Having the runtime as a separate crate is a good thing, as we can switch it without touching the language.
But I wish we had async versions of stdlib directly in stdlib, not in a separate crate, i.e.: sleep in tokio::time - Rust because currently we have to make our code depend on a particual runtime anyway.
The sleep feature of tokio is surprisingly complicated. You can't call it outside a runtime context. tokio uses a pretty sophisticated timer wheel under the hood. It doesn't really seem like a good candidate for stabilization in the standard library.
This is just saying that you want an async runtime in the stdlib though, because the runtime is actually what implements sleep and other async I/O functions.
This isn’t strictly true. It’s potentially more efficient for the async runtime (or I would rather say, async executor) to be the same library as what handles time and IO (the "reactor"), and Tokio takes advantage of that, but they don’t have to be. And, in fact, Tokio will tolerate you using its reactor and not its executor, if you do it right; for example, this code creates a Tokio sleep future and runs it on futures::executor:
fn main() {
let rt = tokio::runtime::Builder::new_multi_thread()
.enable_time()
.build()
.unwrap();
let sleeping_future = {
// must be in scope for CREATING the future, not running it
let _g = rt.enter();
tokio::time::sleep(std::time::Duration::from_secs(2))
};
futures::executor::block_on(sleeping_future);
println!("*yawn*");
}
Tokio complicates this by making its IO and time functions depend on thread-local state pointing to the Runtime (which, here, is set up by enter().) but after they are called, the future itself can be polled from anywhere. Unfortunately, the thread-local state is awkward enough to manage that it’s a common belief that Tokio-using futures need to be actually polled by Tokio.
Personally, I wish that std simply had a copy of pollster::block_on() in it, so that applications which just need to await some future can do that.
I meant it would be nice to be able to write code in runtime-agnostic manner, without any explicit tokio dependency. I understand that sleep interacts with Runtime API and can't be written without touching the tokio API, but I wonder if it would be possible to create a "runtime specification" we could write code against, just like we write code against C standard without mentioning libc or musl or VisualC. Currently, any async code will have tokio imports anyway, right?
No, there are minimal runtimes and embedded runtimes and so on that aren't based on tokio. (And async code can exist independently of any runtime.)
Do you have any cases when async is essential? I use Rust for several years, and still have no cases where async can be useful. Sure, I do not program micro-controllers, but otherwise all type of applications are in my portfolio.
Other than the standard answer of "anything that's IO bound rather than CPU bound" async APIs implicitly support cancellation, so a simple example like "run until the user presses a key" is actually much simpler with an async API in my experience.
There are philosophical differences between tokio and smol though. E.g., in smol, when you drop a Task, it gets canceled. You have to detach it to let it run in the background. But in tokio, dropping a JoinHandle does nothing. You have to abort if you want to cancel.
More significantly, tokio has fairness guarantees that I don't think smol provides (please correct me if I'm wrong). It's often not apparent looking at code that it relies on tokio fairness but replacing tokio channels with smol can result in bad performance or worse; on the other hand, fairness can cause deadlocks in some obscure situations.