Demystifying Async/Await - sort of ;)

[2ndTaleStudio]

well you can create a PR to this repo: https://github.com/RusPiRo/ruspiro-async-book

Thanks for the support.

[2ndTaleStudio]

Hey,

I've refactored the waker chapter based on your (@Yandros) proposal. I also find it a bit more clear now
However, the double Arc indirection you mentioned I do not fully grasp. I thought the RawWaker was required to get the type erased representation of the trait object of the actual Waker/ Wakeable - so I did not touched anything on that double indirection thingy

Would you consider it to be important to be mentioned as a reason for the VTable and RawWaker things?

[Yandros]

@2ndTaleStudio

However, the double Arc indirection you mentioned I do not fully grasp. I thought the RawWaker was required to get the type erased representation of the trait object of the actual Waker / Wakeable - so I did not touched anything on that double indirection thingy

Would you consider it to be important to be mentioned as a reason for the VTable and RawWaker things?

The idea is that, given that:

@Yandros

a Waker is basically:

type Waker = Arc<dyn WakerMethods>;
// where
trait WakerMethods {
    fn wake (self: Arc<Self>);
    fn wake_by_ref (self: &'_ Arc<Self>);
}

Then,

why aren't things like so in practice? Why the whole unsafe RawWaker and RawWakerVTable ordeal?

That would be a valid concern from a reader, and not directly answering that question will keep some of that halo of mystery around async / .await that your blog post tries to prevent. So, answering it is both nice, and ... complicated. Hence the suggestion for a drop-down menu. This way, quick readers can keep the mystery, but knowing there is a place they can refer to if and when they wish to elucidate it

So, regarding the limitation, in and on itself, is quite basic: the above code causes the following compilation error:

error[E0038]: the trait `WakerMethods` cannot be made into an object
 --> src/lib.rs:9:12
  |
3 | trait WakerMethods {
  |       ------------ this trait cannot be made into an object...
4 |     fn wake (self: Arc<Self>);
5 |     fn wake_by_ref (self: &'_ Arc<Self>);
  |                           -------------
  |                           |
  |                           ...because method `wake_by_ref`'s `self` parameter cannot be dispatched on
  |                           help: consider changing method `wake_by_ref`'s `self` parameter to be `&self`: `&Self`

Which contradicts the "intuition" that this can be implemented. And the reason for that is that Arc is not a completely honest abstraction. A good way of seeing what an Arc is, is that it is a PseudoBox< ArcInner<T> >, where ArcInner is just the T and the atomic counters put together.

So, if we had such hypothetical types, we could define WakerMethods as:

trait WakerMethods {
    fn wake (self: PseudoBox<ArcInner<Self>>);
    fn wake_by_ref (self: &'_ ArcInner<Self>);
}

and both methods would be valid-to-use-as-virtual-methods / valid-to-use-in-a-trait-object / object-safe / dyn-safe, since both selfs usages above would have had the layout of a single, direct pointer.

But since we don't have such hypothetical types, we end up having to do it manually:

trait RawWakerMethods {
    /// implementation interprets `self` as Arc<Self>
    fn wake (self: *mut Self);

    /// implementation interprets `self` as &'_ ArcInner<Self>
    fn wake_by_ref (self: *const Self);

    /* Plus compiler-generated:
    fn clone (self: *const Self) -> *mut Self { ... }
                    ^^^^^^^^^^^     ^^^^^^^^^
             &'_ ArcInner<Self>     Arc<Self>

    fn drop (self: *mut Self) { ... }
                   ^^^^^^^^^
                   Arc<Self>
}

And since the above cannot really be written as a trait either, we end up manually hand-rolly these four virtual methods (RawWakerVTable), then bundled with the raw-ified Arc<Self> as *const ..., yielding the RawWaker.

[Aclassifier]

Thanks for the discussion, and the book. I am trying to follow the Rust community and the language in itself, especially everything concurrent. But alas, I have not coded a line of Rust! And to be honest, not read all the words above, either. (But I have censored papers on Rust.)

The purpose of this comment is to tell that there also is "another half of the field", the synchronous. I will refer to matters out of scope for Rust per se - not meaning to harm or discredit any of the above - which seems right in its own context.

But I do have experience with languages that support concurrency. Namely occam 2 in the nineties, go (some, when it arrived) and now I do XC. Their concurrency (and even parallelism for parts of occam 2 and XC) all have a theoretical foundation in Communicating Sequential Processes (CSP) process algebra.

Those languages are basically synchronous. Should one need asynchronism it's built on top. If I keep libraries out of the reasoning, in occam and go I would, in order to add asynchronism, add extra PROC or goroutines, in XC it would be by the help of a so-called interface, with several patterns being possible, to get help from the compiler. But under the hood it's synchronous or state machines or locks or use of different types of tasks (standard, combinable and distributable). For the latter one can also set time constraints by pragmas, it's possible to build multi-core (or rather multi logical core) deterministic processing. From a log: "Pass with 14 unknowns, Num Paths: 8, Slack: 400.0 ns, Required: 1.0 us, Worst: 600.0 ns, Min Core Frequency: 300 MHz." When other tasks are also running.

Some times asynchronousity is necessary (most often close to I/O) and some times it's nice, but often synchronous systems are just as fast and responsive. Some would argue: safer. And reasoning seems to be most often done when having synchronous systems to reason on (like CSP).

Blocking is ok, not pathological if it doesn't let other jobs pathologically wait. So, it often has to do with parallel granularity (not my phrase). Have two threads in a large system and ten independent jobs, then blocking does not seem acceptable. But with 10 threads (task, processes) and 10 jobs reasoning seems easier.

I have over the years blogged quite some about this. Disclaimer: no ads, money. gifts etc. So I take the opportunity to link a few relevant here.

The first is Rick Hickey's lecture on core.async: Clojure core.async at my blog note on clojure core.async.

I have a note where I try to go discuss what "blocking" might imply, see Not so blocking after all.

Then I compare a lecture by C++'s Herb Sutter and go's Rob Pike: search for "Pike & Sutter: Concurrency vs. Concurrency" in my TECH NOTES. (Also commented by Herb Sutter).

[Aclassifier]

André: typo: in About "with it's promises" → "with its promises" ie. (not "it is").

[maxuuell]

Thanks for putting this together.