Hello, I'm writing a custom executor and can provide happens-before relationship between wake and poll. But should I? Is it expected by code that after wake changes that lead to that wake would be visible to a waked future? If futures are handling it themselves, can I just make everything relaxed in executor?
The documentation of Waker doesn't say you have to, so you don't have to.
And I'd generally expect that if some Future's implementation needs such a relationship, it will be imposed by the synchronization that its internal shared state[1] must have anyway. A Future may be polled at any time, regardless of what the Waker says, so it must not rely on being polled on any specific schedule, so if it touches data that's shared between threads, synchronization is necessary for soundness.
[The above advice is incorrect. See later discussion for why.]
the state that’s read by
Future::poll()and written by the thing that’s also doing the waking ↩︎
3 Likes
You should provide a happens-before relationship. I've written lots of async synchronization primitives where a happens-before relationship is necessary for correctness. (Despite tolerating spurious polls.)
For example, with a message passing channel, you generally do this:
- Sender stores the message somewhere.
- Sender wakes the receiver's waker.
- Receiver gets polled.
- Due to happens-before relationship imposed by the waker, the receiver is guaranteed to see the message that was sent.
Without a happens-before relationship, there's no guarantee that the receiver sees the message when it gets polled. I don't think there's any way to make the channel work without it.
4 Likes
Shouldn't channel have some kind of state variable and release/acquire on it?
Regardless of the answer, the mere fact of.the confusion indicates, that documentation must be updated. I'm not having expertise to do it, how can I initiate the process? Opening an issue in rust-lang/rust is enough?
Of course the channel has internal state with synchronization. The channel's internal synchronization makes sure that you're always allowed to call poll on the receiver. The problem is, the attempt to receive may fail with "no messages available right now". If you want a guarantee that the message is available, then you must have a happens-before relationship between sending and receiving.
Opening an issue is a good start.
1 Like
Opened an issue
There has to be some happens-before relationship, but why is providing it necessarily the responsibility of the executor? In particular, the step “Sender stores the message somewhere” must involve synchronization of whatever it's writing to, so why can’t it be the case that the sender writes with Release and the receiver reads with Acquire?
Intuitively, it seems to me like this approach should be harder to get wrong. Is it less efficient?
That's not enough. Sure, if the acquire read sees the release store, then the message will be delivered. But how do you know that the acquire read will see the store? Without a happens-before, the read could see an earlier value.
1 Like
Ah, I think I see. I was thinking, incautiously, that things were necessarily connected by ordering within a thread, and ordinary physical causality, such that this sequencing must occur:
But from the perspective of thread 2, the store of the task state might be visible before the store of the channel state. Within the sequential code of one thread, the vertical arrows are guaranteed ordering relationships, but from another thread looking at the same memory, they aren't. The release ordering in the channel send guarantees that writes before the release are visible, but doesn't say that relaxed (or non-atomic) writes after the release aren’t visible. So, to thread 2, the wake might be visible before the send completes, and poll() would conclude there's nothing to receive.
Have I got that right?
3 Likes
That's exactly right.
2 Likes
Before "Thread 2 load(acquire) channel state" the Future has to replace waker-prior-poll with waker-current-poll. Would this not have adequate synchronisation?
I don't know what you mean with waker-prior-poll and waker-current-poll, but the assumption is that the operations on the task state are relaxed atomics, so they provide no synchronization.
What I am hinting at is there is no guarantee a executor will (under certain circumstances) call "thread 2 load(any_order) task state" as Future could be moved to another task. So synchronisation has to be done as part of Future when poll is called (a thread 3)
(Not disputing that it would still be better with extra precautionary synchronization on task state.)
Sorry, I don't understand what you're trying to say. Could you be more concrete?
let channel // (something pin box Future)
spawn(async move {
let sleep = time::sleep(Duration::from_millis(50));
pin!(sleep);
select! {
_ = &mut sleep => {
println!("operation timed out");
}
_ = &mut channel => { // first call to poll (thread 2) takes waker-prior-poll, returns pending
println!("operation completed");
return;
}
}
spawn(channel); // second call to poll (thread 3) takes waker-current-poll
})
I am concentrating on the second poll call happening at same time as Thread 1 in @kpreid diagram.
In this case the synchronization happens as part of sending the task to another thread.
(note (being pedantic): the task is a new task it is just the Future that has moved.)
Such synchronization is only between thread 2 and thread 3. It is (in this scenario) before thread 1 make progress and thread 3 calls poll.
Ultimately, as I seem them, the rules are:
- Between any two calls to
Future::pollon the same future, there must be a happens-before relationship. - If
Waker::wakeis called, then there must be a call toFuture::pollthat happens-after the call toWaker::wake.
By the way, does condvar provide happens-before relationship? I'm not sure is that popular crate (executor) actually providing happens-before relationship. Can you please check it out?