An API I am using to load a file wants you to pass a closure as a callback.
I want to load several files and avoid callback hell.
Is there any documentation or tutorial or something that takes you from a callback to an .await?
Eventually it would be great to learn how to make it concurrent, but for now I would just be very happy to write code synchronously, loading one file another another.
Thanks in advance.
It depends a bit on the situation. Do you call the library code that calls your callback from async code? Or in other words, will async code be anywhere in the callstack going up from this callback?
Hi Alice,
Those are some big words 
Are those two questions that I have control over, or are you asking what the library does?
There is no async code anywhere in the library. And I am happy to asnyc-ify my code if that gets me some nice goodies (such as preloading images from files (what I am trying to do))
I'm asking how you call the library.
In one of the two situations, the following solution will work:
Runtime object.rt.block_on on the future using that runtime. (or through a Handle)If you are in the other situation, the above will panic with a message saying something about starting a runtime inside a runtime.
I think you're missing a step? I have to wrap load_file(path, callback) into a function that returns a Future first? And the Future has to return a buffer with the file data as bytes.
Show me your callback contents, and I'll show you what I mean
Use oneshot channel to send a completion signal from the callback to async receiver:
For future readers: I have found some code that wraps a callback and moves the loaded file data out of the callback and into a Future. (it has some compilation errors, and I'm working on fixing them now)
https://gist.github.com/niksaak/e29590df2a23b3736ce996199c63ff82
What about using a futures::channel::oneshot channel? You give one side to the callback which will send down the result when it gets called, and the receiver side gets returned as an impl Future<Output = Result<T, Canceled>>.
You see this pattern for transforming callbacks into futures (reading a channel will pause execution until a result is available0 in Go quite a lot.
I'm kinda scared of Tokio. And I feel like if I write this myself, I will understand the process/logic much better.
I'm not attempting networking or concurrency, so I worry that Tokio may be overkill
To run async code you need something which can poll futures to completion. The two main libraries that do this are tokio and async_std.
If you aren't using async in your library anywhere, you can use a normal blocking channel (e.g. std::sync::mpsc channel).
Your thread title explicitly mentions "Future" and "Async/Await" though, both of which have a very specific meaning in the async world (namely std::future::Future and async-await syntax). If you aren't using async functions or tokio or async_std then I'd suggest choosing a different title or updating the original post.
This is a completely foreign concept to me, that I have mostly ignored because I thought it involves threads, and I wanted to leave that to be the final boss in programming
Can you talk to me a little bit about what this concept of channels is, how I can begin to look at the problem through the lens of channels, and point me to further reading/watching content to learn about it. thank you!
A channel is a pair of two objects, a sender and receiver. The sender allow you to send messages (values) from one place in the code to another. In your specific case, you could send a message from the callback, and have the event be handled somewhere else, perhaps in another thread.
As for Tokio, when using async/await, it is mandatory to use an executor, and Tokio is by far the most popular choice of executor.
Well what are you doing with async then?
Well, namely, I just want to preload a bunch of assets for my game without chaining a bunch of callbacks; the last of which actually runs the game.
Thanks to some comments in this thread I was able to get this working elegantly with mpsc:
fn load_image(path: &str) -> Vec<u8> {
let (tx, rx) = std::sync::mpsc::channel();
// this is the library function in question that uses a callback
fs::load_file(path, move |response| {
// process the response as you wish. In this case we are decoding a png file
let img = image::load_from_memory(&response.unwrap()).unwrap();
let img = img.into_rgba().into_raw();
// now we can send that data out!
tx.send(img).unwrap();
});
rx.recv().unwrap() // blocks and waits for message
}
however it doesn't appear to be WASM friendly, and even if it can be modified to be, it has still added a considerable amount of weight to the binary size.
If you want to write it yourself, you can implement Future directly. It would look something like this (compiles, but untested):
use std::task::{Waker, Context, Poll};
use std::cell::Cell;
use std::future::Future;
use std::rc::Rc;
use std::pin::Pin;
#[derive(Default, Clone)]
pub struct CbFuture<T>(Rc<CallbackFutureInner<T>>);
#[derive(Default)]
struct CallbackFutureInner<T> {
waker: Cell<Option<Waker>>,
result: Cell<Option<T>>
}
impl<T> CbFuture<T> {
// call this from your callback
pub fn publish(&self, result:T) {
self.0.result.set(Some(result));
self.0.waker.take().map(|w| w.wake());
}
}
impl<T> Future for CbFuture<T> {
type Output=T;
fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output> {
match self.0.result.take() {
Some(x) => Poll::Ready(x),
None => {
self.0.waker.set(Some(cx.waker().clone()));
Poll::Pending
}
}
}
}
Wow! This is some dark arts man! If you can link me to some learning material that teaches me how it's all put together, that would be amazing. I want to learn this!
So I made some tweaks, I gave CbFuture a new()
impl<T> CbFuture<T> {
pub fn new() -> Self {
Self(Rc::new(CallbackFutureInner::<T>::default()))
}
(Curiously, It's not necessary to turbofish CallbackFutureInner)
I also couldn't rely on #[derive(Default)], So I created it:
impl<T> Default for CallbackFutureInner<T> {
fn default() -> Self {
Self {
waker: Cell::new(None),
result: Cell::new(None),
}
}
}
Now, when actually using it, how am I supposed to use it?
fn load_image(path: &str) -> impl std::future::Future {
use callback_future::CbFuture;
let cb_future = CbFuture::<Vec<u8>>::new();
fs::load_file(path, move |response| {
cb_future.publish(response.unwrap())
});
cb_future
}
This gives me this error:
error[E0382]: use of moved value: `cb_future`
--> src/main.rs:92:9
|
88 | let cb_future = CbFuture::<Vec<u8>>::new();
| --------- move occurs because `cb_future` has type `CbFuture<Vec<u8>>`, which does not implement the `Copy` trait
89 | fs::load_file(path, move |response| {
| --------------- value moved into closure here
90 | cb_future.publish(response.unwrap())
| --------- variable moved due to use in closure
91 | });
92 | cb_future
| ^^^^^^^^^ value used here after move
I don't think we want CbFuture to impl Copy do we?
Edit: I'm doing some reading; it looks like RefCell instead of Cell, may fix this. But it is late right now, I'll have a look tomorrow.
You can clone() it; move one copy into the callback closure, and await the other one. Because of the Rc, both copies will refer to the same object.
The Cells then lets those copies replace the field contents without a &mut reference.
The rest comes from the requirements of Future::poll() in the docs:
Poll::ReadyWaker somewhere and return Poll::Pending. When the value is ready, call Waker::wakeYou’ll also need some kind of executor to drive the Future (await is only legal inside an async block, which compiles to a Future itself), and none are provided in the standard library. It’s possible to write your own, but that requires being comfortable writing unsafe code. Your best bet is picking up a crate that provides one.
You may want to check out The Async Book. Their Under the Hood: Executing Future s and Tasks goes into more detail on how async code works in the Rust world and might shed some light on @2e71828's code.
For those in the future, With great thanks to this thread, and a lot of bugging in the Discord chat, I have solved this problem (by copy/pasting other peoples code mostly! )
You can see the fruit of my labour here: https://github.com/Arifd/miniquad_async/
in src/ there is both a safe (main.rs) and unsafe (unsafe.rs) version. pick one.