How to deal with WebAssembly and Javascript Async interactions with Synchronous Crate APIs


I'm trying to move some standalone rust code into web-assembly so it can be used as part of a Javascript app, but I'm running into some walls when it comes how to get file data into the rust code.

Obviously I can't use the rust file APIs directly from web-assembly so instead I've written some Javascript using the File API to get a file from the user, and I want to pass this down into rust.

The problem comes from the fact that the file i want to process is >4GB, so I can't just read it into memory and pass it down as an Uint8Array. Instead I want to process it chunk-wise.

The JS File API provides the ability to get a slice of a file as an ArrayBuffer, but this is a Promise based API so asynchronous.

So I already understand the following:

  1. I can interface with the Javascript APIs via wasm_bindgen, web_sys and js_sys.
  2. The Javascript API returns a Promise.
  3. I can use the wasm_bindgen_futures crate to convert the JS Promise into a Rust Future.

However at this point I'm confused as most of the guides on the internet expect you'll be using tokio/async-std to deal with futures.

I presume I can't just use one of these modules as I'm essentially using the Javascript engine to provide my asynchronous runtime.

wasm_bindgen_futures provides a spawn_local method which I suspect I how I'm suppose to "wait" for this future. The documentation isn't clear but I presume it converts to a JS Promise pushes back up to javascript when has a then which then calls back into rust and resumes that call?

However the problem is that that function only works on Future<()> which makes sense as I can't see how you'd wait for a result using that mechanism.

The problem here is that I actually want to feed the data read from this file into the Gimli crate's Dwarf.load method. This method expects to be passed a FnMut which is can call whenever it a particular section's data.

I can't see how it'd be possible for me to write a callback function that can then await the Future/Promise which reads the data before passing it back to Gimli as this requires maintaining Gimli's call stack upto that point.

If this weren't in WASM I think I could just have Gimli running in a thread which then blocks on a mutex until the data is ready, but currently WASM has no threading support either.

So ultimately think my question here is, do i have to rewrite/modify gimli to be entirely async in order to be able to use it from this context? Or am i missing something?

When using futures, you would not be spawning the future you get from the JS file api directly, but instead you would wrap it using an async block or function, and then you would spawn the async block instead.

let fut = async move {
    loop {
        let chunk = get_next_file_chunk_js_future().await;

You can't really block in webassembly, so it is not possible to write a closure that performs async operations inside like gimli appears to require.