Creating a web worker from Rust/Wasm using a bundler

Hi all,

The way to create a web worker from inside Rust using the web_sys crate is:

let worker = web_sys::Worker::new("./worker.js")?;

If I am using Webpack 5 as the bundler for my app, is it at all possible to specify the worker.js script like this from inside Rust, or am I absolutely forced to forgo a bundler just so that the script URL can be resolved?

For context, I'm working on a mixed JS/Rust application that needs to perform some heavy calculations that would block the main thread. I'd like to offload this to a web worker. It would be easiest for me to pass Rust objects to the worker, not JS objects. For this reason the shared memory approach discussed at Threads and messages with Rust and WebAssembly is appealing since I could use Rust's mpsc channels for the worker communication.

Unfortunately, it has been a bit difficult for me to make sense of what is and isn't possible today using Rust/Wasm and web workers. Most of the examples and discussions I've found on this topic are for projects that don't already use bundlers. For example:

  1. Parallel Raytracing - The `wasm-bindgen` Guide
  2. web-sys: Wasm in Web Worker - The `wasm-bindgen` Guide

Link 1 above states:

Setting up a threaded environment is a bit wonky and doesn't feel smooth today. For example --target bundler is unsupported and very specific shims are required on both the main thread and worker threads.

So bundlers aren't supported by wasm-bindgen. But in my question I'm really trying to understand why they are not supported.

Many thanks!

1 Like

With @wasmer/sdk, we've made a Rust library that gets compiled to WebAssembly and uses Web Workers to implement a thread pool.

When spawning workers, we first create a worker.js bootstrap script.

This waits for an init message containing the SharedArrayBuffer shared between all workers and the WebAssembly.Module for our wasmer_js_bg.wasm file, then it'll dynamically import the wasm-bindgen glue code and pass those to it's init() function.

We've also got a WorkerHandle abstraction in charge of starting workers and communicating back and forth with it. Instead of passing the URL for worker.js to new Worker(), we encode the script as a blob URL. That way you don't need to worry about bundlers moving your worker.js script around or deleting it because it looks unused. This is actually pretty important when your package will be consumed by other people and you don't know what bundler they'll use or how the code will be deployed.

1 Like

If the question is, instead of specifying "worker.js", can we start a webworker from a string 's', what we can do is turn s into a blob and derive a URL for it as follows:

  pub fn new_str_blob_js(code: &JsValue) ...

        let mut blob_property_bag = web_sys::BlobPropertyBag::new();
        let blob = web_sys::Blob::new_with_str_sequence_and_options(&arr, &blob_property_bag).unwrap();
        let mut worker_options = web_sys::WorkerOptions::new();
        let url = web_sys::Url::create_object_url_with_blob(&blob).unwrap();
        let worker = web_sys::Worker::new_with_options(&url, &worker_options).unrap();

Thanks a lot for the replies. I need to study them a bit more, but I think I get the gist of how to instantiate a worker without having to pass its constructor a simple string with the name of a script.

If you're trying to derive from scratch, it may be helpful to start with URL: createObjectURL() static method - Web APIs | MDN . This is what allows us to convert certain JS objects to URLs.

Unfortunately, there's no string there, So we have to do string -> Blob -> URL.

After that, there's some work with mimetypes / worker options, but those are things that can be 'derived' by looking at the chrome dev console errors and fixing them.

1 Like

Thanks a lot for the examples; I have a much better understanding about how this works now by studying it.

One thing I'm still not clear about is how to instantiate the Wasm module inside the worker, or to pass a WebAssembly Memory chunk to a worker and instantiate the Rust/Wasm module with the given memory chunk. As far as I can tell, the latest versions of wasm_bindgen/webpack don't expose anything to JS that supports this when the Wasm package is built with --target bundler.

Do you have any clues as to how to do this? Thanks a lot!

                  const url_js = "url to js";
                  const url_wasm = "url to wasm";
                  var module = await import(url_js);
                  await module.default(url_wasm);

Are you trying to have a single shared array buffer and have MULTIPLE webworkers RUN ON THE SAME SHAREDARRAYBUFFER ? If so, you should @Michael-F-Bryan 's template. All my stuff is for : each webworker has their own memory, and talk only via postmessage.

1 Like

Maybe this is where I misunderstand something, but wouldn't url to js and url to wasm get moved around by the bundler so as to make them not resolvable by the worker? Currently I'm following the pattern in the Rust/Wasm guide where wasm-pack build will output the Wasm binary and JS glue to the pkg directory of the Rust/Wasm app, and then the package is made this available to Webpack by adding it as a dependency in package.json.

To be honest, at this point I'd be happy with either approach, though I think I would favor yours as it seems simpler to get started.

Ah, sorry, I don't know how to get any of this to work with a bundler. My build process is just:

cargo build --target=wasm32-unknown-unknown --release --timings
~/.cargo/bin/wasm-bindgen --target web ./target/wasm32-unknown-unknown/release/XYZ.wasm --out-dir ../www/rust   

This very predictably puts the fiels at ../www/rust/XYZ.js and ../www/rust/XYZ.wasm

1 Like

No problem.

For anyone else that finds this thread, I discovered that the demo project of wasm_bindgen_rayon effectively does what I have been ultimately trying to do: deploy an app using WebAssembly threads that's bundled with Webpack5. I think a minimal example can be created using by following it.

1 Like

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.