How to bundle a dependency as a WASM blob

1

I've got a project that consists of two components:

  1. A processing script that ingests data and spits the processed result out in some format
  2. A viewer for the data format which you can either run locally or through a browser (as a WASM blob)

At the moment, the WASM blob is hosted at a central location, which permits (in theory) the user to upload the processed data to any webserver, so they can share it. In practice this doesn't work nearly as well as we'd like because most users don't have the right Content Security Policy headers set and do not have the know-how (or even the ability) to set them correctly.

So we'd like to just bundle the WASM blob directly so that the data dump from (1) above is self-contained and capable of viewing itself if the user uploads it to any webserver.

If I were going to do this by hand, I could do it something like:

git clone https://.../path/to/viewer.git
cd viewer
trunk build --release
cp ... # copy distribution files somewhere

And then we can include the blobs as necessary via include_bytes! or similar.

The trouble I'm having is that I don't actually know how to drive the build via Cargo. I'm aware of build.rs, but I don't know how to fetch source for a dependency, locate it, build something with it, bundle that into a known location, etc.

Does anyone have experience doing something like this, and have any tips?

2

Cargo is a tool primarily for building Rust code; wanting to bundle things isn't necessarily in Cargo's scope, and using a different packaging system for creating distribution builds isn't unusual. (cargo-dist is currently my favored tool.)

However, if you want to embed the WASM blob directly via include_bytes!, a Cargo buildscript is the correct way to do that and plenty sufficient.

Rather than git cloneing the viewer, it's probably a better option to make it a submodule of the project. The buildscript can then run commands (e.g. via std::process::Command, or cc if appropriate) to build the viewer and place the blob into the OUT_DIR. The main crate can then include_bytes!(concat!(env!("OUT_DIR"), "viewer.wasm")) to embed the newly compiled blob.

It's also a good idea to use the correct rerun-if-changed directives to control when the buildscript gets rerun and thus the viewer gets rebuilt.

Also note that if you recursively call into Cargo from the buildscript, you need to pay attention to avoid cases where Cargo can deadlock due to recursively acquiring the build lock in a single workspace. (That happens less than it used to, but it still can happen.)

In the future you might be able to use Cargo binary dependencies for this purpose.

2 Likes
3

Thanks for the response. I do very much get the sense that I'm pushing the boundaries of what's intended here.

One thing I found is that Cargo has a feature called "links", intended mainly for external C libraries, that happens to allow you to pass arbitrary key/value pairs around the system.

https://doc.rust-lang.org/cargo/reference/build-scripts.html#the-links-manifest-key

It's a bit tricky to use because you need to specify the links property in Cargo.toml even if you do not plan to link anything:

But using this it's possible pass information around from one crate's build.rs to another one. In my case I can pass the path to the dependency source code so that I can build it a second time with trunk.

That still leaves some other issues such as getting trunk itself (would be solved by Cargo binary dependencies) and some unrelated deployment issues.

4

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.