Build library with wasm and cmake-rs

thewh1teagle · December 25, 2024, 9:54am

Hi everyone
I'm trying to build the crate sherpa-rs to wasm using wasm-pack on macOS
I can build it for Windows/Linux/macOS
But when I tried to build for wasm with

cd /tmp
git clone --recursive https://github.com/thewh1teagle/sherpa-rs
cd sherpa-rs
rustup target add wasm32-unknown-unknown
cargo build --no-default-features --target wasm32-unknown-unknown -p sherpa-rs-sys

it failed the following error:

   Compiling sherpa-rs-sys v0.6.1 (/private/tmp/sherpa-rs/crates/sherpa-rs-sys)
error: failed to run custom build command for `sherpa-rs-sys v0.6.1 (/private/tmp/sherpa-rs/crates/sherpa-rs-sys)`

Caused by:
  process didn't exit successfully: `/private/tmp/sherpa-rs/target/debug/build/sherpa-rs-sys-ddc28712cd8e2cd0/build-script-build` (exit status: 101)
  --- stdout
  cargo:rerun-if-changed=wrapper.h
  cargo:rerun-if-changed=./sherpa-onnx
  cargo:rerun-if-changed=dist.txt
  cargo::rerun-if-env-changed=SHERPA_BUILD_SHARED_LIBS
  cargo::rerun-if-env-changed=CMAKE_BUILD_PARALLEL_LEVEL
  cargo::rerun-if-env-changed=CMAKE_VERBOSE
  cargo::rerun-if-env-changed=SHERPA_LIB_PATH
  cargo::rerun-if-env-changed=SHERPA_STATIC_CRT
  cargo::rerun-if-env-changed=SHERPA_LIB_PROFILE
  cargo::rerun-if-env-changed=BUILD_DEBUG
  cargo:rerun-if-env-changed=TARGET
  cargo:rerun-if-env-changed=BINDGEN_EXTRA_CLANG_ARGS_wasm32-unknown-unknown
  cargo:rerun-if-env-changed=BINDGEN_EXTRA_CLANG_ARGS_wasm32_unknown_unknown
  cargo:rerun-if-env-changed=BINDGEN_EXTRA_CLANG_ARGS
  cargo:rerun-if-changed=wrapper.h
  cargo:rerun-if-changed=./sherpa-onnx/sherpa-onnx/c-api/c-api.h
  cargo:rerun-if-changed=/opt/homebrew/Cellar/llvm/19.1.5/lib/clang/19/include/stdint.h
  CMAKE_TOOLCHAIN_FILE_wasm32-unknown-unknown = None
  CMAKE_TOOLCHAIN_FILE_wasm32_unknown_unknown = None
  TARGET_CMAKE_TOOLCHAIN_FILE = None
  CMAKE_TOOLCHAIN_FILE = None
  CMAKE_GENERATOR_wasm32-unknown-unknown = None
  CMAKE_GENERATOR_wasm32_unknown_unknown = None
  TARGET_CMAKE_GENERATOR = None
  CMAKE_GENERATOR = None
  CMAKE_PREFIX_PATH_wasm32-unknown-unknown = None
  CMAKE_PREFIX_PATH_wasm32_unknown_unknown = None
  TARGET_CMAKE_PREFIX_PATH = None
  CMAKE_PREFIX_PATH = None
  CMAKE_wasm32-unknown-unknown = None
  CMAKE_wasm32_unknown_unknown = None
  TARGET_CMAKE = None
  CMAKE = None
  running: cd "/private/tmp/sherpa-rs/target/wasm32-unknown-unknown/debug/build/sherpa-rs-sys-4e1b650b1e950cf4/out/build" && CMAKE_PREFIX_PATH="" "cmake" "/private/tmp/sherpa-rs/target/wasm32-unknown-unknown/debug/build/sherpa-rs-sys-4e1b650b1e950cf4/out/sherpa-onnx" "-DSHERPA_ONNX_ENABLE_C_API=ON" "-DSHERPA_ONNX_ENABLE_BINARY=OFF" "-DBUILD_SHARED_LIBS=ON" "-DSHERPA_ONNX_ENABLE_WEBSOCKET=OFF" "-DSHERPA_ONNX_ENABLE_TTS=OFF" "-DSHERPA_ONNX_BUILD_C_API_EXAMPLES=OFF" "-DCMAKE_SYSTEM_NAME=unknown" "-DCMAKE_SYSTEM_PROCESSOR=wasm32" "-DCMAKE_INSTALL_PREFIX=/private/tmp/sherpa-rs/target/wasm32-unknown-unknown/debug/build/sherpa-rs-sys-4e1b650b1e950cf4/out" "-DCMAKE_C_FLAGS= -ffunction-sections -fdata-sections -fPIC -fno-exceptions --target=wasm32-unknown-unknown" "-DCMAKE_C_COMPILER=/opt/homebrew/opt/llvm/bin/clang" "-DCMAKE_CXX_FLAGS= -ffunction-sections -fdata-sections -fPIC -fno-exceptions --target=wasm32-unknown-unknown" "-DCMAKE_CXX_COMPILER=/opt/homebrew/opt/llvm/bin/clang++" "-DCMAKE_ASM_FLAGS= -ffunction-sections -fdata-sections -fPIC -fno-exceptions --target=wasm32-unknown-unknown" "-DCMAKE_ASM_COMPILER=/opt/homebrew/opt/llvm/bin/clang" "-DCMAKE_BUILD_TYPE=Release"
  -- The C compiler identification is Clang 19.1.5
  -- The CXX compiler identification is Clang 19.1.5
  -- Detecting C compiler ABI info
  -- Detecting C compiler ABI info - failed
  -- Check for working C compiler: /opt/homebrew/opt/llvm/bin/clang
  -- Check for working C compiler: /opt/homebrew/opt/llvm/bin/clang - broken
  -- Configuring incomplete, errors occurred!

  --- stderr
  cp: /private/tmp/sherpa-rs/crates/sherpa-rs-sys/sherpa-onnx/scripts/go/_internal/lib/x86_64-apple-darwin: No such file or directory
  cp: /private/tmp/sherpa-rs/crates/sherpa-rs-sys/sherpa-onnx/scripts/go/_internal/lib/aarch64-apple-darwin: No such file or directory
  cp: /private/tmp/sherpa-rs/crates/sherpa-rs-sys/sherpa-onnx/scripts/go/_internal/lib/x86_64-unknown-linux-gnu: No such file or directory
  cp: /private/tmp/sherpa-rs/crates/sherpa-rs-sys/sherpa-onnx/scripts/go/_internal/vad-spoken-language-identification/run.sh: No such file or directory
  cp: /private/tmp/sherpa-rs/crates/sherpa-rs-sys/sherpa-onnx/scripts/go/_internal/vad-spoken-language-identification/main.go: No such file or directory
  System is unknown to cmake, create:
  Platform/unknown to use this system, please post your config file on discourse.cmake.org so it can be added to cmake
  System is unknown to cmake, create:
  Platform/unknown to use this system, please post your config file on discourse.cmake.org so it can be added to cmake
  System is unknown to cmake, create:
  Platform/unknown to use this system, please post your config file on discourse.cmake.org so it can be added to cmake
  CMake Error at /Applications/CMake.app/Contents/share/cmake-3.30/Modules/CMakeTestCCompiler.cmake:67 (message):
    The C compiler

      "/opt/homebrew/opt/llvm/bin/clang"

    is not able to compile a simple test program.

    It fails with the following output:

      Change Dir: '/tmp/sherpa-rs/target/wasm32-unknown-unknown/debug/build/sherpa-rs-sys-4e1b650b1e950cf4/out/build/CMakeFiles/CMakeScratch/TryCompile-Hdvjw6'
      
      Run Build Command(s): /Applications/CMake.app/Contents/bin/cmake -E env VERBOSE=1 /usr/bin/make -f Makefile cmTC_f38b3/fast
      /Applications/Xcode.app/Contents/Developer/usr/bin/make  -f CMakeFiles/cmTC_f38b3.dir/build.make CMakeFiles/cmTC_f38b3.dir/build
      Building C object CMakeFiles/cmTC_f38b3.dir/testCCompiler.c.obj
      /opt/homebrew/opt/llvm/bin/clang   -ffunction-sections -fdata-sections -fPIC -fno-exceptions --target=wasm32-unknown-unknown  -MD -MT CMakeFiles/cmTC_f38b3.dir/testCCompiler.c.obj -MF CMakeFiles/cmTC_f38b3.dir/testCCompiler.c.obj.d -o CMakeFiles/cmTC_f38b3.dir/testCCompiler.c.obj -c /tmp/sherpa-rs/target/wasm32-unknown-unknown/debug/build/sherpa-rs-sys-4e1b650b1e950cf4/out/build/CMakeFiles/CMakeScratch/TryCompile-Hdvjw6/testCCompiler.c
      Linking C executable cmTC_f38b3
      /Applications/CMake.app/Contents/bin/cmake -E cmake_link_script CMakeFiles/cmTC_f38b3.dir/link.txt --verbose=1
      /opt/homebrew/opt/llvm/bin/clang  -ffunction-sections -fdata-sections -fPIC -fno-exceptions --target=wasm32-unknown-unknown  -L/opt/homebrew/opt/ruby/lib  CMakeFiles/cmTC_f38b3.dir/testCCompiler.c.obj -o cmTC_f38b3
      clang: error: unable to execute command: Executable "wasm-ld" doesn't exist!
      clang: error: linker command failed with exit code 1 (use -v to see invocation)
      make[1]: *** [cmTC_f38b3] Error 1
      make: *** [cmTC_f38b3/fast] Error 2
      
      

    

    CMake will not be able to correctly generate this project.
  Call Stack (most recent call first):
    CMakeLists.txt:8 (project)


  thread 'main' panicked at /Users/user/.cargo/registry/src/index.crates.io-6f17d22bba15001f/cmake-0.1.50/src/lib.rs:1098:5:

  command did not execute successfully, got: exit status: 1

  build script failed, must exit now
  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

I would appreciate any guidance or tips on how to resolve this issue. Thank you in advance!

Maybe related: Status of rust+C bindings for wasm targets · Issue #291 · rustwasm/team · GitHub

kpreid · December 25, 2024, 5:12pm

Indeed, your “maybe related” is related. Currently, attempting to include C or C++ code in a Rust Wasm build will always fail to build. I see from your link there’s some progress on fixing this, but even if it works, you’ll still need a C++ toolchain that can cross-compile to Wasm (e.g. installing the missing wasm-ld it looked for), and for the C++ library to be compatible too. It definitely won’t work “out of the box”.

So, you have two options:

dig into setting up a complete cross-compilation toolchain, using the unstable ABI option, and generally making the build work, or
avoid mixing languages — write a Rust Wasm module and a C++ Wasm module and have them talk to each other through the JavaScript host environment.

thewh1teagle · December 26, 2024, 11:39am

Thanks. I tried to compile the c++ library with emcc and link it in rust.
I can build and link it with wasm32-unknown-emscripten or wasm32-unknown-unknown
However when I try to build with wasm-pack with

RUSTFLAGS="--cfg=web_sys_unstable_apis --Z wasm_c_abi=spec" rustup run nightly wasm-pack build --target web

It failed with

thread 'main' panicked at crates/wasm-interpreter/src/lib.rs:245:21:
__cxx_global_var_init: unknown instruction Block(Block { seq: Id { idx: 1 } })
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Error: Running the wasm-bindgen CLI
Caused by: Running the wasm-bindgen CLI
Caused by: failed to execute `wasm-bindgen`: exited with exit status: 101
  full command: "/Users/user/.cargo/bin/wasm-bindgen" "/Volumes/Internal/sherpa-wasm-repro/target/wasm32-unknown-unknown/release/sherpa_wasm_repro.wasm" "--out-dir" "/Volumes/Internal/sherpa-wasm-repro/pkg" "--typescript" "--target" "web"

I created basic repro in github.com/thewh1teagle/sherpa-wasm-repro

If I comment out this line

github.com

thewh1teagle/sherpa-wasm-repro/blob/49095434cee423dffcafd7a86530a4dee8cbb1e4/build.rs#L12


      
          use std::path::Path;
          
          fn main() {
              let project_dir = env::var("CARGO_MANIFEST_DIR").unwrap();
              let lib_dir = Path::new(&project_dir).join("lib");
          
              // Search path
              println!("cargo:rustc-link-search={}", lib_dir.display());
          
              // Link sherpa
              println!("cargo:rustc-link-lib=static=sherpa-onnx-c-api");
              println!("cargo:rustc-link-lib=static=sherpa-onnx-core");
              // println!("cargo:rustc-link-lib=static=kaldi-decoder-core");
              // println!("cargo:rustc-link-lib=static=kaldi-native-fbank-core");
              // println!("cargo:rustc-link-lib=static=sherpa-onnx-fst");
              // println!("cargo:rustc-link-lib=static=sherpa-onnx-fstfar");
              // println!("cargo:rustc-link-lib=static=sherpa-onnx-kaldifst-core");
              // println!("cargo:rustc-link-lib=static=ssentencepiece_core");
          
              // Link onnxruntime
              // println!("cargo:rustc-link-lib=static=onnxruntime");

Then it compile successfully (But of course I won't be able to use the functions from it later)

kpreid · December 26, 2024, 4:14pm

I don’t have experience with this myself, but from what I have heard, you cannot mix Emscripten and wasm-bindgen builds at all. They make incompatible assumptions about the Wasm ABI and how Wasm code should talk to the JavaScript environment. You should not use any Emscripten tools, unless you choose to use wasm32-unknown-emscripten instead of wasm32-unknown-unknown and not use any Rust code that depends on wasm-bindgen.

thewh1teagle · December 26, 2024, 8:54pm

Hmm. so I currently cannot mix WebAssembly modules compiled with emcc or that use WASI. I tried to see how to compile with CMake to wasm target without emcc but didn't find anything. Then how can I compile a CMake C/C++ project and integrate it with Rust nightly that should support C/C++?

kpreid · December 26, 2024, 9:25pm

Perhaps someone else knows, but I cannot answer that question because I have never worked with C/C++ cross-compilation at all, with WebAssembly or not.

Topic		Replies	Views
How to set CMAKE_C_COMPILER and CMAKE_CXX_COMPILER? help	3	19666	November 13, 2019
error[E0463]: can't find crate for `wasmi` help	6	1302	January 12, 2023
How to get a C header-only library to compile for wasm32 with cargo web help	1	857	July 17, 2019
Execute `cargo build` inside a build script help	3	196	September 16, 2024
Failed to run custom build command for `backtrace-sys v0.1.31 help	12	2474	December 22, 2019

Build library with wasm and cmake-rs

Related topics