Rust to JS with Emscripten


#1

So it’s a bit rough (especially the std parts, I mutilated the library a little bit), but it works!

A hosted example: http://bl.ocks.org/AerialX/1041460cb9dd5876658c

Basically, a light Cargo wrapper is used to do all the heavy lifting. To get this all working, compile the cargo-build project with cargo (make sure to provide LLVM_PREFIX), then follow the instructions to build the std variant for the i386-unknown-emscripten target, place it in a rustc sysroot hierarchy, then use cargo-build to compile projects using it.

For more details… A flexible target spec is used to tell rustc to build i386 with emscripten’s data layout, so it actually generates rlibs with x86 + LLVM bitcode for LTO purposes. The final binary linking step then uses LTO to spit out LLVM IR, so all dependencies get pulled in as one bitcode file. Then I run a hacky string replace over the IR to get LLVM 3.5 to accept it, and some optimization passes over it to get rid of llvm.assume. The cargo wrapper just exists to handle all this for us, since it’s usually not flexible enough to control what arguments are passed on to rustc. Then std is changed slightly to pull out most of the native parts, mainly threading and unwind functionality - sync primitives are stubbed out.

Looking toward the future…

  • Proper support for native dependencies. These will probably work well enough right now, but need to be manually linked in with emcc after Cargo has generated the LLVM IR.
  • Piston support! The existing SDL+OpenGL backends hopefully won’t be too hard to get working.
  • Ideally rustc would be able to just store/create/link bitcode in the rlib. Assembling doesn’t always make sense for platforms like Emscripten.
  • Bring Emscripten (and its PNaCl passes) over to LLVM 3.6. This is all a hack that just happens to work, and it’d be much nicer if we didn’t have to worry about the incompatibilities between the two versions.
  • Make emscripten an actual supported platform in std. Decide on a decent strategy for it and consider what to do with the system components that don’t apply to it.

#2

:heart_eyes:

I just tried it, and I’m suprised how well it’s working!
Things are still incredibly hacky though. I wish there was some way for it to “just work”, though that’s highly unlikely.

Since I can’t fork your repo and issues are disabled, I’m just leaving a few notes here:

  • In the build script of cargo-build, I had to change LLVM_PATH to LLVM_PREFIX because the repo being compiled expected the latter.
  • It’s probably better to use multirust to compile cargo-build. The required version is in src/rust_version.txt in the Cargo repo.
  • In order to compile cargo-build, I copy-pasted the Cargo.lock file from the cargo repo. Cargo is smart enough to fix everything and still keep the right versions of dependencies.
  • I had to replace piped() by capture() here because piped is too recent.
  • If emcc and opt are not in the PATH, the error message is very confusing (Os error 2 or something), but that’s more a problem with
  • The std::os::self_exe_path() does exactly what you’re doing here but in a cleaner way.
  • I wanted to pass -s ASYNCIFY=1 to emcc, so that’s another modification to cargo-build.

#3

Thanks for the feedback! Most of this should be addressed in the repo now.

I wanted to pass -s ASYNCIFY=1 to emcc, so that’s another modification to cargo-build.

I’m not sure whether I want to add support for emcc flags, I can see that ending up turning into a mess quickly. Ideally maybe they’d go in a section in the Cargo.toml? That may require actual changes to cargo though… For now, using --emit llvm35-ir instead of --emit em-html and then calling emcc on the output under target/ works, and can be integrated into a script easily enough. Could potentially add an option for it though that takes comma-separated arguments?


#4

Good stuff!

Re: “Bring Emscripten (and its PNaCl passes) over to LLVM 3.6. This is all a hack that just happens to work, and it’d be much nicer if we didn’t have to worry about the incompatibilities between the two versions.”

We’ve had a few patches recently to improve Rust support on Emscripten/PNaCl. PNaCl is close to tip-of-tree and will be past 3.6 soon, and I expect Emscripten to follow. The bitcode version issues shouldn’t matter if the Rust frontend is built with the Emscripten/PNaCl codebase, which I would say is a medium-term goal.

There’s a GSoC project to improve this:
https://developer.chrome.com/native-client/reference/ideas#rust


#5

Sorry, late response, but thanks for the info! It will be nice to see if that can come together soon - emscripten has seen some work done on merging with llvm-pnacl/master recently.

For a bit of a final update, I’m satisfied with getting this working to demo the kind of complex projects we can get working with Rust + Emscripten right now. It makes use of many dependencies such as piston, GL, gfx-rs, rustc-serialize… It also uses compiler plugins and FFI with a couple native C libraries (SDL and for PNG image decoding). Overall it’s a great test, and cargo makes it all easy to build.

hematite demo (https://github.com/pistondevelopers/hematite)
(warning, 25MB web app and may use a fair bit of system resources. Controls are WASD, Space, and Left Shift)

The only real stumbling block I’ve run into so far was this issue - gfx-rs happens to use a lot of struct layouts that end up triggering it. The LLVM compatibility issues would also be nice to solve, but they haven’t gotten in the way too much thus far.


#6

Would it bug you if I mentioned this demo as well fails for me in Safari and Chrome?

(In Firefox, it didn’t crash, but instead locked up my system. O tempora.)

Failed to load resource: the server responded with a status of 404 (Not Found)
pre-main prep time: 2841 ms
Calling stub instead of signal()
thread 'main' panicked at 'called `Option::unwrap()` on a `None` value', src/libcore/option.rs:362
fatal runtime error: panic: called `Option::unwrap()` on a `None` value
trap!
trap!
Uncaught abort("trap!") at Error
    at Error (native)
    at Wa (http://myth.aaronlindsay.com/test/hematite.js:1058:26)
    at w (http://myth.aaronlindsay.com/test/hematite.js:1341:97)
    at s.Je._llvm_trap (http://myth.aaronlindsay.com/test/hematite.js:1225:431)
    at Tx (http://myth.aaronlindsay.com/test/hematite.js:1289:61887)
    at Wx (http://myth.aaronlindsay.com/test/hematite.js:1289:65315)
    at Cx (http://myth.aaronlindsay.com/test/hematite.js:1289:33131)
    at Dx (http://myth.aaronlindsay.com/test/hematite.js:1289:33776)
    at Az (http://myth.aaronlindsay.com/test/hematite.js:1289:118176)
    at zz (http://myth.aaronlindsay.com/test/hematite.js:1289:117889)
If this abort() is unexpected, build with -s ASSERTIONS=1 which can give more information.

#7

Heh, not at all! A 404 is weird though, can you check the network tab to see what it’s trying to download and choking on? (if that’s Chrome… no idea what debugging tools Safari provides)

I’m not surprised that Safari fails, as you mentioned that last week and I haven’t had a chance to look into it. On my own OS X system though, Firefox is perfectly fine, Chrome “works” but doesn’t load the textures for some reason so all you see is the solid glClear colour. On Linux, both Chromium and Firefox work here. Windows, no idea.

Lock up probably happens because it requests 250MB of memory for the asm.js heap. That is a lot, but the game seems to use that much once the whole map is loaded… Could just be webgl going boom though, I wouldn’t be surprised at that. Unfortunately all this is really hard to debug because, due to LLVM compatibility issues, I can’t actually compile the game without optimizations… >.>


#8

The 404 is just for favicon.ico. Didn’t notice the URL didn’t carry over into the saved log, sorry :slight_smile: