Tips for slow compilation / iterative development

Some of us (self included) are coming from scripting languages where performance might be slow at runtime but re-"compile" is very quick... years of working that way (Flash,Unity,JS) have ingrained habits where I'm really used to making changes and hitting save / seeing results within seconds, often milliseconds.

In Rust with WebAssembly, it's just slow. Perhaps it's an issue of getting webpack to pick up the changes faster and that will improve- but whatever it is, changes easily take 5-10 seconds minimum till I can see them on the screen, even for a small proof-of-concept, and I expect it's going to get worse as a project grows (or maybe not, maybe this is just the baseline and it stays constant...)

For a C++ gamedev programmer, this might seem like a silly complaint... big C++ projects can take MUCH longer to recompile and see results, even with caching of artifacts (or so I've heard... maybe that's not right? Surely one of the points of Jai is that recompile should be quick...).

Not sure what I'm asking exactly - but any tips for coping or changing my habits would be appreciated. e.g. maybe I should just get into the headspace of making lots of changes before seeing results. I dunno...

Just putting this out there in case it resonates :slight_smile:

I can offer some suggestions, even if they're as much about refining the question as they are about other improvements. The main thing is to think about breaking the problem apart, and understanding what it is that you need to see faster/sooner, under which circumstances.

The first simple thing, which you're probably doing already, is to have an IDE with iterative error highlighting and compiler / clippy lints. That takes care of the save->compile->error cycle (and that takes care of more kinds of errors than it might in JS or other interpreted langs).

That gets you to seeing the effects of logic changes. You perhaps don't need to run the whole program to see the effect of a small change, especially a logic change. Write a small test and call that (VScode has a 'click to run just this test' hover). At least some of these tests may be just as useful run natively, and can bypass the extra steps for WASM.

If what you're changing is frontend visual behaviour that really needs to be run to see the result, so be it - but you can probably short-circuit some cases before that, as above.

WASM is going to add some unavoidable extra steps. The tooling is improving and changing rapidly, at least. I think I recall seeing some comments about hot-loading code that might be relevant and helpful, but others will have to comment on those because it's not something I've looked into (and could be misremembering the context)

1 Like

dakom,

Yes it resonates. I have no particular suggestion for what you are doing but some random thoughts having spent a lot of time working in C++ and Javascript in recent years and now testing the waters of Rust:

Tradtionally there were two appraoches to writing code:

  1. For compiled code, compilers and builds could be so slow one soon learned that it's best to know your language very well, spend time writing your programs/modules, spend much time inspecting what you have written, as if you wre a compiler, then submitting the job for compilation. My first high level language code was in Algol, a compilation took all night. Seriously, that was from submitting a stack of cards to the university computer operators to getting the printed listing back in the morning. With that turn round time one was very careful.

  2. For interpreted code, we had BASIC, just hack it into the termoinal and run it. Instant feed back for errors. Terrible slow run time for significant programs.

This basic scheme of thing continues to this day with the likes of C++ and Rust vs JS, Python etc.

Except I find now that method 1) is broken. C++ and Rust are such complex languages that it is impossible to know that anything will compile first time, or second, or third... C++ because it is just a massive ball of complexity of syntax and semantics, Rust because as well as language complexity there is all the work of the borrow checker and type checking to contend with. (Mind you, that could be because my old brain is slow to handle it all now a days.)

I also find method 2) can be broken as well. I was intrigued by your webpack experience. I have one project using JS, React and webpack and it drives me nuts. I never figured out how to stop it rebuilding and repacking everything on even the smallest change. That takes a minute or two. Worse than working with C++ or Rust! I came to the conclusion that something has gone horribly wrong with webpack, Javascript was not designed to be subject to building like that. webpack defeates the whole point of JS being an interpretted language designed for ease of use and quick development.

More practically...

Now a days for C++ and now Rust I have fallen into a rythm of making small changes, one line at a time, one function, one method. Or if there are more global changes do one of those at a time. With the help of Make for C++ and Cargo for Rust only the changed module is recompiled and turn around times are tolerable.

To get into this rhythm and achieve quick dev iteration cycles one needs to get organized. Keep modules small. Minimize dependencies between modules. All good software engineering practice anyway.

I find that this method works better for Rust than C++ as the error messages are much more helpful. It's the only way I can work in Rust as I am still very unfamiliar with it all.

Have your IDE setup properly. Having the IDE highlight problems before you get near the compiler is a boon.

Make use of "cargo check". It shakes out the syntax and other errors without spending too much time on actually trying to compile anything.

I find trying out new code/ideas in a separate small module as an experiment speeds things up greatly. When it's working nicely then adopt it into the main work.

When you get going like this dev iteration cycles can match those of JS and Python.

Don't forget, the Rust compiler is so fussy that your code is far more likely to work properly if you ever manage to get it to compile cleanly. So you are actually saving time you would otherwise be spending testing your JS properly after every little change.

1 Like

a Rust board is probably not the right place to discuss webpack config in detail, hehe - but just a heads up, there are things like Hot Module Replacement and even with rebuild multi-process plugins and things that can make it much faster...

Hmm.... I'm using VSCode and it's not very fast for me.... sometimes it even hangs on RLS :\

I mentioned this in the past and I don't think there's much to do about that (as @dcarosone pointed out - the tooling is improving) , but I hear it - solving things at the compiler level and not needing to see it is definitely part of the problem, thanks!

I neglected the WASM part of the question.

So far my experience of web assembly is limited to a few experiments. The first of which was this: https://otaniemi.conveqs.fi:3000/public/fibo.html. Which despite what it says uses Rust and WASM.
The second of which is a much bigger project compiling C++ to WASM.

My take on this so far is that it's best to develop ones C++/Rust as a stand alone C++/Rust program first. Use the regular language tools and techniques to develop and test it.

When one has working, tested code then it's time to start building it as WASM and integrating to the web project. At that point you have no rebuilding to be doing constantly whilst working on the JS etc.

1 Like

It's a good idea, especially for library code

Unfortunately, using the DOM for UI, or more generally using the web api's means having very generic abstractions that come with their own headache/cost.

But perhaps there is a middle ground of separating it out more. food for thought...

Ah yes. I had not really thought of things like that.

My assumption was that one would continue to JS for all the DOM/web API stuff. WASM would be another tool in the box for when heavy weight computing is required.

Mind you, I started to toy with the idea of using Rust/WASM for deploying server side processes and even remote embedded system code. It seemed to me that Rust provides the safe. reliable language, WASM provides the sandbox for it to run in. No messing with Docker containers and such required!

I was amazed to find this is not such a crazy idea and there is now WASI being developed for exactly that!

Aside: As for webpack, I'm done with it. That is such a complex mess nobody can pay me enough to want to spend time on it.

1 Like

Yeah - and even though one can use Rust for DOM stuff (e.g. Dominator, or web-sys in general), I'm finding it nice to use lit-html for rendering the app state and keeping that side in JS like you say.

The problem is that it still needs to exist as one project. So if I have the position of an element being entirely controlled on the Rust side, to see that change on the screen means the browser needs to reload the WASM (or even if it were maintained separately, be notified of the change via websockets or something, which I'm guessing would be slower than the native file watching)

I just changed things so it's no longer loading the wasm through webpack, but is now loading it completely externally. Hopefully that'll help a bit... but I think ultimately, there's no getting around it, I'm going to have to change my habits and start "thinking like a compiler" more :slight_smile:

I'm not sure if I'm imagining things, but it feels much snappier now o_O