Is there any rust crate with the following properties ?
It takes some real language (i.e. not whitespace, brainf***, ...)
User types in an statement
crate transpiles statement into wasm32
wasm32 is JIT-ed/loaded-as-module (via say, wasmer or wasmtime)
corresponding x86-64 code is executed
result displayed back to the userr
This is basically a standard read-eval-print-loop , except during the 'eval' step, instead of running an interpreter; we transpile to wasm32, jit the wasm32, and execute the machine code.
I would prefer seeing this for lisp-like languages, but any 'real' language is fine; mainly I am interested in seeing how the transpile/jit step is used.
Hopefully not, considering that WASM is an unnecessary step. Better skip the WASM step and compile directly to Cranelift IR. Wasmer and Wasmtime are wrappers over Cranelift anyway.
A regular compiler takes source and produces executable. Then that executable is run.
A JIT compilation system takes source and interprets/compiles/runs it all at the same time. Compilation to native codes happens on the fly, as and when required as it runs. That is why it is called Just In Time.
What you have asked for there is a Just In Time system that does all the compilation first then runs the binary. That is to say an ahead of time system.
In computing, just-in-time (JIT) compilation (also dynamic translation or run-time compilations)[1] is a way of executing computer code that involves compilation during execution of a program (at run time) rather than before execution.[2] This may consist of source code translation but is more commonly bytecode translation to machine code, which is then executed directly.
In particular, note the part stating:
is a way of executing computer code that involves compilation during execution of a program (at run time) rather than before execution.
====
No. The WASM part allows JIT both in browser and on desktop.
That Wikipedia link for Just-in-time compilation only reinforces what I said already. Or a at least says it more clearly. Which does not look like your steps 4 and 5 in the OP.
But never mind. The key thing is your later statement that you want this to run in the browser. Hence the requirement for the WASM stage.
So if I understand correctly the requirement is for a REPL that accepts statements in some language, say lisp. Converts those compiles those statements to WASM and runs the resulting WASM.
I am not a Python / Ruby expert. Do they do dynamic x86_64 code generation, or is it just a
while let Some(instr) = get_next_instr() {
match instr {
.... => ....
}
}
^-- because this above is not JIT, as there is no code generation. All the x86_64 code was created at compile time (of the interpreter). The interpreter itself is not creating new x86_64 code on the fly.
I interpret as you do. With JIT the compilation to actual executable machine code happens at run time.
I guess we are talking at cross-purposes because you have two levels of what we might call "JIT" going on there:
The first is when you want to compile your input language statements, from the REPL, into WASM. Sort of at run time because such a compilation happens on every statement as they are entered rather than the whole program when the user has finished typing/editing it. That is steps 2, 3 and 4 in your OP.
The second is what ever JITing the WASM run time does as it executes those WASM statements.
My feeling is that the first level there is not JIT. It is compilation of language statements to WASM in this case. All other JIT's I have ever heard of go from source to actual native executable machine code, WASM is not native executable machine code.
The only actual JIT you have there is whatever goes on in the WASM run time when it does compile WASM to native executable machine code on the fly.
A language interpreter can be JITed or not. Should make no difference to the user apart from improved performance.
Traditionally an interpreter is a program that takes high level language statements as input and then performs whatever action they specify. The language the interpreter is written in need not be the same as the one it is interpreting. At no time does the interpreter generate actual machine code for the machine it runs on. I believe this was first done, or at least first made popular with Dartmouth BASIC.
Later we get interpreters with Just In Time compilation. Here they can take in a high level language source and can generate actual machine code from it. They then jump to that machine code to get the work done. My understanding is that they can do this statement by statement or function by function as and when control flow leads them there. Famously Javascript run-times in browsers started to do this some years ago. And Java I guess.
For this reason I don't see a source to WASM step as JIT.
Still, in the modern world we have so many layers of compilation, interpretation, emulation, virtual machine etc we can loose sight of what is real and what is not.
To my mind the only JITing going on there is in the WASM run time. I'm not quite ready to accept compiling the input source to WASM as JIT. That is ahead of time compilation.
I think we have binary searched our disagreement down to one line. Out of curiosity (I've lost track) -- is this an assertion I made, or is this just something you are arguing against on your own?
My assertion is that the only JIT going on is in the WASM run time. So if you want to see how the JIT works crack open the WASM run-time source and see how it does it. There is no need for the rest of the system you have described there.
Not that such a project would not be interesting anyway.
No where in that statement do I assert that "input -> wasm32" is JIT. In fact, in line 3, I call it "transpile", namely
3. crate transpiles statement into wasm32
4. wasm32 is JIT-ed/loaded-as-module (via say, wasmer or wasmtime)
5. corresponding x86-64 code is executed
in "transpile/jit", "transpile" is referring to 3, while "jit" is referring to 4 & 5
I do acknowledge that I should have said "transpile/jit stepS" instead of "transpile/jit step"
I am currently not interested in seeing how the wasm32 -> x86_64 jit works -- previously played with that, and it was quite complex.
I am interested in how the "transpile/jit step is used" in the sense of how it is integrated into the rest of the runtime. So in particular, even after getting JIT to generate the x86_64, there is still all types of work to be done in making sure the stack / heap / ... are properly setup before the generated x86_64 code is called, and there may be tear down related tasks to be after the x86_64 code is called. That is what I mean by "transpile/jit step is used"
Now, I do realize this all sounds a bit messy with lots of grunt work -- which is precisely why I was hoping there was some crate which already figured out a nice way to glue all this together that I could just study.
How might this actually work? My understanding of your requirement so far is:
User types a statement in some language (as yet unspecified)
Your compiler compiles that statement into some WASM.
That WASM is passed to a WASM run time for evaluation. And it returns the result(s).
The results are displayed.
Continue from 1).
Traditionally this is called a READ, EVAL, PRINT, LOOP. A REPL. Not a JIT, but never mind.
My first though is that somewhere you need to maintain a big glob of memory that will hold the current execution state of the program the user is entering. That is the state of all the programs global variables and its stack. Call it a "context". Every time you fire up WASM to evaluate a newly entered statement you will have to also pass in that context to the WASM run time.
Example:
User types:
a = 3
Which is compiled to WASM.
2) WASM evaluates that and sets a to 3. Returns the value 3.
3) The 3 is printed out.
4) User types:
a = a + 1
Which is compiled to WASM.
5) WASM evaluates that and sets a to 4. Returns the value 4.
6) The 4 is printed out.
Well, the evaluation at 5) depends on the value a having been remembered from step 2) somewhere. That somewhere being the program context.
Anyway, I think we have the same problem statement in mind now.
I'm really looking for a crate, not a discussion, since discussions (1) aren't machine executable and (2) they are sufficiently vague that it is hard to point out how they are wrong.
The thing that surprises me about the apparent non-existence of such a crate is that I can't find a simple "scheme -> wasm32" interpreter in Rust.
Basically, a Rust scheme -> wasm32 -> run (either browser or wasmtime or wasmer) 'JIT interpreter' would have all the components I am looking for.
A quick google around turns up some stories of peoples adventures in writing interpreters in Rust. I did not see anything that fits your bill exactly.
If it were me, ignorant as I am of language interpreters/compilers, I'd be rewriting in Rust my one and only attempt at a compiler, some years ago, and adapting it to to generate WASM instead of x86.
That compiler would itself be compiled to WAS to run in the browser.
Then I would create the user interface in Javascript to run in the browser, passing user input to the compiler.
I did something along these lines for fun a while back. I took a compiler for the Spin language which was written in C++. Compiled it with Emscripten into asm.js (could be WASM now a days). Then ran the whole show in the browser. Syntax highlighted editing and all. It was basically a Spin IDE in the browser.
I don't know if you have skimmed the wasm spec. It is remarkably close to scheme. I am still convinced the simplest solution is scheme -> wasm32 in Rust (in fact, I think it might be possible to do this in < 2000 lines of code).
