I also know there are already dynamic code generation options in Rust via: cranelift-simplejit, dynasm, wasm.
Now, here is my question: can eBPF be used as a JIT target for Rust code doing dynamic code generation ? If so, are there any well known examples? If not, what are limitations of eBPF that make it bad for generic-dynamic-code-generation / nothing-to-do-with-packet-filtering tasks ?
Rust BPF targets were recently merged, and they should reach stable for Rust 1.54. However, these are at tier 3 to start, so the standard library builds won't be shipped by rustup yet.
The "emit bitcode, codegen in the linker" approach is identical to the cuda target, which is tier 2. There are a couple of reasons for emitting bitcode. Like NVPTX, BPF supports a subset of Rust (eg, arguments can only be passed in registers, up to a maximum of 5), so it makes sense to push codegen after linking has happened to avoid potentially hitting codegen failures on unused code. Also when targeting older kernels, bpf-linker needs to run more aggressive inlining/loop unrolling optimizations.
^-- What is going on here? My understanding of eBPF is very limited; my understanding of Rust backends is also very limited.
Is the above saying that eBPF, for whatever reason, is not a generic VM, and it's limited, (like CUDA) ? The comment about only passing 5 args is also confusing me.
===
So I guess my question here is: is the tier3 backend saying "any Rust -> eBPF" or "limited subset of Rust -> eBPF" ?
Correct. eBPF is not meant to be a generic VM, but limited to ensure that a userspace process can't crash or hang the kernel by supplying malformed or malicious eBPF bytecode. Especially the no hang guarantee requires that eBPF as supported by the kernel is not turing complete. The not crash guarantee is implemented by having a tiny stack (512 bytes) and running a verifier that checks at every load instruction the stack location is initialized, that arrays are never read out of bounds, ... Both combined place very big limitations on what is supported.