Point cloud visualizations?

I would like to make efficient visualizations of big point clouds (around a million points), similar to what is done in DSO visualizations, but in a web browser (compatibility with wasm).

I did not find a library for point cloud rendering and am not very familiar with graphics in general. Do you think this is doable? how hard? where should I start looking?

This is completely do-able. At this point, because you are entering a new territory (WASM + Rust + Gfx rendering), I would look for a rendering library. Or, write your own using vector calculus that handles matrices of data (point clouds in this case). I've used Potree before (js), so if you managed to write this in WASM + Rust, then it would be faster than some of the other renderers out there. You can do it!

1 Like

If you can find a #no-std rendering library, then it would be most compatible with WASM

Thanks for the supportive words :wink: I've first tried to use kiss3d which seemed aligned with my goals of a simple solution and has support for wasm. With the help of @sebcrozet who made an example persistent point cloud renderer, I could render my point cloud during the visual odometry computations. Below is a video showing the result on one sequence.

Unfortunately, the performances dropped when compiled to wasm and I am down to roughly 1fps on my laptop with chromium at a million points. I'm not sure what is the exact reason. Maybe it is due to some data marshaling at the wasm frontier, maybe it is something else. I think I might have a look at how to render point clouds with gfx / rendy / web-gpu.

1 Like

PS: let me know if you know how to do that or you are aware of good reads regarding this.

This performance difference is quite surprising. Using Kiss3d, I did in the past some point cloud visualization for 3D streaming with millions of points (and a pretty complex vertex shader) and it ran at 60fps (though it will depend on your graphics card of course).

  • Did you compile in release mode when targeting WASM?
  • Does the framerate goes back to 60fps if you disable the point rendering (by making camera_and_renderer_and_effect return None instead of the points renderer)?
  • Did you try with another browser (firefox or google chrome)?
  • Are you certain your browser is using your GPU for rendering? You can check this by going to https://webglreport.com/

Yes, all green on webgl 1 and 2. I have intel HD 5500 integrated graphics. I've run cargo web start --release on the persistent_point_cloud.rs example with same setup as the wasm example. I've "manually" timed the loops to a million points. In native, all 100_000 chunks take roughly 3 seconds. In wasm with firefox and chromium, the timings are roughly:

0 -> 100_000: 3s
100_000 -> 200_000: 4s
...
900_000 -> 1_000_000: 12s

And when 1_000_000 points is reached, I have around 10 fps in firefox and 2 fps in chromium when rotating.

Yes, here is the perf profile of firefox. You see the fps go from 30, rapidly dropping to 10 and going back to 60 fps when it reaches 1_000_000 points and start returning None for the renderer in this function.

kiss3d-wasm-perf-firefox

I just run threejs interleaved buffergeometry example which displays a cube of 500_000 points. I'm getting 25 fps on this example. So I guess I'm doomed XD. What really surprised me is that there is such a big performance drop (one order of magnitude since the native build runs 60 fps smoothly up to 2 million points and the web build runs at 10 fps for 1 million points).