So I put SurrealDB into my project because I have some data better modelled as a graph and, well, just because I wanted to dip my toes into graph database world.
This works well but it's driving me mad as a full build of my workspace now takes over 5 mins on my MacBook M3, over half that time is building SurrealDB crates.
Then I find a build of my workspace ran for over 50 minutes on a cheap DigitalOcean droplet before crashing out for lack of memory. There would be no hope of building it on a Raspberry Pi or such.
So:
Why is building SurrealDB so demanding of resources?
Is there any other graph database I could use?
I'm leaning towards thinking I might as well use SQL and live with the problems of doing graph like things there.
Release or debug builds? is LTO on or off (LTO makes builds very slow)?
I build it in debug and release on my PC (Intel core i7-12700K),
Debug takes 27 seconds, release takes 37, both use ~2GB.
I can't reproduce the issue on x86-64.
Looking at the benchmarks of the 2 CPUs I'd expect yours to take 85-150% of mine, not >500%.
If a "cheap droplet" is the 4 or 6 dollar droplet then it has less than 2GB, so I wouldn't expect it to compile successfully.
if it's the 12 or 18 dollar droplet then it depends on how much memory the rest of the system uses.
The 24 dollar droplet has more than enough RAM, but it's not cheap.
It's a relatively beefy crate, with up to 1.5M lines(!) including all dependencies, but that should only be an issue for working on the crate itself, not using it as a dependency.
I'd suspect you're somehow including multiple TLS implementations and/or building openssl from source, those are easy ways to bloat times with those dependences.
Try using cargo build --timings (after cargo clean) to get a nice graph of exactly what's taking time in your build, you're looking for any "skinny" periods with only one crate building and holding up everything else.
Our site is based around ArangoDB. They've gone down the dark path. Currently I'm looking at Tantivy to replace ArangoSearch, but perhaps one day we'll need to look at alternatives to ArangoDB itself.
Interesting. On my MacBook M3 building my workspace takes 9 minutes with --timings. The chart shows surrealdb core starting after pretty much everything else 240 seconds into the build and finishing codegen at 510 seconds. A time of 270 seconds or 4.5 minutes.
Mostly in parallel with that it builds surrealdb which takes 3.2 minutes.
Together they are taking over half the time of the build.
OK. I fixed it. Removed SurrealDb use sqlx instead.
For my data the graphiness of it is not so much a few joins cannot handle and we only have a small , fixed set of queries to run on it. I don't need 2 million lines of graph database code for that.
My workspace now builds on my MacBook M1 in 3 minutes and on the DigitalOcean droplet in 13 minutes.
