Debugging of Non-Deterministic Behavior

[koehlma]

Hey there,

I am pretty new to Rust and just managed to write a program that seems to behave non-deterministically. Sometimes it provides the correct result, but sometimes, it does not. I am not using unsafe or any concurrency in my code. Presumably, this behavior is rooted in my usage of HashMaps and HashSets. Do you have any tips on how to debug such problems? Is there a way to fix the seed used for hashing such that I can get reproducible results and track down the problem?

To provide you with a bit more context: I am working with Markov Decision Processes (MDPs) and essentially read a description of an MDP with variables from a JSON file using Serde. This description is then turned into a more efficient representation following Building fast interpreters in Rust. The program then explores the entire state space of the MDP and returns the number of states. Unfortunately, this number changes between multiple executions (producing what seems to be a finite set of wrong results and sometimes the correct state count).

Cheers,
Maximilian

[cole-miller]

You can build a deterministically-seeded HashMap using the with_hasher constructor.

[koehlma]

Is there a way to do this globally for the whole program? Some of the HashMaps are created by Serde.

[cole-miller]

That's trickier. I think you can deserialize into a FnvHashMap (or FnvHashSet) instead to get deterministic behavior, but I might be misunderstanding the documentation/source code.

ETA: there's also IndexMap which is non-deterministically seeded but offers a consistent order of iteration, if you think that's where your problem lies.

[H2CO3]

serde_json has a linked_hash_map feature IIRC, which preserves entry order. Maybe crates for other serialization formats have similar features, too.

[qaopm]

Never tried but if you're on Linux, you might be able to use LD_PRELOAD to provide consistent random results. Some details: https://github.com/rust-lang/rust/pull/78785 .

[mbrubeck]

The rr debugger (which also requires Linux) may also be useful if you want to examine one of the incorrect runs inside a debugger, reproducibly.

[koehlma]

Thanks for your tips. Unfortunately, I am stuck with Windows for now, but I might give this a try in WSL sometime. I was able to track down the problem by replacing individual maps with IndexMap.