Hello everyone
I have a design problem I am trying to solve using Rust that I have previously solved for relational databases using SQL for many years now (example: SQL Server checksum, checksum_agg, binary_checksum functions, or Apache Spark SQL built-in functions).
I am aware of Rust std::hash, but I dont know if this is best or even applicable.
Consider an in-memory relational-like table (using SQL terms) which has a bunch of rows each row having a number of columns. Let us say each row is a Rust struct or enum or tuple and a group of such rows is represented in-memory as a Rust BTreeMap (to ensure same sort order) where K is the BtreeMap Key and V is a Rust struct/enum/tuple with data columns.
I would like to be able to compute a checksum across all data-fields in V so I can store checksum as part of the V itself (like a hidden-signature/fingerprint column). I want to use checksum to detect if any of V fields is changed because recomputing the checksum would mean a new checksum will not match the stored original checksum.
So I want to use the checksum only for detecting data changes in any field value of the struct/enum/tuple and to be able to simply compare two rows for equality/not-equality (I dont need > greater or < less support, only need == and != support).
My second (related) problem is ability to thus checksum the entire table (entire BTreeMap) with the above Keys and Values(structs) so I can quickly compare the entire table to detect any data changes in any of the rows or to detect if some rows were removed or some new rows were inserted. I want to quickly determine it by comparing two tables "total-checksums", even of row count may match (a delete followed by insert but new row contents are different).
I am not planning to use this for security/crypto purposes, my only use is for data equality and for data-changed detection purposes. The checksum function needs to be inexpensive/fast with very low probability of checksum collisions so that I dont end up with undetected data change (i.e. producing same checksum value for different struct or rows of structs).
Essentially (as a former C-programmer) it is a matter of taking any struct/tuple/enum in its entirety represented as a slice of unsigned bytes [u8], computing a checksum to compare against another instance of same type.
Or perhaps there are other good Rust based solution that you may suggest.
I prefer to use existing checksum-library-crate rather than writing one myself because bit-twiddling is no longer my specialty. LOL.
Many thanks for any guidance.