Bit level packing library, request for feedback

If anyone else is tired with cryptic bit shifting, byte swapping and general spaghetti code when dealing with registers, fixed packets and stuff like that, you might want to take a look at a preview of my packing code generator:

https://github.com/hashmismatch/packed_struct.rs

LSB or MSB bit positioning, little and big endian integers, arbitrary integer widths (at least for MSB), arrays and a documentation generation should be the standout features.

I've written it as a part of a sensor interfacing library, so I've dogfooded as much as possible, but any feedback is welcome. Hopefully coming soon as well.

8 Likes

Looks great! I especially like the ability to use enums for flags. I made a much simpler version of this bit-tools for emulator development. Only thing I'd like to see is some benches of manual bit shifting and masking vs your lib.

Is the future generics with type-level integral values feature going to help improve some of your APIs?

1 Like

For this iteration of the library, I haven't looked into the final, optimized assembly code as to me the API and the self-documenting nature are the primary motivators. However, I've previously looked into the final output for an ARM microprocessor target and LLVM did a fine job of optimizing away the deep call stack.

Of course, the point of a code generation library is that you can do optimizations after you've written 500+ structs that actually use it :slight_smile:

1 Like

The end-user API would probably stay mostly the same, other than being able to remove unit types like "Bits10" and "Bytes3". Integer<u8, Bits10> might become Integer<u8, 10>, but that's probably it at the moment.

However, behind the scenes, a lot of things would get simplified. Rust's current array type is very limited with most of the useful traits being implemented only up to 32 elements wide. I've decided to embrace its limitations anyways, instead of using byte slices instead, for additional compile-time safety. The lib's codegen emits a lot of byte array copying and such, but LLVM does a fine job of optimizing that.

1 Like

Beside the API, the future integer generics must allow you to avoid allocating the bounds as run-time values, and keep them as compile-time only template constants. This also allows this lib to statically refuse mixing types with different intervals (because they become different types! And this is how it should b in my opinion, and I think this is how Ada type system works).

Another important factor is how much time/memory Rustc takes to handle such values. If I use 20_000 different ranged integral values in a large Rust program how much time/RAM does it take to compile that stuff? :slight_smile:

This lib's Integer type is something like Integer<T, B> { num: T, bits: PhantomData<B>> }, where T is usually u16, u32 or something similar and B is a unit type denoting the number of bits, I would assume that runtime performance and memory use is exactly the same. A bit mask is applied whenever converting from an outside integer value. Compilation time isn't an issue for me.

My primary motivation for this project were device drivers, as their reference sheets contain the registers, values and bit positions in all sorts of ways. I'm quite happy that writing this Rust structure (USB Power Delivery driver):

image

..emits a documentation table that provides information for non-developers:

Benchmarks, compilation times, memory usage, etc... are currently not my personal priorities. Runtime traceability, self-documenting expressive types and struct/field attributes with ease of use are things that excite me. It's what's missing in low-level systems programming, badly.

6 Likes

I've published the first release on Crates, so if anyone wants to poke it within their own project, it's quite easy to do so now! Works on stable Rust. https://crates.io/crates/packed_struct

2 Likes

Version 0.2.1 released, with lots of useful ergonomics changes. I've also set up a little homepage to show off the crate's workflow in various stages - source code, documentation, runtime inspection.