I want to make it as efficient as possible by validating the bytes then doing unchecked memory offsets. This would be unsafe if someone used interior mutability to change the content. Is this possible?
Kornel is correct that the bytes of a &[u8] cannot change while that value exists, but note that reassigning the buf field of a MyThing value would still be problematic if you're doing unchecked stuff to it in safe methods. You have to make the field private and ensure that none of the code in your crate reassigns it after creation.