Providing the illusion of a contiguous type, backed by non-contiguous memory?

Don't think this is possible, but would love to be proven wrong!

I would like to stitch together chunks of non-contiguous memory and use them to provide the illusion to a client of an instance of a type allocated upon contiguous memory. That is, if you have some struct MyStruct with arbitrary internal structure, my service can provide you a PossiblyDiscontiguous<MyStruct> for you to call methods upon, load from fields, etc. In doing so, I would potentially lazily copy-on-access to make fields of the struct contiguous (allowing making a reference to a field to be a well-formed thing).

The closest thing I can find to what I want is the Deref and DerefMut traits, which basically provide some limited overloading of the . operator for smart pointers. However, Deref's method does not provide any indication of the field or method requested, which would be required for my intended purpose.

Is there a crate or language feature that anyone knows of to provide this kind of functionality?

Your description sounds very much like a transparent paging structure built in user-space on top of whatever paging structure the OS (if any) uses. While it's certainly feasible to build an arena with internal paging, is there sufficient motivation for that complexity? How about an alternate method where you "read" a structure into program-accessible memory, work with it via references, then either release it or "write" it back to whatever you're using as discontiguous backing storage?

You can definitely look at it as an additional layer of paging.

I'm working on a software transactional memory algorithm and am trying to provide the illusion of mutation of thread-shared objects by performing copy-on-write to a new immutable version, keeping the old version around as long as any transaction is still considering it (similar to multi-version concurrency control in databases). The goal is to allow the user to share data between threads with fast, almost completely unblocking reads. In addition, they would not have to worry about lock order or the hairy problems of lock-free concurrency, with their only burden describing what data items should be thread shared and where transactions upon those thread shared items start and stop.

I have an approach that doesn't require this discontiguous-backed object functionality (it is somewhat similar to the method you recommend), but the price I pay is either unnecessarily copying data that did not change or additional pointer indirections to arbitrary-ish locations. I was playing with the idea of expressing version updates that touched only portions of the object as providing only the small number of chunks that changed, with the remaining portions deferring to the prior, mostly-unchanged version. However, to do that, I would have to provide the illusion that these few changed chunks and the prior version were, holistically, one contiguous object.

This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.