Can I statically ensure that an object is "owned" by another?

sboo · November 8, 2022, 3:43pm

I've got something like this, where a "writer" owns a buffer, and allows allocating things within:

let mut writer = Writer::new();

let handle = writer.create::<Something>();
writer.create_with_param::<AnotherThing>(handle);

A handle is specific to a given writer, and you cannot do anything with it except pass it back to the writer.

But I would like to avoid mixing handles between writers:

let mut writer1 = Writer::new();
let mut writer2 = Writer::new();

let handle1 = writer1.create();
writer1.create_with_param(handle1); // This is OK
writer2.create_with_param(handle1); // I would like this to not compile

Is there a way to achieve this ?
I'm using lifetimes in the full code, so the handle is something like Handle<'writer>, but then with:

fn create<'writer_borrow, 'writer: 'writer_borrow>(&'writer_borrow mut self) -> Handle<'writer>

the calling code gets to choose any lifetime that suits it, so 'writer is too big, and for example fits writer2 in the example above.

H2CO3 · November 8, 2022, 4:25pm

Make illegal states unrepresentable.

If your requirement is that "method X of object Y must always take object Z as parameter", then it's not really a parameter! It's not something that your caller can choose; instead it is an internal invariant that must be upheld by the type itself.

So, don't take a parameter – store it instead!

struct Handle<'a> {
    parent_writer: &'a mut Writer,
}

impl Handle<'_> {
    fn create_with_param(&mut self) {
        // `self.parent_writer` is always the correct writer
    }
}

impl Writer {
    fn create(&mut self) -> Handle<'_> {
        Handle { parent_writer: self }
    }
}

fn main() {
    let handle1 = writer1.create();
    handle1.create_with_param(); // this will always use `writer1`
}

sboo · November 8, 2022, 6:14pm

The issue is that I need several handles alive at a given time. They need to be passed back to the writer, and some of the create methods take several handles. So I don't think I can have them borrow the writer mutable like you suggested, nor own it.

truppelito · November 8, 2022, 6:46pm

I'm not sure this works for you, and maybe requires some more manual work than ideal, but... would it be possible that

writer is generic over some type T
when you create a writer, you set T to some concrete zero-sized struct that you create specifically for that writer. This concrete struct type in essence becomes your writer identity.
handler is also generic over T and when writer creates a handler, the returned handler has the same concrete T type
when you pass the handler to the writer, you use type constraints to guarantee that the identity type T is the same for both the writer and the handler

I will try to add a code example in a moment.

tczajka · November 8, 2022, 6:53pm

My solution.

struct WriterData {
    x: i32,
}

struct Writer {
    data: Arc<Mutex<WriterData>>,
}

struct Handle {
    writer_data: Arc<Mutex<WriterData>>,
}

impl Writer {
    fn create_handle(&self) -> Handle {
        Handle {
            writer_data: self.data.clone(),
        }
    }
}

impl Handle {
    fn write_x(&self, x: i32) {
        let mut data = self.writer_data.lock().unwrap();
        data.x = x;
    }
}

truppelito · November 8, 2022, 6:55pm

Code to idea from above:

use core::marker::PhantomData;

struct Handle<T> {
    _marker: PhantomData<T>,
    // etc
}

struct Writer<T> {
    _marker: PhantomData<T>,
    // etc
}

impl<T> Writer<T> {
    fn new() -> Self {
        Self {
            _marker: PhantomData
        }
    }

    fn get_handle(&mut self) -> Handle<T> {
        Handle {
            _marker: PhantomData
        }
    }

    fn use_handle(&mut self, handler: &Handle<T>) {

    }
}


struct Marker1;
struct Marker2;

fn main() {
    let mut writer1 = Writer::<Marker1>::new();
    let mut writer2 = Writer::<Marker2>::new();

    let handle1 = writer1.get_handle();
    writer1.use_handle(&handle1); // Compiles
    // writer2.use_handle(&handle1); // Does not compile

    let handle2 = writer2.get_handle();
    // writer1.use_handle(&handle2); // Does not compile
    writer2.use_handle(&handle2); // Compiles
}

Potentially, the creation of the writer along with the marker type (the struct MarkerX; and let mut writerX = Writer::<MarkerX>::new(); in the code) could be combined, hidden and automated using a macro. Something like let mut writer1 = new_writer!("params to Writer::new function") where every time the macro is called, a new struct MarkerX; is created specifically for that writer.

truppelito · November 8, 2022, 7:04pm

In fact, here is the code for that:

use core::marker::PhantomData;

struct Handle<T> {
    _marker: PhantomData<T>,
    // etc
}

struct Writer<T> {
    _marker: PhantomData<T>,
    // etc
}

impl<T> Writer<T> {
    fn new() -> Self {
        Self {
            _marker: PhantomData
        }
    }

    fn get_handle(&mut self) -> Handle<T> {
        Handle {
            _marker: PhantomData
        }
    }

    fn use_handle(&mut self, handler: &Handle<T>) {

    }
}

macro_rules! new_writer {
    () => {{
        struct Marker;
        Writer::<Marker>::new()
    }};
}

fn main() {
    let mut writer1 = new_writer!();
    let mut writer2 = new_writer!();

    let handle1 = writer1.get_handle();
    writer1.use_handle(&handle1); // Compiles
    // writer2.use_handle(&handle1); // Does not compile

    let handle2 = writer2.get_handle();
    // writer1.use_handle(&handle2); // Does not compile
    writer2.use_handle(&handle2); // Compiles
}

tczajka · November 8, 2022, 7:08pm

This doesn't work:

fn main() {
    let mut writers = Vec::new();
    for _ in 0..10 {
        writers.push(new_writer!());
    }
    
    let handle = writers[0].get_handle();
    writers[2].use_handle(&handle);
}

truppelito · November 8, 2022, 7:28pm

Hmm I see. The macro I proposed is flawed. Either: 1) don't use the macro and provide types manually (not as practical), 2) use the macro but avoid these cases (not guaranteed) or 3) fix the macro (although I don't know how, but maybe there is a way to guarantee a new type on every invocation).

EDIT: maybe const generics can also be used (at least in the non-macro case), which I find simpler:

use core::marker::PhantomData;

struct Handle<const X: u64> {
    // etc
}

struct Writer<const X: u64> {
    // etc
}

impl<const X: u64> Writer<X> {
    fn new() -> Self {
        Self {}
    }

    fn get_handle(&mut self) -> Handle<X> {
        Handle {}
    }

    fn use_handle(&mut self, handler: &Handle<X>) {

    }
}

fn main() {
    let mut writer1 = Writer::<1>::new();
    let mut writer2 = Writer::<2>::new();

    let handle1 = writer1.get_handle();
    writer1.use_handle(&handle1); // Compiles
    // writer2.use_handle(&handle1); // Does not compile

    let handle2 = writer2.get_handle();
    // writer1.use_handle(&handle2); // Does not compile
    writer2.use_handle(&handle2); // Compiles
}

EDIT EDIT: regarding the vector case, that seems like a flawed example from the start. The point is to have compiler guarantees. Once you put a few writers into a Vec, the compiler is no longer able to help you regarding which writer you can use for a specific handle, right? At most, you can check at run time with some kind of cast or relfection check. But the compiler should at least give you an error to state that the writer-handle pair are not guaranteed to match, so the macro is still wrong. Personally, I think I would use const generics here.

sboo · November 9, 2022, 6:11am

The vector case is similar to my first example: the issue is that several writers have the same type, so the Writer<X> is not enough to prevent mixing them. Also, given a writer, several handles can be used; there are several create(...) methods.

sboo · November 9, 2022, 6:14am

The easier way would be if I somehow could have the "actual lifetime" as a parameter of the writer, rather than "any suitable lifetime", making each instance have its own type. (That maybe wouldn't solve the case with the vector, but if it works for other cases, that would be ok)

sboo · November 9, 2022, 6:39am

The real code looks very much like this (sorry I should have posted that from the start):

use core::marker::PhantomData;


struct Writer<T> {
    ids: Vec<usize>,
    phantomdata: PhantomData<T>,
}

struct Handle<W, X> {
    id: usize,
    phantomdata: PhantomData<W>,
    phantomdata2: PhantomData<X>,
}

struct SimpleA;
struct SimpleB;
struct Composite;

impl<T> Writer<T> {

    pub fn create_simple_a(&mut self) -> Handle<Writer<T>, SimpleA> {
        self.ids.push(1);
        Handle { id: 1, phantomdata: PhantomData, phantomdata2: PhantomData }
    }

    pub fn create_simple_b(&mut self) -> Handle<Writer<T>, SimpleB> {
        self.ids.push(2);
        Handle { id: 2, phantomdata: PhantomData, phantomdata2: PhantomData }
    }

    pub fn create_composite(
        &mut self,
        ha: Handle<Writer<T>, SimpleA>,
        hb: Handle<Writer<T>, SimpleB>,
    ) -> Handle<Writer<T>, Composite> {
        let id = ha.id + hb.id; // Some kind of construction in the real code here instead
        self.ids.push(id); // Some kind of allocation in the real code here instead
        Handle { id, phantomdata: PhantomData, phantomdata2: PhantomData }
    }
}

struct DomainX;

pub fn typical_usage() {
    let  mut w = Writer::<DomainX> { ids: vec![], phantomdata: PhantomData };
    let ha = w.create_simple_a();
    let hb = w.create_simple_b();
    w.create_composite(ha, hb);
}

pub fn wrong_usage() {
    let  mut w1 = Writer::<DomainX> { ids: vec![], phantomdata: PhantomData };
    let  mut w2 = Writer::<DomainX> { ids: vec![], phantomdata: PhantomData };
    let ha = w1.create_simple_a();
    let hb = w1.create_simple_b();
    w2.create_composite(ha, hb); // I'd like this to not compile, since ha and hb were not obtained from w2, despite otherwise having the correct type
}

The create_* methods are generated by macros.

sboo · November 9, 2022, 1:55pm

I managed to get something satisfactory using closures and the fact that each closure has its own concrete type. With:

struct Writer<T> {
    ids: Vec<usize>,
    phantomdata: PhantomData<T>,
}

struct OpaqueBox<T: FnOnce()> {
    phantomdata: PhantomData<T>,
}
impl<T: FnOnce()> OpaqueBox<T> {
    pub fn new(f: T) -> Self {
        Self { phantomdata: PhantomData }
    }
}

Now this properly fails to compile:

pub fn wrong_usage() {
    let  mut w1 = Writer::new(OpaqueBox::new(|| ()));
    let  mut w2 = Writer::new(OpaqueBox::new(|| ()));
    let ha = w1.create_simple_a();
    let hb = w1.create_simple_b();
    w2.create_composite(ha, hb);
}

The OpaqueBox is just there to force a move of the closure (in case a caller was tempted to reuse the same closure twice), since a regular Box or lack of it allows T to be just a reference.

The code to create a writer is quite ugly with this, but I might just use a macro like new_writer!() for that.

H2CO3 · November 9, 2022, 2:09pm

That can unfortunately be broken trivially by coercing to fn():

let mut w1 = Writer::new(OpaqueBox::new((|| ()) as fn()));
let mut w2 = Writer::new(OpaqueBox::new((|| ()) as fn()));

This compiles again.

sboo · November 9, 2022, 2:21pm

I think I'll go with something like this:

macro_rules! new_writer {
    () => {
        unsafe {
            Writer::new(OpaqueBox::new(|| ()))
        }
    };
}

I already have macros for the whole thing anyway.
With this, anyone using the macro cannot get it wrong.

Another trick/hack that I'm using is to declare some functions (here new()) unsafe, and have the macro-generated code call it knowing that. (Since the macro-generated code lives in the client crate, I cannot use regular access control).

Again, that leaves an opening for a really motivated user to break things for themselves, but my goal is to prevent accidental mistakes, rather than malicious manipulation.

2e71828 · November 9, 2022, 2:44pm

sboo:

I'm using lifetimes in the full code, so the handle is something like Handle<'writer>, but then with:
fn create<'writer_borrow, 'writer: 'writer_borrow>(&'writer_borrow mut self) -> Handle<'writer>
the calling code gets to choose any lifetime that suits it, so 'writer is too big, and for example fits writer2 in the example above.

Take a look at the qcell crate. In particular, if you make Writer contain an LCellOwner and Handle an LCell, I think it will do what you want.

cuviper · November 9, 2022, 4:46pm

There are still ways to hide and share that anonymous type.

fn hack_same_writer_type() -> Writer<impl Any> {
    new_writer!()
}

quinedot · November 9, 2022, 9:31pm

This thread reminds me of various others I've seen about trying to produce singletons (e.g. for embedded projects). All the type-based approaches basically boil down to "for a given span of code with all parameters monomorphized, there can be only one type, because Rust is statically typed". So if you're in a loop for example, a given closure will have a unique type compared to other closures, but the same type every time through the loop.

So to get it wrong with your macro, I just need to have something like

// Off the cuff but I think you'll get the idea
let mut writer1 = None;
let mut writer2 = None;
loop {
    let writer = if writer1.is_none() {
        &mut writer1
    } else if writer2.is_none() {
        &mut writer 2
    } else {
        break
    };

    *writer = Some( your_macro!() );
}
let writer1 = writer1.unwrap();
let writer2 = writer2.unwrap();

Or as @cuviper pointed out, just the return of any function, modulo its parameters.

Which is to say, your macro can make things less error prone, but I haven't seen any way to get a static type-based guarantee.

It's been a year or two though, maybe look around for "singletons" to see where the state of the art is at.

(The example above may seem contrived, but consider collecting a Vec of these or what-have-you.)

sboo · November 10, 2022, 7:15am

I don't really want a singleton, there can be two writers for a given "Domain", I just don't want the handles to get mixed.

I will try to live with the loop/function issue being a risk for now, assuming the creation code will probably straightforward like:

let main_writer = new_writer!();
let auxiliary_writer = new_writer()!;

I think the LCellOwner suggested by @2e71828 looks like a good working solution, for the longer term, when I get to refactor things more. It forces a weird usage (having to write the consuming code in a closure) but I guess it's the only way.

system · February 8, 2023, 7:16am

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.

Topic		Replies	Views
Mutable reference to owned value?	6	2626	March 6, 2020
Feedback for Tutorial about Borrowing & Co tutorials	9	800	September 11, 2022
Mutable owner and mutable borrow help	9	1055	January 12, 2023
API That let's the user decide who owns data help	5	444	October 18, 2020
Need help in resolving lifetime issue for borrowed slice help	4	247	November 26, 2023

Can I statically ensure that an object is "owned" by another?

Related topics