How safe is mem::transmute for changing life time of an owned type?

The core idea of what I'm trying to achieve is simple. There is a container-like type that supposed to support zero-copy for deserialization. Yet, if an owned copy is requested by user then it replaces all lifetimed references with owned types. In order to avoid extra re-allocations the owned values replace the references in-place by converting from one enum variant to another. What I didn't like is that the container's type lifetime remains the same from compiler's point of view and it is bound to the deserialization source. To get around the limitation I came up with the following method:

    pub fn own<'b>(mut self) -> BorOw<'b, B> {
        if let BorOw::Borrowed(b) = self {
            let _ = std::mem::replace(&mut self, BorOw::Owned((*b).to_owned()));
        }
        unsafe { std::mem::transmute::<BorOw<'a, B>, BorOw<'b, B>>(self) }
    }

BorOw here is the inner enum which does the trick for the outer struct. It is parameterized over B which has the following bounds:

    B: ToOwned + ?Sized + 'a,
    <B as ToOwned>::Owned: Clone + Hash + Debug + PartialEq + Eq + Ord + PartialOrd + DeserializeOwned,

As I see this case, say, in a particular case of &str -> String the struct becomes, de-facto, 'static, so any 'b not tied up to 'a is OK. But I am not missing something here?

Why not just

pub fn own<'b>(mut self) -> BorOw<'b, B> {
    match self {
        BorOw::Borrowed(b) => BorOw::Owned((*b).to_owned()),
        BorOw::Owned(o) => BorOw::Owned(o)
    }
}
1 Like

It would be safe in this case, assuming to_owned returns a value without any lifetimes.

However, you don't need it. return BorOw::Owned((*b).to_owned()) will just work in safe code, because it will infer the correct lifetime from the return type.

You're only fighting the wrong lifetime, because you're trying to overwrite the old value with the wrong lifetime.

Same as @drewtato said (best as I can tell from your OP), for Cow.

It could be generalized to a general lifetime extender for non-'static T. E.g.

Thank for your suggestions! Since everybody here suggests the same solution, I'd better not reply individually.

BorOw here is only part of wider solution to avoid extra re-allocations. For example, a Map type uses a BorOw-based string type for its keys. This allows to implement in-place key replacement making hash duplication or key removal/insertion unnecessary. Eventually, let map = map.own(); gives me the same structure at the same memory location, but owned and with a different lifetime.

If the suggestions don't work, supply further context, such as the definition of BorOw or better yet a playground.

UPD Ignore this reply. I'll leave it here as an example of stupidity and how it is a bad idea to consider complicated things when already tired... Of course, keys of BtreeMap are immutable and what I wanted to achieve won't work without inner mutability pattern which would be too expensive to make sense. It was a while since I was able to work on the project, enough to forget certain details. Now I need to think twice if in-place replacement is still reasonable or it is utterly excessive... Thanks for letting me to have a second look into own code.

It'd be too much for a playground, but declarations alone should be sufficient to demonstrate my plan.

#[derive(Hash, Debug, PartialEq, Eq, Ord, PartialOrd)]
pub enum BorOw<'a, B>
where
    B: ToOwned + ?Sized + 'a,
    <B as ToOwned>::Owned: Clone + Hash + Debug + PartialEq + Eq + Ord + PartialOrd + DeserializeOwned,
{
    Borrowed(&'a B),
    Owned(<B as ToOwned>::Owned),
}

#[derive(Debug, Clone, PartialEq, Eq, Hash, PartialOrd, Ord)]
pub struct ValueString<'a>(BorOw<'a, str>);

#[derive(Debug, Clone, PartialEq)]
pub enum MapBackend<'a> {
    #[cfg(feature = "ordered_objects")]
    Ordered(IndexMap<ValueString<'a>, Value<'a>>),
    Unordered(BTreeMap<ValueString<'a>, Value<'a>>),
}

#[derive(Debug, Clone, PartialEq)]
pub struct Map<'a>(MapBackend<'a>);

ValueString and Map implement a trait ValueBorOw:

pub trait ValueBorOw<'a> {
    type Owned: 'a;

    fn own(self) -> Self::Owned;
    fn make_owned(&mut self);
    fn clone_owned(&self) -> Self::Owned;
}

BTW, clone_owned here is exactly what's suggested. But the trick is about make_owned which is responsible for modifying objects in place without changing life time; and own is the same, but changes it. Map does it like this:

    fn own(mut self) -> Self::Owned {
        self.make_owned();
        unsafe { mem::transmute(self) }
    }

    fn make_owned(&mut self) {
        match &mut self.0 {
            #[cfg(feature = "ordered_objects")]
            MapBackend::Ordered(map) => {
                let old_map = std::mem::replace(map, IndexMap::new());
                for (mut k, mut v) in old_map {
                    k.make_owned();
                    v.make_owned();
                    map.insert(k, v);
                }
            }
            MapBackend::Unordered(map) => {
                let old_map = std::mem::replace(map, BTreeMap::new());
                for (mut k, mut v) in old_map {
                    k.make_owned();
                    v.make_owned();
                    map.insert(k, v);
                }
            }
        }
    }

Since I guarantee that ValueString shares the same hash value for both borrowed and owned variants, I make sure that hash internal structures remain intact and the only allocation that takes place is creation of new String objects. But the latter is, obviously, inevitable.

Another point is this implementation of ValueBorOw trait (clone_owned is intentionally omitted):

impl<'b, T> ValueBorOw<'b> for Box<T>
where
    T: for<'a> ValueBorOw<'a>,
{
    type Owned = Box<<T as ValueBorOw<'b>>::Owned>;

    #[inline]
    fn own(mut self) -> Self::Owned {
        self.as_mut().make_owned();
        unsafe { std::mem::transmute(self) }
    }

    #[inline]
    fn make_owned(&mut self) {
        self.as_mut().make_owned();
    }
}

Though I strongly suspect now that replacing Box content would be as fast as the in-place replacement.

1 Like

BTW, correct me if I'm wrong, but to_owned doesn't have to return a value without any lifetimes. Generally speaking, it is OK if the other lifetimes are not related to the original borrowed value.

Oh! In that case, happy to help rubber-duck :slight_smile:

Irrelevant now probably

There doesn't seem to be any problem with @drewtato's suggestion.

Reads some more

Ok, I think I get it. Your question wasn't really about changing BorOw by value, it was about something like Map with some more deeply nested data types. I can see why you would prefer a transmute in that case, if possible.

If I understand your question, correct. The trait looks like this:

// No parameters on the trait
pub trait ToOwned {
    // No parameters on the associated type
    type Owned: Borrow<Self>;
    // Without elision
    fn to_owned<'this>(&'this self) -> Self::Owned;

And from that we can conclude

  • Any non-'static lifetimes nameable in <T as ToOwned>::Owned are also part of T[1]
  • T::Owned can't name 'this, that's a lifetime introduced later, not part of T

So yes, to_owned can return a value with temporary lifetimes, so long as they are also in T (which does not include 'this).

I don't think it's common (in the case of ToOwned), but the type system allows it. Here's a contrived example.


  1. there is nowhere else they could have come from ↩ī¸Ž

Right. I've meant that it can't be allowed to use the lifetime that you're transmuting, because that would be unsound (an invalid implementation in safe code could cause unsafety).