Is this proper, safe usage of transmute_copy?

#1

Essentially, I need to store a NetworkVariableInner<Box<dyn NetworkTransferable + 'static>> inside a hashmap, and want to give the user a pointer to it inside a wrapper called NetworkVariable<T>:

#[derive(Clone)]
pub struct NetworkVariableInner<T: NetworkTransferable + 'static> {
    inner: Accessor<T>
}

pub struct NetworkVariable<T: NetworkTransferable + 'static> {
    ptr: NetworkVariableInner<Box<dyn NetworkTransferable + 'static>>,
    // we store a T here, that way, when we deref, we can map Box<dyn NetworkTransferable> to Box<T>
    _pd: PhantomData<T>
}

impl<T: NetworkTransferable + 'static> NetworkVariable<T> {
    pub fn new(ptr: NetworkVariableInner<Box<dyn NetworkTransferable + 'static>>) -> Self {
        Self { ptr, _pd: Default::default() }
    }
}

impl<T: NetworkTransferable + 'static> Deref for NetworkVariable<T> {
    type Target = Accessor<T>;

    fn deref(&self) -> &Self::Target {
        let ref reinterpreted = unsafe { std::mem::transmute_copy::<_, NetworkVariableInner<Box<T>>>(&self.ptr) };
        &reinterpreted.inner
    }
}

Accessor<Box<T>> is the inner element I'm trying to deref to

Since the type is stored inside PhantomData, the size/alignment should be preserved. The data type won't change over time, so I figured that reinterpreting the type-variable from Box<dyn NetworkTransferable + 'static> to Box<T> should be okay. I haven't used transmute_copy before, so I want to make sure I'm doing it correctly, and if possible, if there is a safer way of doing what I'm trying to do

#2

An easy way to look at the problem: How can one properly downcast a type parameter from a trait object to a concrete type?

#3

Box::<dyn Any>::downcast()

#4

The thing is that it's a type parameter that does not have an easily accessible variable (the type variable is the type to a variable inside a Mutex), so, I'm not sure if downcast would do the job

#5

What do you mean by this? I see no Mutex in the code you posted. Are you trying to Deref to the value inside the mutex? That's not possible safely, as it circumvents the guarantees of the Mutex.

#7

With that type Accessor<T> = Arc<tokio::sync::Mutex<T>> definition, I get the compiler error:

error[E0308]: mismatched types
   = note: expected reference `&Arc<tokio::sync::Mutex<T>>`
              found reference `&Arc<tokio::sync::Mutex<Box<T>>>`

So it looks like even the existing unsafe solution isn't correct w.r.t. types. I don't think you can project like that – how would the Box disappear suddenly? IOW, you can't really convert a Mutex<Box<T>> to a Mutex<T>.

#8

It's not actually circumventing the mutex. I have an async mutex that requires network synchronization. The idea is to deref properly into Accessor<T>, that way, once the mutex does become deref'd after getting locked (done by the user after dereffing past the NetworkVariable<T> shown in the code), it gets deref'd to the concrete type.

This is how I'm creating it:

    fn create_variable<T: NetworkTransferable + 'static>(&self, value: T, var_type: VariableType) -> NetworkVariable<T> {
        let (notifier_tx, notifier_rx) = channel(1);
        let (updater_tx, updater_rx) = channel(1);
        let net_var_inner = NetworkVariableInner::new(Box::new(value) as Box<dyn Any + 'static>, var_type, notifier_rx, updater_tx);
        let user_net_var = NetworkVariable::<T>::new(net_var_inner.clone());
        // [...]
        let mut lock = self.map.write().unwrap();
        lock.insert(self.get_next_vid(), net_var_inner);
        // register the variable with the internal local mapping
        user_net_var
    }

Also: I'm going to have to use dyn Any + 'static now, because the compiler told me that NetworkTransferrable can't be Sized

The easy solution would be to do an enum dispatch, and storing an enum inside the hashmap where the NetworkVariableInner's are stored. It would narrow the domain of possible T's, is the only issue.

#9

What you could do is move the Mutex out one level, and coerce it to Any, since it's CoerceUnsized. Playground.

2 Likes
#10

This constructor does not enforce that that dyn Network… is a type-erased T, it could have been type erased from some other type. So either the constructor needs to be unsafe fn, and express that requirement, or you can "simply" replace dyn Network… input by T:

  impl<T: NetworkTransferable + 'static> NetworkVariable<T> {
-     pub fn new(ptr: NetworkVariableInner<Box<dyn NetworkTransferable + 'static>>) -> Self {
+     pub fn new(ptr: NetworkVariableInner<Box<T>>) -> Self {
          Self { ptr, _pd: Default::default() }
      }
  }
  • (Note that in this simplified design I then fail to see the point of having a dyn if a PhantomData<T> is gonna be kept around)

Then, even when that dyn … came indeed from a T, the transmute_copy is still incorrect (even if there is a number of crates out there, such as ::dyn_clone, which make that wrong assumption): you are assuming that a pointer to a dyn Trait has the layout of a data pointer followed by the vtable pointer, with no padding nor reordering whatsoever. And this is not something the language guarantees: it could very well put the vtable pointer first!

The correct way to get the data pointer out of a fat pointer is with an as cast:

let fat_ptr: &dyn Trait = …;
//                                                         Sized
//                                                          v
let slim_ptr: &T = &*(fat_ptr as *const dyn Trait as *const T);
//                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
//                            extracts the data ptr from the fat ptr

In your case, however, this cannot be done, since there is that Accessor layer in the middle: if the Accessor struct is an opaque Rust type with an undeterminate layout, you can't transmute Accessor<T> to Accessor<U>, even in the case where T and U can be transmuted between each other (and example of this: transmuting a Vec<u8> to a Vec<i8> is, for instance, not guaranteed to work).

All in all, try to stick to letting Any handle the downcasting, following, for instance, @H2CO3's last code snippet :slightly_smiling_face:

1 Like
#11

Thanks for the help. @H2CO3's method worked well in this case, and, as always, @Yandros provides an interesting deeper understanding for the discussion