Could Arc have Arc::aliased that would behave like C++ shared_ptr's aliasing constructor?

1

In C++, shared_ptr has an aliasing constructor with the signature

template< class Y >
shared_ptr( shared_ptr<Y>&& r, element_type* ptr ) noexcept;

Where this shared_ptr shares the counters of the derived shared_ptr, but contains different data when accessed.
Like a lot of things in C++, this requires you to only pass in a pointer that matches the lifetime of the aliased shared_ptr, which could lead to UB if you don't.

In Rust, I believe we could have an API that does the same thing, but safely:

pub fn aliased<'a, U, F>(this: &Arc<T>, f: F) -> Arc<U>
where
    F: FnOnce(&'a T) -> U + 'a,
    T: 'a // This might be necessary

The lifetime 'a would come from the implementation, which would borrow the Arc's data, and pass it to f. Since the return value is required to live as long as the parameter, it would be impossible to create an Arc with something that's not either a reference to something in the stored object or 'static.
With the returned U + 'a, the Arc implementation would use the same counter and Drop implementation as the original Arc, but would return the different data pointer.

This API could be useful for exposing only parts of shared structs, or 'static data that would only be accessible as long as the original Arc is. But mostly, I wanted to discuss this as an exercise.

  1. Would this API really be sound, or did I overlook something?
  2. How different would the implementation of today's Arc need to be to allow us including this API (hypothetically)? Would there be any performance penalty for Arc that might make this API not worth adding? Would we need !Drop as an additional constaint on F to make this work (so that the stored data doesn't need to be Dropped separately; and is !Drop even a thing that can exist?), and is the standard library allowed to do that in public APIs today?
  3. Would a simplified version where the return type would be just &'a U, and thus only allowed aliasing a single field (or subfield) make things easier without needing negative trait constraints?
2

This signature is somewhat off:

pub fn aliased<'a, U, F>(this: &Arc<T>, f: F) -> Arc<U>
where
    F: FnOnce(&'a T) -> U + 'a,
    T: 'a // This might be necessary
  • T: 'a is implied by the presence of &'a T
  • You can't borrow **this for 'a in order to call f
  • The other 'a bound is on F, it's not on U if that's what you meant
  • AFAIS putting the bound on U doesn't help anything

So I guess you meant

pub fn aliased<'a, U, F>(this: &'a Arc<T>, f: F) -> Arc<U>
where
    F: FnOnce(&'a T) -> U,

I'm not really seeing any benefit over returning U though [1]. Assuming U captures 'a, this can't be dropped until there are no U left, so the underlying data can't be dropped either. What's the point of sharing the Arc counters?

  1. which is trivial, and I don't see the point over just using &*the_arc to get a &T for that matter ↩︎

3

Today's Arc is a single pointer, whereas C++'s shared_ptr is two pointers.

That distinction is what allows the aliasing constructor to work in C++: it uses the same pointer to the reference counts, but has the data pointer going wherever you want.

So I don't think this is possible in just the Arc type without pessimizing all the users that don't need this behaviour.

Maybe you could make a "projected arc" type that holds a copy of the original Arc and a pointer to the new data, though?

4 Likes
4

I guess the point is when U doesn't capture the lifetime. In which case I thought of the same "projected arc" concept.

Very naively:

#[derive(Clone)]
pub struct QuoteShareUnQuoteTheArcCounters<T, U> {
    arc: Arc<T>,
    derived: U,
}

// derive Deref<Target = U> for ...

pub type Stac<T, U> = QuoteShareUnQuoteTheArcCounters<T, U>;

pub fn share_the_counters<'a, U, F, T>(this: &'a Arc<T>, f: F) -> Stac<T, U> 
where
    F: FnOnce(&'a T) -> U,
{
    let derived = f(this);
    let arc = this.clone();
    Stac { arc, derived }
}
5

Returning Arc<U> also has the problem that you no longer know how to drop T statically, so you're going to need to have a dyn drop vtable for T if you want to drop it. If you aren't borrowing from T, it reduces to just (Arc<dyn Destruct>, U).

If you are, it becomes annoyingly difficult, fast, and is essentially the "self referential" problem.

With this signature, very much no; you're allowing the caller to choose whatever 'a they want. (I choose 'static. :slightly_smiling_face:)

The complexity of a properly sound API is nearly irreducible from yoke. I should know; I tried and failed. The real insidious soundness hole that is difficult to patch effectively is variance.

(...I really need to get around to yanking that crate, given it's unsound...)

3 Likes
6

This is a great answer, thank you. Looking at yoke is frankly intimidating, and it made me realize I'd need to guarantee that the returned type is covariant over the lifetime, which isn't possible to guarantee for arbitrary types (at least AFAIK, that's why the Yokeable trait in yoke exists).

I came up with this much simpler version, that only works on references, and only projects using fn, which should hopefully solve the issue of local captured references that happen to be 'a, and only allow projecting members (or references that are already 'static).
Something like this might still be useful for niche use-cases (but much less so than what yoke can do):

use std::{ops::Deref, sync::Arc};

#[derive(Debug, Clone)]

/// Parc because Projected Arc
struct Parc<P, T>
where
    P: Send + Sync + 'static,
{
    _arc: Arc<T>,
    projected: &'static P,
}

impl<P, T> Parc<P, T>
where
    P: Send + Sync,
{
    pub fn aliased<'a>(arc: &'a Arc<T>, f: fn(&'a T) -> &'a P) -> Parc<P, T>
    where
        P: 'a,
    {
        let projected: &P = f(arc);
        // SAFETY: fn shouldn't be able to capture any local references
        // which should mean that the projection done by f only contains things
        // that will live as long as the Arc (?)
        let projected: &'static P = unsafe { std::mem::transmute(projected) };
        Self {
            _arc: arc.clone(),
            projected,
        }
    }
}

impl<P, T> Deref for Parc<P, T>
where
    P: Send + Sync + 'static,
{
    type Target = P;
    fn deref(&self) -> &Self::Target {
        self.projected
    }
}

The most annoying thing about this reduced version is that it's still necessary to name T, or make anything that you want to use Parc in generic over T.

7

The Debug implementation looks unsound as it exposes the the &'static P reference with a false lifetime to user code. It’s possible for users that define their own type P to and an impl Debug for &'static P.

To be fair, it looks like the yoke crate contains the same soundness hole… gotta open a new issue :innocent:

Here we go ^^

10 Likes
8

I guess the takeaway is: "Folks, don't write (or use) self-referential type crates" :sweat_smile: Yoke is not the first attempt at creating a "safe" self-referential type, and all major previous efforts so far have been full of soundness holes.

1 Like
9

I'm writing another one just now :smiley:

I'm excited to see what soundness holes I created

1 Like