Question about dyn Trait coercions

help
10

It is definitely a good and legitimate question, that's for sure!

1) It does not seem possible to exploit this with a legitimate impl

Click to hide

I'll start with @steffahn points, which are the main argument about why the following function ought to be unsound no matter the trait involved:

fn somehow_swap<'usability>(
    a: &mut (dyn SomeTrait + 'usability),
    b: &mut (dyn SomeTrait + 'usability),
)

  1. First and foremost, "the vtable problem": while Rust represents these wide pointers as:

    &mut dyn Trait

    In practice, layout-wise, this is rather:

    Dyn<Trait, &mut ?>

    • In fact, if we write the + 'usability here, we end up with:

      Dyn<Trait + 'usability, &mut ?>

      So we can see this + usability is not really a property of the pointee, but rather of the pointer!

    insofar:

    • it is a special &mut type (w.r.t. the usual &mut impl Sized[1] references).

    • with the dyn Trait metadata inline rather than behind the &mut ? indirection (even if this metadata happens to be, itself, behind &'static indirection): every such &mut dyn Trait = Dyn<&mut ?, Trait> packs its own copy of this metadata!

    This, thus, already makes overwriting the pointees with different backing concrete types unsound. To illustrate, if different backing types were allowed, then already something as simple as:

    trait ToBool {
    fn to_bool(self: &Self) -> bool;
}

impl ToBool for bool { fn to_bool(self: &bool) -> bool { *self } }
impl ToBool for u8 { fn to_bool(self: &u8) -> bool { *self != 0 } }

    would be problematic. Indeed, consider:

    // note: the `<bool as ToBool>::to_bool` is thus, machine-wise,
// probably implemented as `transmute_copy::<?, bool>(self)`.

let a = &mut true as &mut dyn ToBool;
let b = &mut 42_u8 as &mut dyn ToBool;

somehow_swap(a, b);

a.to_bool(); // <- does `transmute_copy::<?, bool>(&42_u8)` 💥

    Can only swap the same type!

  2. Then, therefore, we have the problem of type identification. This leads us to Any, which in turn requires : 'static. So no problem there.

  3. But what about user-implemented dyn LtAny<'some_lifetime>? (these can be written in a sound manner, although that's a topic for another thread). Well, in that case, on top of that + 'some_lifetime "dangerously covariant" lifetime parameter, now we also have this explicit <'some_lifetime> parameter, which is invariant. Hence no problem either.

Now, this explanation seems to suggest that the "accountability for UB" seems to lie in the somehow_swap() function more than in the + 'lt "acting covariantly when directly behind a &mut", but it does so with a list of cases that we hope is exhaustive rather than with a more direct conceptual point, so we may still have a lingering doubt of "have we truly considered all cases" looming over us, as is often the case with unsafe and trying to reason about 100% air-tight soundness.

2) Back to basics to illustrate: the unsized coërcion

Click to hide

The observation that convinced me about all this was, rather than focusing on the re-unsizing coërcion case, to first think about the simpler unsizing coërcion case:

/// For any `dyn`-safe `Trait`:
fn demo<'usability>(
  r: &mut (impl 'usability + Trait /* + Sized */),
) -> &mut (dyn  'usability + Trait)
{
  r /* as _ */
}

Now, depending on the point of view, this should both:

  • be blatantly obviously fine and sound;
  • be screaming the "but &mut is invariant!" issue: the very issue we are ourselves wondering about.

The latter may not be obvious at first, but I hope that once I present it, and thanks to the former, we'll be convinced of the soundness of the latter, and by extension, of re-unsizing coërcions, even behind &mut, as well.

Indeed, lets pick:

  • impl 'usability + Trait = &'static str (and Trait = Display or w/e),
  • and 'usability something shorter than 'static:
fn demo<'usability>(
  r: &mut (&'static str),
) -> &mut (dyn 'usability + Display)
{
  r
}

Indeed, &'static str : 'static : 'usability, so we do have an impl 'usability + … in the input, and thus are allowed to have a dyn 'usability + … in the output, despite having had to deal with a &mut all along.

Moreover, conceptually, the implicit + Sized bound on the impl 'usability + Trait in that function was not playing a critical role, API-wise, only a technical role, implementation-wise.

So we could consider replacing it with ?Sized + TechnicallyDynCoërcible in pseudo-code / conceptually:

fn demo<'u>(
  r: &mut (impl 'u + Trait + ?Sized + TechnicallyDynCoërcible),
) -> &mut (dyn  'u + Trait)

and from there, we could then pick, for any 'big : 'u:

impl 'u + Trait + ?Sized + TechnicallyDynCoërcible = dyn 'big + Trait

The re-unsizing coërcion is thus really, conceptually, just an extended case of unsized coërcion!

  • For those considering that my TechnicallyDynCoërcible is using a circular argument, the reason I've gone with such a roundabout name rather than Upcast<dyn 'u + Trait> (an actual proper trait expressing this), was precisely to avoid the circular problem: I'm trying to argue that dyn 'big + Trait : Unsize<dyn 'u + Trait> even behind &mut, by assuming there being some technical (i.e., compiler-intrinsic and/or unsafe operation) allowing the transformation, and trying to illustrate that, conceptually, the impl 'u + … -> dyn 'u + … coërcion, even behind &mut, is fine, modulo the technical implementation, which happens to be trivial in the case of dyn 'b + Trait.

  • another way of seeing this is to consider having a Super Rust language (with perfect type introspect/reflection), which would allow perfect downcasting, even in the face of lifetimes:

    fn perfect_downcast(
  r: &mut (dyn  'u + Trait),
) -> &mut (impl 'u + Trait)
{
    // would yield the original thin / "concrete" type, but which
    // happens to be existential as far as the caller is concerned.
}

    Armed with such a tool, we could then implement re-unsized coërcions using plain unsized coërcions:

    fn reunsized_coercion<'b : 's, 's>(
  r: &mut (dyn 'b + Trait),
) -> &mut (dyn 's + Trait)
{
    let r: &mut (impl 'b + Trait) = perfect_downcast(r);
    let r: &mut (impl 's  + Trait) = r; // `impl 'b + … : 'b : 's`
    r as _ // <- unsized coërcion with `'u = 's`
}

3. Why somehow_swap() cannot be sound

Click to hide

The idea is to observe there is a fundamental difference between the meaning of <'lt>, and that of + 'usability, to which I hinted, regarding the LtAny<'lt> design:

  • <'lt> expresses the property of being infected by exactly 'lt. A corollary is that for Self : 'u to hold, it is necessary that 'lt : 'u hold.

  • + 'usability expresses an existential and erased lifetime property.

    It may seem suprising to talk of erasing lifetimes when dyn Traits always have, at the very least, this seemingly pesky + 'u around, but that's just because "sane"/frequent Rust types only involve types infected with one lifetime parameter. In such a case Type<'lt> : 'lt follows, in both directions, which is why it is easy to conflate both notions.

    • Box<&'a mut &'b str> -> Box<dyn 'a + Debug>, for instance, is a nice example of this: we had two infecting lifetime parameters in input, and end up with fewer lifetimes in output: effectively, we have achieved lifetime erasure!

    • Another example could be the typical BoxFuture that #[async_trait] yields:

      #[async_trait(?Send)]
trait Demo {
    async fn foo<'a, 'b>(
        a: Arc<Mutex<&'a str>>,
        b: Arc<Mutex<&'b str>>,
    )
    {}
}

      is basically:

      trait Demo {
    fn foo<'a, 'b, 'intersection>(
        a: Arc<Mutex<&'a str>>,
        b: Arc<Mutex<&'b str>>,
    ) -> Pin<Box<dyn 'intersection + Future<Output = ()>>>
    where
    //     ⊇
        'a : 'intersection,
        'b : 'intersection,
    {
        return Box::pin(async move {
            // impl InfectedBy<'a> + InfectedBy<'b>
            let _captures = (&a, &b);
        });
    }
}

      In which case we have erased all lifetime params, both 'a and 'b, down to their "lowest common denominator": their intersection lifetime. Anything else was "superfluous lifetime info" (w.r.t. the Future API), which could thus be erased. Should the Trait API need to use any of these lifetime parameters explicitly (e.g., dyn FnOnce() -> &'a str), then it means that lifetime parameter is to be part of the trait definition itself, as one of its generic lifetime parameters: it gets infected by it! (e.g., back to dyn LtAny<'lt>).

All that to say that what

  • + 'usability

conveys, in fact, is the property:

  • exists<'a, 'b, … where
    'a : 'usability,
    'b : 'usability,
       ⋮
    '… : 'usability,
> InfectedBy<'a> + InfectedBy<'b> + …

To illustrate, we could say there is an "electronic cloud" of 'lifetimes orbiting around our type, but all contained within the 'usability..'∞ range (using '∞ as syntax for 'static).

And knowing that:

  • it is correct to "lose information" and just know that this "electronic cloud of lifetimes" is within the 'shorter..'∞ range (where 'usability : 'shorter);

  • it has to be wildly unsound to pretend that one "electronic cloud of lifetimes" is allowed to substitute/replace another one such.

    In other words, somehow_swap() is unsound.

4) If we replace + 'usability with + Trait, reünsizing is just upcasting

Consider:

&mut dyn 'big -> &mut dyn 'small

and compare it to:

&mut dyn Fn() -> &mut dyn FnMut()

In the world of existential capabilities which + … means, these two operations are conceptually the same.

  • it just so happens that we know, implementation-wise / at runtime, that the former conversion does not involve machinery (since lifetimes don't exist at the machine code level), which is why we just talk of a coërcion in that case, versus trait_upcasting in the latter case.

  • see my following post for a short demo that ought to remove any outstanding doubt (using unsized coërcions to reïmplement reünsized coërcions).

  1. &mut impl Thin, actually ↩︎