Manually-proven bivariance

I'm making a crate (tentatively named variance-family) for requiring that a family of types parameterized by a 'varying lifetime (like T<'varying>, using a made-up HRT syntax) is covariant or contravariant over 'varying.

TLDR of the below is that I want to know when transmuting &'a mut T<'v1> to &'a mut T<'v2> is sound; is "manually-proven" bivariance sufficient? (As opposed to needing compiler-proven bivariance, where the compiler can prove that 'varying is entirely unused in T<'varying>, making T<'varying> bivariant over 'varying.)

("Bivariance" over 'varying, at least in this post, means being both covariant and contravariant over 'varying. I read that the compiler doesn't internally represent bivariance as covariance + contravariance, but I don't know much about that.)

Manually-proven covariance

I want to permit manually-proven variance; that is, I want to permit people to unsafely assert that transmuting a lifetime in a covariant (or contravariant, or bivariant) way is sound even if the compiler considers the lifetime invariant. For instance,

/// # Variance
/// `unsafe` code (such as usage of `mem::transmute`) is allowed to assume that
/// covariantly changing the `'varying` lifetime of `Foo<'varying>` is sound.
//
// Safety of asserted guarantee to `unsafe` code:
// This struct trivially only uses `'varying` in a soundness-relevant covariant way.
// `PhantomData` is a ZST without any typestate-ish guarantees about its generic
// parameter, so even though the `PhantomData` here is invariant over `'varying`, the
// lifetime can soundly be transmuted in a covariant way.
struct Foo<'varying>(
    &'varying u32,
    PhantomData<fn(&'varying ()) -> &'varying ()>,
);

// This type could implement an `unsafe` trait from `variance_family` which
// indicates that transmuting `Foo<'v1>` to `Foo<'v2>` in a covariant way is sound.

I'm fairly sure that manually-proven variance is useful in nontrivial cases as well; for instance, when I made a concurrent skiplist (one supporting only insertions and not removal/deletions, if that's relevant), the nodes used in the skiplist had the following definition:

pub(super) struct Node<'herd> {
    /// The `AtomicErasedLink`s have internal mutability that allow the pointers' values to be
    /// changed. Nothing else is allowed to be changed.
    ///
    /// Vital invariant: for any `AtomicErasedLink` in `skips` which semantically refers to another
    /// node `node`, that `node` must have been allocated in the same [`Herd`] allocator as `self`.
    ///
    /// Callers in this crate also generally put `Some` skips at the start and `None` skips at the
    /// end, though that is not a crucial invariant.
    ///
    /// [`Herd`]: bumpalo_herd::Herd
    skips: &'herd [AtomicErasedLink],
    entry: &'herd [u8],
}

(The entry values are immutable after a node's creation. A "link" is an Option<&'herd Node<'herd>>, and an AtomicErasedLink is a lifetime-erased link that wraps AtomicPtr<()> to let the link be changed atomically.)

That struct is covariant over 'herd, though if AtomicErasedLink were not lifetime-erased, it would be invariant over 'herd. The only parts of the struct's interface which write to the skips field are unsafe methods. (In other words, all parts of the type's interface which are contravariant over 'herd are gated behind unsafe with safety conditions that are sufficient to let all the covariant parts of the interface be safe.) In that particular case, I think covariance was largely a coincidence and my main goal was just "nail down the bump allocators", but I imagine that there's also code out there doing similar tricks for the sake of getting covariance. I might as well, then, support manually-proven covariance and contravariance in variance-family.

Soundness of manually-proven covariance (or contravariance)

Is this sound, both with respect to std and third-party crates using generics?

In particular, as far as I'm aware, covariance and contravariance are additive, like Send or (most) crate features, so attaching importance (beyond something like "being !Send and not covariant in a parameter are infectious, it will make my wrapper type be !Send and not covariant in that parameter") to some generic parameter not being covariant or being !Send is immensely likely to be unsound. AFAIK it'd be perfectly sound for a particular !Send type to provide some means of sending values of that type to other threads... or for a T<'varying> invariant over 'varying to provide methods that are effectively covariant casts of 'varying. I think, then, that it'd also be sound for a T<'varying> type to document that using mem::transmute to covariantly change the 'variant parameter in T<'varying> is sound (and that it'd be sound for unsafe code to actually use mem::transmute to do so).

On the other hand, a type might force itself to be !Send or invariant over a parameter in order to allow unsafe code associated with that particular type to soundly make some assumptions. One concern I'm not at all worried about is someone using a Foo<'varying> field to force invariance (where Foo is the struct defined near the start of this post); AFAIK, it would actually be sound to use Foo to force invariance; there's no good reason for someone to peek into a struct's definition, see "aha, you must not actually be invariant in any important way", and proceed to write or use unsafe code that ignores the invariance. Whether a hypothetical counterexample relies on auto-traits, derive macros and specialization, or someone just reading the source code, I'm fairly sure that using Foo<'varying> to force invariance over 'varying does not let sound code be broken even if using mem::transmute to change the 'varying lifetime of Foo<'varying> is permitted. Of course, I don't think someone writing unsafe code should rely on Foo<'varying> to force invariance over 'varying, but any unsoundness that could result from composing a covariant transmute of Foo<'varying> with other unsafe code really seems like it'd be the fault of the other code.

Soundness of manually-proven bivariance

If manually-proven covariance and contravariance are acceptable, then AFAIK, together they imply manually-proven bivariance. As far as I'm aware, the only way for the compiler to prove that T<'varying> is bivariant over 'varying is for 'varying to be entirely unused in T<'varying>; e.g., maybe T is a type constructor that always returns u32. Manually-proven bivariance would allow for other examples, but I can't imagine any useful examples. At the very least, there's something like Baz<'varying>(PhantomData<*mut &'varying ()>); where Baz promises not to do anything reliant on the 'varying lifetime.

I suspect that if T<'varying> unsafely asserts that bivariant casts of its 'varying parameter are sound, then transmuting &'a mut T<'v1> to &'a mut T<'v2> should be sound (where both types are well-formed).

Another way to phrase this claim is that if T<'v1> can be soundly transmuted to T<'v2>, and T<'v2> can be soundly transmuted to T<'v1>, then &'a mut T<'v1> can be soundly transmuted to &'a mut T<'v2> (if both types are well-formed). Perhaps I could even go so far as saying that "T<'v1> and T<'v2> are the same type" (modulo a 'static bound that could provide access to a TypeId) since codegen cannot (soundly) depend on a lifetime parameter.

What makes me so cautious is that I know U being a subtype of V and V being a subtype of U do not together imply U = V; in particular, two higher-ranked types without canonical normalizations may be subtypes of each other while being distinct types. Therefore, even if both covariant casts and contravariant casts of U to V are sound, it seems unlikely to generally be sound to cast U to V in an invariant position like &mut U, as they could be distinguished by their TypeIds. However, as the problem in #97156 could be seen as the compiler effectively transmuting a private field and/or transmuting a struct with private fields with safety invariants... maybe it'd still be sound to transmute any public &mut U value to &mut V (and other transmutes of a publicly-reachable U to V in invariant positions) when U and V are subtypes of each other? I don't know if there's any sound code capable of escalating that to UB. However, without for<T> binders I don't think it's useful (even if sound). Someday, if for<T> binders are possible, I might revisit that thought.

In any case, that concern about higher-ranked types shouldn't apply to two types which differ only in a lifetime parameter. Am I missing anything that would make transmuting &'a mut T<'v1> to &'a mut T<'v2> unsound (when T<'v1> can be soundly transmuted to or from T<'v2> in both covariant and contravariant positions)?

Note on compiler support

Maybe, in some future version of Rust, it'd be nice to have something similar to unsafe impl Send for _ but for unsafely asserting that covariant (or contravariant) casts of a parameter are sound. See also Manual variance unsafe escape hatch? - language design - Rust Internals. We already stabilized unsafe attributes, so I think something like struct Foo<#[unsafe(covariant)] 'varying, ..> { .. } would be nice to have at least for lifetime parameters; that is, effectively, what variance-family would implement (modulo support for more than one lifetime with unsafe variance, which, for ease of implementation, variance-family will not include).

I don't think it's critical, though, as absent more support for type constructors or higher-ranked types and a way to require that a type constructor is covariant/contravariant/bivariant in its parameters, my use case(s) for variance-family will still need custom traits. Support for manually-proven / unsafely-asserted variance beyond the variance proven by the compiler is merely a small detail on top of those traits.

2 Likes

This is an impressive nerd snipe :slightly_smiling_face:.

I attempted a couple replies but they ended up being very long and also, I think, retreading ground you've already thought about. So let me try to narrow the scope instead.

Are we talking about this use case specifically?

As in calling transmute::<&'m mut T<'a>, &'m mut T<'b>>(...).

If so, why not just provide that as a method instead of appealing to transmute specifically?

Or are we also talking about transmuting where T<'_> appears in any position?

1 Like

I'm talking about where T<'_> appears in any position. &mut _ is more-or-less a placeholder for "an invariant position which is publicly visible, not private". I think that the .0 field of

struct Foo<T>(pub fn(T) -> T);

or even just

struct Bar<T>(pub PhantomData<fn(T) -> T>);

would have very similar (and perhaps identical) concerns.

Using Invariant<_> (in some invariant crate) and Bivariant<'_, _> (in some bivariant crate) with what I think are the obvious meanings, the conclusion I've come to is that Invariant<Bivariant<'a, P>> should be able to be soundly transmuted to Invariant<Bivariant<'b, Q>> provided that no fancy typestate-ish stuff is going on in Invariant<_> that could depend on the exact type or TypeId of its parameter. That is, the type of the parameter should be relevant for reads or writes of values whose type is that parameter, and not signify the state of other data. I'm also not completely sure whether typestate-like techniques are infectious enough to make transmuting &mut Bivariant<'a, P> to &mut Bivariant<'b, Q> unsound in some crazy example, but I'm leaning towards "that's probably possible but dumb, and I'll consider variance-family to be sound, even if this is a replace_with vs partial-borrow situation". Worst-case scenario, some crazy unsafe code manages to expose a &'c mut Bivariant<'a, P> in such a way that no safe code could possibly allow a &'c mut Bivariant<'b, Q> to exist during 'c, and it could provide a way to escalate to UB from the existence of a &'c mut Bivariant<'b, Q>.

For that to harm variance-family, Bivariant<'varying, T> would need to implement variance_family::BivariantFamily, and P = Q would need to be sufficient to escalate to UB since variance-family only provides lifetime casts. Presumably, the unsafe code in invariant wouldn't be able to soundly use the concrete type Bivariant<'varying, T> (else invariant would have bivariant as a dependency and the fact that Bivariant implements a problematic trait should be known to the invariant crate), unless this is a situation where the crate providing Bivariant reasoned "adding a trait impl isn't a breaking change" and accidentally broke invariant. That's... unfortunate. I think that does provide a narrow window where independently-sound unsafe code could be unsound in conjunction with variance-family.

As another route to unsoundness, perhaps the pathological unsafe code in invariant could instead have some unsafe trait ParameterForInvariant { type WithLifetime<'varying>; } documenting that the sole safety requirement is that WithLifetime<'varying> must not leave 'varying entirely unused. (That way, *mut _::WithLifetime<'varying> would be invariant in 'varying, according to the compiler.) Hypothetically, there's a risk that bivariant::Bivariant could implement both variance_family::BivariantFamily and invariant::ParameterForInvariant without realizing there's a problem, and dutifully follow the safety docs of ParameterForInvariant... but the safety docs in variance-family call attention to this potential problem, so I think that Bivariant would not soundly be able to implement both traits.

Anyway, I'm close to having the core of variance-family in a good state; it won't be published all that soon (still a bunch of helper macros to write), but I'll at least push it to GitHub and share a link here (likely by the end of this evening).

1 Like

I am going to be honest, I spent almost half an hour trying to understand what was being discussed here :'D But it did improve my understanding of what variance is. I would like to see how you implement this, when you make the source code public that is.

1 Like

Ok, a more general bivariance. Some of this topic may be better suited to IRLO (language design), or perhaps UCG? But I'll try to convey my thoughts.

Bivariance of any type constructor

In a general sense -- more general than you are attempting, I think -- my answer would be "one should not assume that is sound", as (if nothing else) it is almost surely the case that there are crates which rely on the current behavior of invariance transform in a generic way. I.e. they design a custom struct S<Generic> with the Generic being invariant and rely on that for soundness.

(In this section I mean any type constructor, not "a type constructor with one lifetime parameter". Skip ahead if you don't care about it.)

By "transform" I mean: Say T in Foo<T> is $variance_outer, and U in Bar<U> is $variance_inner. What is the variance of V in Foo<Bar<V>>? It is the "transform" of the two variances, $variance_outer x $variance_inner.

So a critical part of the general use case is: What is the variance of invariant x bivariant? You seem to want the answer to be bivariance. But in the compiler it is currently invariance. The cited paper says

Is the type equality “S = T” syntactic or semantic? (I.e., does type equality signify type identity or equivalence, as can be established in the type system?) If type equality is taken to be syntactic, then the only sound choice is o ⊗ ∗ = o.

(o ⊗ ∗ = o is what we have.)

And my take-away from this PR is that type equality is syntactic.

So even without considering the assumptions of other crates, it may be unsound. That said, the immediately following portion of the paper is over my head. I don't know if there are actually things that would instantly expose unsoundness in Rust if it was flipped (outside of code makes their own assumptions and use unsafe), or not.

But what I can do is produce an example of invariance x bivariance = bivariance being unsound based on someone relying on the current implementation. If there is a possibility of allowing bivariance in invariant context generally, it is at least a case of soundness conflict ("replace_with vs. partial-borrow situation").

The playground is clearly contrived. But I suspect there are examples of practical crates relying on invariance of some PrivateTy<InvariantParam> in a way that would be broken by changing the status quo, via reliance on TypeId if nothing else. (Though it's true I didn't go looking, so I don't have a citation.)

So my gut says that invariance x bivariance = bivariance would be too breaking.

Or without language support: my gut says that "transmute in any position" would be in conflict with something other crates can reasonably expect.

One way to think about it: invariance x bivariance = bivariance takes away the ability of a crate author to ensure a type parameter is invariant (and my gut says that ability is load bearing).

A reliance on the transform is distinct from any particular type having some invariance or not,[1] so arguments around "one should assume types may become more variant non-breakingly" don't necessarily apply. (But I'll return to that in a less general setting later.)

(I don't think it matters a ton for what I wrote, but in Rust,[2] bivariance of a type parameter is the ability to convert from one type to any other type, not just "covariance + contravariance". This does mean bivariance at all, but particular when invariance is expected, is probably surprising to most.)

Bivariance with a single lifetime parameter

You weren't talking about full-blown bivariance though, you're talking about a type constructor with a single lifetime parameter, if I understand correctly.

Let's talk about the general case -- transmuting when T<'v1> is in any position.

Consider this playground.[3] Here's the critical step:

    let foo2: for<'out> fn(&'out &'short (), &'short T) -> &'out T = foo1;

    // subtyping: function arguments are contravariant,
    // go from 'short to 'static, but only for one of the inputs
    //
    // DANGER: for `&'out &'static ()` to be valid, `foo3` has an implied bound `'static: 'out`
    //
    // This is wrong! we lost the bound for `'short: 'out`, even though
    // that one is needed by `foo` to soundly return `value`
    let foo3: for<'out> fn(&'out &'static (), &'short T) -> &'out T = foo2;

...and imagine you had &T<'_> instead of &&() in the first argument. Now, the fact that this compiles is a bug. If it can compile soundly in any way in the future, it will need to be something like

//                 vvvvvvvvvvvvvvvvvv
let foo3: for<'out where 'short: 'out> fn(&'out &'static (), &'short T) -> &'out T = foo2;

And the same is true of your desired transmutes: they can be unsound if implied bounds are lost.

I believe this is true if we get rid of the binders too -- you need something higher-ranked to exploit the compiler bug, but types have implied bounds even outside of a higher-ranked context.

So I think you at least need some caveat about implied bounds.

That is the kind of "relying on the transform being invariant" I had in mind in the general section. But given that we're only considering type constructors with a single lifetime parameter here, for a given type where the only non-'static portions are lifetimes considered bivariant, there's only one possible way to get something with a TypeId -- make all the lifetimes 'static.

(Note that this is not the same as going from for<'a> ... Ty<'a> to ... T<'static>. Those would be two distinct types with distict TypeIds.)

So maybe those considerations are okay. That is to say, if a crate is depending on the invariance of a parameter for the sake of "preserving TypeId". Your transmutes could make something stop meeting a 'static bound, but then it wouldn't have a TypeId.[4]

Incidentally, from the OP:

"Modulo" how? There's no "anything but 'static" bounds if that's what you meant. But I'm probably just not understanding what you meant in that section.

Other hesitations

This section is more brain-storming than fleshed out. It's a bit long-winded, sorry about that.

I'm still a bit hesitant to think "transmute::<T<'a>, T<'b>> is always sound" is sufficient. I think my hesitation is coming from a couple related places.

One is that foreign crates can do whatever type-related stuff they want with your types.

impl Trait for S<Bi<'a>> {
    type Ty = &'a str;
}

// Could transmutation this cause problems later?  Without `unsafe`?
// (I'm not sure.)

Or generically...

// Applies to `Ty<'static>` only
unsafe impl<T: 'static> Send for S<T> {}

// (It is sound to *prevent codegen from being reached* based on a
// a bound against a lifetime parameter...)

And the other source of my hesitation is this subtopic:

I agree that not implementing Send today is not a guarantee that you never will implement Send.[5] That's the same as most other traits; the primary difference is that you can be Send without explicitly opting in.

Your argument seems to be: covariance, contravariance, and bivariance[6] should work the same way. But I think one can construct cases where assuming the invariance of a parameter is reasonable. Here's one I made (which is also linked from the soundness conflict topic).[7] Or consider:

pub struct MyInvariantDependentType<T> {
    mtx: Mutex<T>,
    other: T,
}

pub fn elsewhere(&self) {
    // SAFETY: We're invariant in `T` due to containing `Mutex<T>`,
    // which cannot logically be variant in `T` due to providing
    // safe shared `&mut T` access through `&Mutex<T>`.
    unsafe {
        // ... something requiring the invariance ...
    }
}

Did the author make a reasonable assumption that T in Mutex<T> is always invariant?

Maybe I'm just retreading what you already considered:

I think the only way around that is to make everything explicit. We will probably some day get negative implementations, which is a code level guarantee that you will never implement the trait.[8] For variance, we'd need something like a notation on parameters.

Even that doesn't change assumptions about variance transform.[9] Perhaps that's where my hesitation is coming from -- being able to always transmute T<'varying> in any position whatsoever still weakens a property of invariant type positions. If someone intentionally relied on your T<'varying> not changing somehow, I think I would put that on them (the point is to allow it to change). But as I covered when talking about the general case, it seems reasonable to rely on how variance transform works today generally.

And finally I'll note that even those who don't assume a type will stay invariant may still be surprised if it becomes bivariant. You have to jump through hoops to get a bivariant type parameter (I don't think I've seen it in "real code"), and AFAIK there is no way to get a bivariant lifetime parameter (so far).[10]


So I guess my gut feeling is that transmuting in invariant position is likely "soundness conflict" territory at best. On the upside, if it is true that the TypeId concerns are harmless due to the "one lifetime parameter" limitation, that removes some probable conflicts.


  1. it's a reliance on a parameter in a given type constructor, which may be your own type constructor ↩︎

  2. as in the Taming the Wildcards paper ↩︎

  3. from this blog post about this well known issue ↩︎

  4. Shades of the better-any crate... ↩︎

  5. on it's own; you could guarantee it in documentation ↩︎

  6. the last one implies the other two (and more) ↩︎

  7. IMO this means that if trait parameters can ever be not-invariant, it will need to be opt-in (or opt-out after an edition). ↩︎

  8. without a SemVer bump ↩︎

  9. Unless you introduce multiple types of invariance? Maybe? (Seems quite unlikely to happen.) ↩︎

  10. I think they exists behind GATs, but that's bivariance in an invariant position (and thus not observable)... if I understand correctly, which I may not. ↩︎

4 Likes

First, here's the (near-)current state of the code: self-ref-box/crates/variance-family at 13d890c3355c3793999e098a10da5e4614ac1c16 · robofinch/self-ref-box · GitHub

Alas, it uses a LifetimeFamily<'lower, 'upper> trait (where 'lower and 'upper place implied bounds on a 'varying lifetime), only for me to find out in my actual use case that the 'upper lifetime unnecessarily infects too many types. (for<'lower> ... gets rid of the lower bound in a useful way, but I want the upper bound to be able to be non-'static.) So, my next revision will most certainly be using LifetimeFamily<_, U> for a better upper bound (which could just be U = &'upper () to recover the original behavior, AFAIK). I'm not aware of a good way to convert the 'lower bound to a type parameter, though, as the 'varying: 'lower implied bound is achieved via &'lower &'varying (). Maybe I haven't thought hard enough, but I don't see a good way to get a "'a outlives all lifetimes in the T type" bound, and given that all bounds between lifetimes and types I'm aware of use the opposite direction, I'm not sure that converting the 'lower bound into a type parameter is possible.

Given @quinedot's wonderfully detailed feedback, I'll probably also fall back to what I initially had before BivariantFamily: an UnvaryingFamily trait, where the 'varying-parameterized "family of types" doesn't actually use the 'varying lifetime at all (so the type "doesn't vary" at all as 'varying changes). I'm not sure if "unvarying" is too easily confusable with "invariant" when it means nearly the dead opposite.

More responses coming soon...

2 Likes

Ooo, I don't think I've looked through there before. Likewise, I only recently came across the better-any crate and have yet to look through the details there. I'm sure both UCG and better-any will have a lot of fun details to read (on more than just variance) :slight_smile:

Mmm, between that and:

Given that I am aware of no practical use case for a bivariant lifetime parameter, I think there's insufficient justification to toe the line with my unsafe. The actually useful case of casting Invariant<T<'v1>> to Invariant<T<'v2>> when T<'varying> leaves 'varying entirely unused is well within the lines.[1]

Doh, I literally read about that elsewhere on URLO. I really need to be more careful when I talk about variance.[2]

Yeah, AFAIK I don't touch higher-ranked function pointers at all, so I shouldn't come close to losing implied bounds. But I definitely need to add more warnings about implied bounds and link to that bug report.

I just meant "one of the types might have strictly more trait implementations that were gated behind a 'static bound, but otherwise they're indistinguishable." I don't know if it's a fully well-formed thought. Something something implementing a trait only for non-'static lifetimes (or for any specific non-'static lifetime) is unsound something something. Come to think of it, there's probably some convoluted usage of negative impls and specialization that makes that statement not entirely true [3].

I was thinking along the lines of "even if there were a trait-bound-ish way to require that T<'varying> is invariant over 'varying, that fact shouldn't be useful". ...Probably. Any sort of unsafe InvariantFamily trait would probably have to be explicitly implemented or derived on types, and it could have sufficient safety conditions beyond solely "the compiler must say the 'varying parameter is invariant" which could make the trait actually be useful. But if I imagine a hypothetical unsafe InvariantFamily auto-trait provided by the standard library (one for a type parameter, rather than a lifetime), then there could be something like

pub struct MyInvariantDependentType<T, ForceInvariance: InvariantFamily> {
    inv: ForceInvariance::OverType<T>,
    other: T,
}

Having an absolute guarantee that MyInvariantDependentType<T, FI> is invariant over T and will not be coerced into MyInvariantDependentType<U, _> is one thing. However, for some user-defined Invariant<T> struct which which automatically implements InvariantFamily and is a ZST, the type might provide trivially-sound casts from &'a Invariant::OverType<T> to &'a Invariant::OverType<U>, since the defining crate of the type could ensure that their code places no safety invariants on when an Invariant<T> can be constructed, and then a &'a Invariant::OverType<U> could be pulled out of thin air.

I can't immediately see a way to break the following example:

pub struct Foo<T, FI: InvariantFamily> {
    pub inv: FI::OverType<T>,
    foo: StuffUnsafelyUsingInvarianceOverT,
}

It may be sound to cast from &'a FI::OverType<T> to &'a FI::OverType<U>, but that shouldn't mean anything when it comes to casting &'a Foo<T, FI> to &'a Foo<U, FI>, right? So, the ability to bivariantly cast a parameter in an invariant position -- in an opt-in way for types with an invariant parameter -- at least doesn't infect everything. However, I'm much less sure of what conditions are necessary for casting between &'a Mutex<T> and &'a Mutex<U> to be sound. It seems tempting to assume that equal size, alignment, niches, safety invariants, and validity invariants (there's surely more I'm forgetting) of T and U can be assumed to be sufficient to imply soundness of that Mutex cast, but maybe third-party unsafe code would end up making additional sound-in-isolation assumptions about Mutex. Some of my concerns about &'a mut Bivariant<'varying> (or via *mut _, Cell<_>, fn(_) -> _, or combining them together) were similar, especially in cases that feel more publicly constructible than Mutex<_>, like fn(_) -> _ or &'a mut _. I suppose there's not much point in worrying about that further if I fall back to UnvaryingFamily.


  1. Incidentally, the trait solver / typeck / whatever is powerful enough to safely cast between those types when provided with a bound indicating that Varying<'varying, 'lower, 'upper, T> is the same type for all (valid for WF) 'varying lifetimes. More precisely, I think that I coaxed them to normalize into the same type. I was pleasantly surprised that the compiler managed that, since some of the silly things I've done in other projects, including type-equality bounds, managed to baffle the trait solver (due to known quirks of type inference, no need to link to all the relevant GitHub issues). ↩︎

  2. One other detail for which I tried to avoid inconsistency in variance-family's docs is whether one speaks of "T<'varying> is (co/contra/in/bi)variant in 'varying" or "'varying is (co/contra/in/bi)variant in T<'varying>". In either case, "in" could, correctly or not, be used with a technical meaning (which would likely be intended only in the first case). In the second case, the colloquial "the 'varying lifetime present in this type over here, T<'varying>" definition of "in" could instead be used. I decided to dodge the issue by saying "T<'varying> is (co/contra/in/bi)variant over 'varying". I'm not mistaken about "in" having two possible interpretations, am I? ↩︎

  3. You could, say, have a NotStatic trait implemented for any type which doesn't meet a 'static bound, and then make a Static trait implemented for any type which doesn't meet a NotStatic bound. It probably shouldn't be useful for anything, but it seems possible. ↩︎

I haven't had time to make a proper reply. But I did happen upon this recently closed issue which involved coercing raw dyn pointers in such a way that a method like so:

    fn get_type_id(&self) -> TypeId
    where
        Self: 'static,

goes from uncallable to callable due to making a lifetime 'static when it wasn't in the implementation. The issue has some discussions you may find interesting.

It also suggested to me a simple way to demonstrate unsoundness related to the concept of "transmuting anywhere". There's a UAF and the only unsafe is the transmute, so the transmute must be to blame.

In general terms, I'd say that "it's always safe to transmute T<'varying> in whatever position" is just too strong of a statement, too sweeping a claim for the type system we have. And there are also just too many ways for people to assume control or understanding of how parameters in "their own" type constructors work. Unfortunately I don't have a good suggestion on how to delimit when it is actually sound.


I don't know if I'll get around to a proper reply or not, so I'm just going to dump some other things I thought about here. At least some of them are off-topic anyway.

I'm pretty loose in my usage too, so I can't say anything useful here, other than it's great that you're making an effort to be clear and consistent. If there's a formally correct phrasing, I don't know it.


Random thoughts:

  • Psuedo-variance behind invariance already exists (in very limited cases)

  • Ralf considers moving out of a &'static mut to be sound, and AFAIK Ralf and Niko still consider the replace_with pattern sound.[1] So fully relying on invariance behind &mut _ specifically is already soundness conflict territory,[2] as the replace_with pattern lets one remove the value from behind the invariance for the duration of the borrow.

  • Manual covariance for otherwise invariant types can sometimes be safely emulated

    This is sometimes achievable without unsafe by using dyn Trait + '_ type erasure. If you coerce a Invariant<'a> to a dyn Trait + 'a, the lifetime parameter is now covariant.

    (OTOH downsides include: all your functionality needs to be representable by a trait, you might need to allocate a Box<_> or whatever for the dyn Trait, dynamic dispatch.)


  1. But there has been no official decision on the topic AFAIK. ↩︎

  2. like if you modified my variance transform playground to return a leaked &'static mut _ instead ↩︎

1 Like