For something I'm working on, I'm using Weak pointers to keep certain allocations of a generic type around, but the way I'm using them I never need to turn them back into Rcs.
The only functions I'm intentionally using are Weak::ptr_eq, Weak::clone, and (indirectly), Weak::drop.
Currently I'm casting them to Weak<dyn Any> which works for my purpose, although I'm a little sad about the 2x size cost of the fat-pointer.
Considering the limited set of functionality I'm using, is it safe to mem::transmute (or otherwise cast) a Weak<T> to Weak<()> to remove the type information without creating a fat pointer?
Even ignoring the fact that Weak is #[repr(Rust)], which means that any transmute is UB per the spec, dropping a wrongly-typed Weak will potentially free the allocated space with the wrong layout, which is definitely UB.
Since it is not #[repr(C)], this means the compiler will order the fields in different orders for different choices of T, so accessing the integers might use the wrong offset.
As for freeing, it is definitely wrong, because it uses the size information in the vtable to tell the memory allocator how large the allocation is.
Weak::clone is basically just RcBoxPtr::inc_weak followed by copy, right? So is that trait how the vtable gets used during clone? Would it still work if Weak implemented inc_weak without the RcBoxPtr trait?
It's true the language "makes no guarantees" about the layout of repr(Rust) structs, so it is theoretically possible that the vtable pointer will be used to reconstruct the layout of RcBox<dyn Any> on any attempt to access strong or weak, but the compiler does not do that today and I find the suggestion quite implausible. That would mean incurring a (potentially pretty hefty) cost to each access of any field of a struct with an unsized member, in exchange for the dubious advantage of being able to more tightly pack fields when T's alignment is less than the maximum.
If you were really desperate to save a pointer, I think it's only slightly sketchy to assume that access to the sized fields of an unsized struct does not need the pointer metadata, so converting Weak<T> to Weak<()> might be an OK thing to do as long as you only clone, never drop, because dropping actually uses the layout of the unsized field.
I don't think this can be true. inc_weak has nothing to do with Any; why should it be stored in Any's vtable? True, it could be stored in RcBoxPtr<dyn Any>'s vtable, but that still has to work for any dyn Any; you can't dispatch to a different version of inc_weak for each concrete T, for the same reason methods with type parameters aren't object safe. (RcBoxPtr<_> doesn't seem to be used as a trait object, anyway.)
Rust never guaranteed its type field layout except for #[repr(C)]. The RcBox<()> and the RcBox<Foo> are completely different types, nothing guarantees the order of strong/weak count fields are same between them.
Yes, I'm aware of that; however, there are different degrees of "not guaranteed". Which is why I said "only slightly sketchy".
The only reason you can unsize custom structs at all is because there is special code in layout.c that puts the one ?Sized field at the end of the struct. (This is why you can't have a struct with more than one generic ?Sized member.) Yes, it is possible that a compiler could exist that breaks this assumption, but in order to maintain the guaranteed semantics of unsizeable structs, it would sometimes have to generate much more complicated code for field access.
(This is completely different from the more common case of a struct with only Sized fields, in which case the compiler can, and does, reorder them freely.)
The main reason I responded to @alice is because their post suggests that the compiler uses the vtable pointer even to directly access sized fields of a struct like RcBox<dyn Any>. If this were true, it would have a pretty severe performance cost.
I'd not mentioned the T field of the RcBox<T> struct for the same reason. What I mentioned was two AtomicUsize fields which represents strong/weak counter, which is enough to break functionality you've expected when reordered.
To repeat myself more concisely: They are not reordered, because reordering them would give field access a high runtime cost and have no meaningful payoff.
It's true the language allows them to be reordered. But the compiler does not do it (and almost certainly never will).
Reordering them give field access zero runtime cost as it performed entirely in compile time and all functions are already monomorphised before get into runtime.
People regularly suggest to pseudo-randomize struct field ordering to prevent language calcification. For example in C there're tons of code in the ENTERPRISE application which relies on unspecified compiler implementation details. This prevents lots of newly available optimizations as it will break existing code.
It is true that field access is zero-cost, but that's only true because fields in a struct with a generic field that is ?Sized cannot be freely reordered. You can't have both:
data layout of RcBox<T> that depends on the concrete type of T
field access of RcBox<dyn Any> that does not depend on the concrete type represented by dyn Any.
Hmm right, I missed the unsize coercion. I agree that implementing CoerceUnsize practically prevents field reordering, though not explicitely specified.
I don't see this. It does prevent reordering the last unsized field of the struct, but does not prevent the compiler from reordering the other fields to minimize code size. In older architectures it was sometimes the case that a register-indirect load or store was a more compact instruction, or executed faster, than an instruction that had an extra post-indirect offset syllable. On such an architecture it would be completely reasonable for the compiler to put the most-accessed field at the zero offset, with the other fields at higher offsets that required extra instruction syllables (i.e., longer-form instructions). Thus the ultimate field ordering could be a function of how many times the code accesses those fields.
Modern CPU instruction-fetch-and-decode pipelines reduce the value of such optimizations, but I imagine that they might still occur in some very-low-end IoT SoC architectures / implementations.
It's possible I have caused some confusion by not precisely specifying what I mean by reordering.
The fields don't have to be in any particular order, so the compiler may reorder them -- but it can only do so once for the whole RcBox type family, instead of many times, for each individual RcBox<T>. That is, RcBox<T> has a layout that consists of a fixed, common header (containing strong and weak) that is the same as for every other RcBox, plus the T itself (at the end). The compiler doesn't have to put strong before weak, but it does have to put them in some order that is independent of the choice of T.
This isn't the case for a generic struct that contains no ?Sized members. The compiler does rearrange those for each different generic type parameter, even including the non-generic members. Structs with a single generic field of ?Sized type are special in order to make unsizing coercions work.
To be perfectly clear: this is correct for the compiler as implemented.
The language as specified explicitly allows the fields to have a different order for different instantiations of #[repr(Rust)] structures, whether they have a ?Sized member or CoerceUnsized or not.
Yes, it's unlikely that the compiler will change the ordering when a ?SizedCoerceUnsized member exists, because this would require putting field access through the vtable that wouldn't otherwise be necessary. I'm not certain if it's even possible to keep track of this kind of reordering "just" with vtable entries.
However, this is valid for rustc or another implementation of Rust to do. So even if it's "LLVM defined behavior," it is still UB at the Rust level.
So "how much UB is it" is only useful academically; in practice, relying on UB, no matter how "obvious" that the compiler will do it the "right" way, is still disallowed.
Here's the implementation of From<&str> for Rc<str> (link):
impl From<&str> for Rc<str> {
#[inline]
fn from(v: &str) -> Rc<str> {
let rc = Rc::<[u8]>::from(v.as_bytes());
unsafe { Rc::from_raw(Rc::into_raw(rc) as *const str) }
}
}
RcBox<T> is not repr(C), but this code indirectly turns a *const RcBox<[u8]> into a *const RcBox<str>. Does this code have undefined behavior?
If the answer is "no", I can see only a few ways to make it so...
The standard library as a whole is special; it's allowed to rely on (certain kinds of¹) UB. This doesn't make me happy for obvious reasons.
Rc is special because it's a lang item and the compiler is specifically guaranteed to make RcBox<[u8]> and RcBox<str> compatible, just to make this impl sound. This doesn't seem as bad as giving a blanket "UB pass" to the entire stdlib, but it seems unlikely. Also, this particular kind of specialness doesn't appear to be documented anywhere.
Alternatively, str is special; it's guaranteed to be treated the same as [u8] for layout purposes (even behind pointers?). Again, this seems unlikely based on what I know of compiler internals, and it's not documented anywhere from what I can tell.
The well-defined semantics of unsizing actually constitute an "implicit guarantee" that generic types with T: ?Sized members will not be reordered with respect to T. This makes sense if you think of "cheap field access" as being a guarantee, which until this thread I was taking for granted.
Of course, there is a fifth possibility that we want to reject:
This is just UB and code that uses Rc<str>::from(&str) is unsound.
¹ Back to the question of whether there are "kinds" of UB. If you assume that UB is UB I think you have a more difficult time with this possibility. It would obviously be incorrect for the standard library to rely on OS-specific behavior, but rustc-specific behavior is a bit of a gray area IMO.
