Coercion problems when passing &Arc<dyn Any>

jbe · November 13, 2022, 5:30pm

Continuing the discussion from How to (idiomatically) borrow a str from a String (using the "as" keyword?), I have another issue with coercions:

use std::any::Any;
use std::sync::Arc;

macro_rules! coerce {
    ($x:expr) => ( { let t: _ = $x; t } );
}

fn takes_any_arc(_arg: Arc<dyn Any + Send + Sync>) {}
fn takes_any_arc_ref(_arg: &Arc<dyn Any + Send + Sync>) {}

fn main() {
    let x = Arc::new(1);
    takes_any_arc(x.clone());
    // These don't work:
    //takes_any_arc_ref(&x.clone());
    //takes_any_arc_ref(&x.clone() as &Arc<dyn Any + Send + Sync>);
    //takes_any_arc_ref(&x.clone() as _);
    //takes_any_arc_ref(&(x.clone() as Arc<dyn _));
    //let t: _ = x.clone();
    //takes_any_arc_ref(&t);
    // But these work:
    takes_any_arc_ref(&{ let t: _ = x.clone(); t });
    takes_any_arc_ref(&coerce!(x.clone()));
    takes_any_arc_ref(&(x.clone() as Arc<dyn Any + Send + Sync>));
    takes_any_arc_ref(&(x.clone() as Arc<_>));
    takes_any_arc_ref(&(x.clone() as _));
}

(Playground)

What's the best way to go? I dislike having to write dyn Any + Send + Sync, especially because changing Sendness and Syncedness would have to be redundantly noted everywhere I do this coercion. Writing as _ is a bit vague, and as Vec<_> sort of misses the point here (but works).

I feel like as _ is the best way to go here. But apart from allowing coercion here, it would also enable primitive type casts (which is what's neither needed nor intended here).

Why doesn't takes_any_arc_ref(&x.clone()); work here? Wouldn't it be good if it was allowed and would perform the coercion of x.clone() here? I guess this doesn't work because it's not a coercion site here (and &x.clone() can't be coerced).

I tried to work around this by writing a coerce! macro which basically inserts an artificial coercion site. Is there some less-messy (or already existing) way to achieve this?

Note that coerce!(x) isn't the same as x as _:

#![allow(unused_macros)]

macro_rules! coerce {
    ($x:expr) => ( { let t: _ = $x; t } );
}

fn takes_byte(byte: u8) {
    println!("Got {byte}")
}

fn main() {
    let i: i32 = 1000;
    //takes_byte(i); // compile-time error, of course
    takes_byte(i as _); // danger!
    //takes_byte(coerce!(i)); // gives a compile-time error
}

(Playground)

Output:

Got 232

LegionMammal978 · November 13, 2022, 7:31pm

std::convert::identity() also works for this purpose: takes_any_arc_ref(&identity(x.clone())) compiles without any issue, since the function argument is a coercion site. You could also import the function under a shorter alias.

quinedot · November 13, 2022, 10:30pm

This:

Is long for:

    takes_any_arc_ref(&{ x.clone() });

which can be considered short for the identity function. ^[1]

Why doesn't takes_any_arc_ref(&x.clone())) work here?

I think it's just because a value of &Arc<NotDyn> can't be coerced to &Arc<dyn ...>. You need to coerce a Arc<NotDyn> value to Arc<dyn ...> instead. You need the coerceable thing in a coercion site decoupled from the outer reference.

// My *guess*
takes_any_arc_ref(&x.clone())
// coercion site  ^^^^^^^^^^ tries to coerce value of type `&Arc<i32>`

// coercion site  vvvvvvvvvvvvvv (outer)
takes_any_arc_ref(&{ x.clone() }) 
// coercion site    ^^^^^^^^^^^ (propogated)
// tries and succeeds to coerce value of type `Arc<i32>`

I have no idea how tenable it would be for in-place & construction to be a coercion site akin to a struct field.

n.b. Not everything in that article is accurate with modern Rust. ↩︎

jbe · November 14, 2022, 1:39am

Tonight I learned:

({ expr }) isn't the same as (expr). (Playground)
x as _ can be replaced with { x } if a type coercion rather than a primitive cast is needed.

Thus the macro is useless:

-macro_rules! coerce {
-    ($x:expr) => ( { let t: _ = $x; t } );
-}

Instead we can just do:

     // But these work:
-    takes_any_arc_ref(&{ let t: _ = x.clone(); t });
-    takes_any_arc_ref(&coerce!(x.clone()));
+    takes_any_arc_ref(&{ x.clone() });
     takes_any_arc_ref(&(x.clone() as Arc<dyn Any + Send + Sync>));
     takes_any_arc_ref(&(x.clone() as Arc<_>));
     takes_any_arc_ref(&(x.clone() as _));

(Playground)

I feel like it might be helpful if it was, but not sure about all consequences.

Even if there is no coercion site in future, I think the compiler could give at least a hint to add curly braces. The current error message isn't soooooo helpful:

use std::any::Any;
use std::sync::Arc;

fn takes_any_arc_ref(_arg: &Arc<dyn Any + Send + Sync>) {}

fn main() {
    let x = Arc::new(1);
    takes_any_arc_ref(&x.clone());
}

(Playground)

Errors:

   Compiling playground v0.0.1 (/playground)
error[E0308]: mismatched types
 --> src/main.rs:8:23
  |
8 |     takes_any_arc_ref(&x.clone());
  |     ----------------- ^^^^^^^^^^ expected trait object `dyn Any`, found integer
  |     |
  |     arguments to this function are incorrect
  |
  = note: expected reference `&Arc<(dyn Any + Send + Sync + 'static)>`
             found reference `&Arc<{integer}>`
note: function defined here
 --> src/main.rs:4:4
  |
4 | fn takes_any_arc_ref(_arg: &Arc<dyn Any + Send + Sync>) {}
  |    ^^^^^^^^^^^^^^^^^ ---------------------------------

For more information about this error, try `rustc --explain E0308`.
error: could not compile `playground` due to previous error

That error message is why I ended up trying things like &x.clone() as &Arc<dyn Any + Send + Sync>, which didn't work.

Cerber-Ursi · November 14, 2022, 3:07am

I have a feeling that it's impossible in general, since to create a reference one must have an owned value, not a coerced temporary.

H2CO3 · November 14, 2022, 6:59am

Apart from coercions, there is a much more fundamental difference between the two. Blocks don't preserve lvalue-ness. A block is always a moved value and never a place; if you write &{ foo }, then foo is moved, and you (potentially) get a different address than &foo. Furthermore, you can't write

{ foo } = bar;

even if foo = bar; would be valid (i.e., if foo is an lvalue).

jbe · November 14, 2022, 8:39am

I see, so I understand now why in the general case, &x must not coerce what's in x, i.e. when I write takes_any_arc_ref(&x), then x must already have the correct type.

But that still doesn't mean that takes_any_arc_ref(&y) might have an impact on type inference, right? So I don't understand why this fails:

    let y: _ = Arc::new(2i32);
    takes_any_arc_ref(&y); // couldn't the compiler do the coercion one line above?

Yet this works:

    takes_any_arc_ref(&{ let y: _ = Arc::new(3i32); y});

Why does it work in the second case but not in the first? I think in the second case, it's the "final line of a block" rule (see below).

Ah, I understand now. When I write, let y: _ = Arc::new(2i32), then this is not a "let statement with an explicit type given" (see Reference), thus it is not a coercion site. Even if the type is inferred, coercion cannot take place, so type-inference must result in Arc<i32> as type .

It's not just an in-place & construction, but also a function result. Function results are coercion sites, but apparently only inside the function, i.e. before returning.

Coercion sites

A coercion can only occur at certain coercion sites in a program; these are typically places where the desired type is explicit or can be derived by propagation from explicit types (without type inference). Possible coercion sites are:
[…]
Function results—either the final line of a block if it is not semicolon-terminated or any expression in a return statement

For example, x is coerced to have type &dyn Display in the following:
use std::fmt::Display;
fn foo(x: &u32) -> &dyn Display {
    x
}

I I feel like it's not impossible to make this work (but I don't really know):

    // Function results can be coerced too,
    // but looks like only *before* returning :-(
    // Couldn't they be coerced after returning as well,
    // so that this would work?
    takes_any_arc_ref(&Arc::new(4i32));

(Playground with all examples)

I understand the rules now, but I don't like them. I feel like they should be changed. What's the exact reason why type-coercion is only performed before returning and not after returning? Maybe there's a good reason?

And my next question would be: Even if there is a good reason for not coercing after returning a value, couldn't takes_any_arc_ref(&Arc::new(4i32)) be made to work anyway? Afterall, it would be okay to move the result of Arc::new here because it cannot be used in any other way than through the reference.

jbe · November 14, 2022, 8:48am

Some more experiments:

fn takes_any_arc_ref(_arg: &Arc<dyn Any + Send + Sync>) {}
fn main() {
    // I understand this *must* not work:
    let a: Arc<i32> = Arc::new(1i32);
    //takes_any_arc_ref(&a);
    
    // Is an explicit type is given here, or not?
    // Apparently it's not a coercion site,
    // so I guess `Arc<_>` doesn't count as an explicit type.
    let b: Arc<_> = Arc::new(2i32);
    //takes_any_arc_ref(&b);
    
    // But funnily this works:
    let c: Arc<_> = Arc::new(3i32) as Arc<_>;
    takes_any_arc_ref(&c);
    
    // But this doesn't work again:
    let d: _ = Arc::new(4i32);
    //takes_any_arc_ref(&d);

    // Yet this works again:
    let e: _ = Arc::new(5i32) as _;
    takes_any_arc_ref(&e);
}

(Playground)

I thought I understood everything, but the following is pretty weird:

fn main() {
    // Funnily this works:
    let c: Arc<_> = Arc::new(0i32) as Arc<_>;
    takes_any_arc_ref(&c);
    
    // But wait, why doesn't this work!?:
    let c2: Arc<_> = { Arc::new(0i32) };
    //takes_any_arc_ref(&c2);
    
    // Also this doesn't work:
    let c3: _ = { Arc::new(0i32) };
    //takes_any_arc_ref(&c3);
    
    // Yet that works:
    takes_any_arc_ref(&{ Arc::new(0i32) });
}

(Playground)

I suspect this is a compiler bug regarding type inference?

H2CO3 · November 14, 2022, 9:24am

Probably not. What you are essentially asking the compiler to do is to both infer a type, and then coerce the inferred type into another type. This is at best ambiguous due to the possible combinations of coercions and the possible list of concrete types that could be inferred for a _ type hole. However obvious it might seem to a human, the compiler has to guess twice, and there's not much that could link the two guesses together unequivocally.

This is essentially the same reason why, given T: Into<U> and U: Into<W>, the code some_t.into().into() doesn't infer/typecheck, either: there are potentially many possible types for the first .into() to return, and thus the compiler can't pick a single best one to act as the middle-man in the chain. The infer-and-coerce story is similar, except that ambiguity is caused by the multitude of coercions instead of an explicit parametric trait.

If you find the behavior inconsistent or feel that you don't understand every last little detail, then it's best to stick to explicit types. Writing out a few types here and there is probably way better in the long run than expecting more magic to happen.

jbe · November 14, 2022, 9:35am

I don't think this is ambiguous because there is only one solution (the other is invalid and must fail to compile).

I would understand if the compiler cannot solve this automatically and demands a (more explicit) explicit type annotation. Not sure if inferring the wrong type is considered correct though (maybe there are other cases where this happens and it's generally not considered an error).

The error is error[E0308]: mismatched types, not "type annotation needed" or something like that:

    // But wait, why doesn't this work!?:
    let c2: Arc<_> = { Arc::new(0i32) };
    takes_any_arc_ref(&c2);

(Playground)

Errors:

   Compiling playground v0.0.1 (/playground)
error[E0308]: mismatched types
  --> src/main.rs:15:23
   |
15 |     takes_any_arc_ref(&c2);
   |     ----------------- ^^^ expected trait object `dyn Any`, found `i32`
   |     |
   |     arguments to this function are incorrect
   |
   = note: expected reference `&Arc<(dyn Any + Send + Sync + 'static)>`
              found reference `&Arc<i32>`
note: function defined here
  --> src/main.rs:6:4
   |
6  | fn takes_any_arc_ref(_arg: &Arc<dyn Any + Send + Sync>) {}
   |    ^^^^^^^^^^^^^^^^^ ---------------------------------

For more information about this error, try `rustc --explain E0308`.
error: could not compile `playground` due to previous error

Also, if this does not work:

    let c3: _ = { Arc::new(0i32) };
    takes_any_arc_ref(&c3);

How can I be sure that this will work in future?

    // Yet that works:
    takes_any_arc_ref(&{ Arc::new(0i32) });

H2CO3 · November 14, 2022, 11:16am

Yes – the point being that the compiler isn't even trying to guess twice in a row.

Technically, the Rust compiler has a lot of tests for edge cases around type inference, to ensure that stuff like this doesn't break.
Less technically, the stability guarantee means that if this stops compiling in the future and breaks your code, that will be considered a bug and it will be fixed.
Pragmatically, that's what my "just annotate the types" comment was trying to suggest.

In any case, I still wouldn't consider it a "bug", but if you think it's a missing feature and a candidate for improvement, feel free to open an issue – the next-generation type checker is under active development, and there are some rough edges to rectify apart from coercions anyway, so I'm sure you won't be alone with such a feature request. (I'm not sure how high its priority will be, though.)

jbe · November 14, 2022, 11:33am

Yeah, I see, so I can rely on &{x} continuing to work in future.

While I generally understand, I sometimes prefer to keep the + Send + Sync to be an implementation detail that I don't have to write out everywhere. On the other hand, I have to add the Arc::new as well. Perhaps the cleanest option would be to provide a function which performs the wrapping:

mod m {
    use std::any::Any;
    use std::sync::Arc;

    pub fn takes_any_arc(_arg: Arc<dyn Any + Send + Sync>) {}
    pub fn takes_any_arc_ref(_arg: &Arc<dyn Any + Send + Sync>) {}
    
    pub fn wrap<T>(inner: T) -> Arc<dyn Any + Send + Sync>
    where
        T: Send + Sync + 'static,
    {
        Arc::new(inner)
    }
}

fn main() {
    let x = m::wrap(1);
    m::takes_any_arc(x.clone());
    m::takes_any_arc_ref(&x.clone());
}

(Playground)

I think the type inference issue isn't really a huge problem in practice, but I found it curious.

However, I think that allowing &Arc::new(x) (but not &x where x is of type Arc) to coerce into an &Arc<dyn …> (by coercing the Arc, not the reference!) would be nice. But I understand too little of the implications.

quinedot · November 14, 2022, 8:30pm

None of the compiling experiments are surprising to me, with the right hat on.

// no free type variables, must be Arc<i32>
let a: Arc<i32> = Arc::new(1i32); 

//         v ...need to infer this
let b: Arc<_> = Arc::new(2i32);
// ...clearly an i32     ^^^^

//         v ...need to infer this
let c: Arc<_> = Arc::new(3i32) as Arc<_>;
// ...something i32 can coerce to...  ^  but otherwise still a free variable
takes_any_arc_ref(&c);
// aha             ^ must be `dyn Stuff`

//     v ...need to infer this
let d: _ = Arc::new(4i32);
//         ^^^^^^^^^^^^^^ ...clearly an Arc<i32>

//     v ...need to infer this
let e: _ = Arc::new(5i32) as _;
// ...something Arc<i32>     ^ ...can coerce to but otherwise still free

// Same as `b`.  {} isn't "defer free variable inference"
let c2: Arc<_> = { Arc::new(0i32) }

// Same as `d`
let c3: _ = { Arc::new(0i32) };

// [dosen't compile]
//
// Not really any different from your `existing_arc.clone()` version,
// you have a temporary acting as-if it was a variable path (place expr)
//
// Then you run into either or both of
// - behind an inline reference isn't a coercion site
// - you have a place expr and you need a value expr to consume and coerce
takes_any_arc_ref(&Arc::new(0i32));

// Whatever the reason for the above, this side-steps it just like it
// does with `& { existing_arc.clone() }`
takes_any_arc_ref(&{ Arc::new(0i32) });

Breaking inference isn't considered a major breaking change, but if it looks like it's going to be disruptive they generally don't do it. But blocks losing some of their coercion site properties seems more like a lang level change to me, and I imagine func(&{ x.clone() }) losing much inference would have too wide of an impact.

system · February 12, 2023, 8:31pm

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.

Topic		Replies	Views
Confusion with `Any` again help	13	738	February 9, 2023
A little problem with &dyn std::any::Any	3	157	June 8, 2024
Assign concrete data to `Arc<Mutex<Box<dyn T>>>` field help	10	1311	October 12, 2022
Coercion of multiple references to one? help	2	329	August 12, 2022
How to (idiomatically) borrow a str from a String (using the "as" keyword?) help	7	958	February 10, 2023

Coercion problems when passing &Arc<dyn Any>

Coercion sites

Related topics