How to constrain by AsRef<T: Trait>?

My library has a struct which works with objects implementing trait Tr, I'd like users to be able to pass Rc<T: Tr> or other smart pointers easily. Ideally using same methods, but introducing new method is fine as well.

I noticed that all smart pointers implement AsRef, so I thought maybe I could use that, but I can't figure out syntax.

trait Tr {}
impl Tr for i32 {}

struct S<T> {
    s: T
}

// Works
impl<T: Tr> S<T> {
   fn new(s: T) -> Self{Self{s}}
}

// How to support smart pointers?
// This fails with "the type parameter `U` is not constrained by the impl trait, self type, or predicates"
impl<U: Tr, T: AsRef<U>> S<T> {
    fn new_as_ref(s: T) -> Self{Self{s}}
}

fn main() {
    let a = S::new(0i32);
    let b = S::new_as_ref(std::rc::Rc::new(0i32));
}

Rust playground link

Interestingly same approach works with impl<U: Tr, T: Deref<Target=U>> S<T> , but Deref is not what I want, because it seems to penetrate all layers getting to the innermost ref if U happens to be Deref too.

The fundamental problem you're running into is that a single type T can implement both AsRef<U> and AsRef<V> at the same time. If both types U and V implement Tr, there's no way for the compiler to know which of U or V should be used as the intermediate type for S<T>.

A common way to support smart pointers in a situation like this is to impl<T:Tr+?Sized> Tr for Rc<T>, and likewise for the other smart pointers you want to support.

1 Like

Why not use Deref instead? That's conventional for smart pointers anyway, and it has an associated type instead of a type parameter, so you won't run into the same issue:

impl<T> S<T>
where
    T: Deref,
    <T as Deref>::Target: Tr,
{
    fn new_as_ref(s: T) -> Self {
        Self { s }
    }
}

Playground.

Better yet, don't constrain constructors at all! Only put trait bounds when you actually need them. There's no reason to apply either the Tr or Deref bounds on new() in either case.

2 Likes

I briefly mentioned at the bottom, it doesn't seem to be a good fit because nested deref go to the bottom most layer

I agree, but it is obviously a simplified example, in my lib struct has methods were constraints are required and same type problem arises.

AsRef isn't meant to dereference (see section on generic implementations in the docs, added with PR #99460).

In theory, it would work well if

  • The (inner) types you want to pass have a reflexive implementation of AsRef,
  • The generic implementation of AsRef for smart pointers is forwarding the as_ref call to the pointee.

But this isn't the case, see issue #45742.

For a longer dicussion on the subject, see the discussion Semantics of AsRef on IRLO, which lists various examples (Playground) where things go wrong.

In short: The AsRef trait in std is flawed in regard to smart pointers, and I don't see any progress to fix it currently. :slightly_frowning_face:


Edit: I missed that you have a trait Tr. I assumed that you want to make a function accept an optional level of "smart pointer wrapping" for a type, which is what my answer above is referring to. But you have a trait. Moreover, the types which implement the trait might not have a reflexive AsRef implementation, such as i32. So in your case, things are different than what I assumed, and I'm sorry I didn't fully read your post before responding. I still assume using AsRef for dereferencing is a bad idea, though, but not sure.

I'm not sure if I'm overlooking your example properly, but I believe what you need is Borrow instead of AsRef, and I feel like you will need a PhantomData and type annotations to make this work. (But I'm not sure if I get it right.)

The following example compiles, but feels a bit unwieldy.

use std::borrow::Borrow;
use std::marker::PhantomData;

trait Tr {}
impl Tr for i32 {}

struct S<T, U> {
    s: T,
    phantom: PhantomData<fn(&T) -> &U>,
}

impl<T, U> S<T, U>
where
    T: Borrow<U>,
    U: Tr,
{
   fn new(s: T) -> Self {
        Self {
           s,
           phantom: PhantomData,
        }
   }
}

fn main() {
    let a = S::new(0i32);
    let b = S::<_, i32>::new(std::rc::Rc::new(0i32));
}

(Playground)


Deref would require a wrapper (like Box, Cow::Owned, or deref_owned::Owned) if you want to store the type directly (without Rc, Arc, etc.), because T doesn't implement Deref<Target = T>.

&T and &mut T does, though – and dereferencing is not required for construction, only for access.

I did consider Borrow, but it has the same problem as AsRef – it's generic. Your solution introduces an unwanted type parameter on S itself, which is likely undesirable.

But

struct S<T> {
    s: T
}

needs to own a value, and a reference wouldn't allow it to own that value.

The problem would then occur later when you want to use the value. A function might require that T: Deref<Target = U>, U: Tr. Then, if T is an Rc<i32>, it works. But if T is an i32, it won't work. And if we make T to be a &'a i32, we introduce another lifetime (and need to store the integer somewhere else).

Yes, that's what I meant with "unwieldy". It's not very nice. I do see Deref as an alternative, but then you need something like deref_owned::Owned if you want to store an owned value directly (or resort to Box or Cow::Owned as workaround).


Example:

use std::borrow::Borrow;
use std::marker::PhantomData;
use std::ops::Deref;

trait Tr {
    fn use_tr(&self);
}
impl Tr for i32 {
    fn use_tr(&self) {
        println!("I'm integer {}.", self);
    }
}

struct S<T, U> {
    s: T,
    phantom: PhantomData<fn(&T) -> &U>,
}

impl<T, U> S<T, U> {
   fn new(s: T) -> Self {
        Self {
           s,
           phantom: PhantomData,
        }
   }
}

impl<T, U> S<T, U>
where
    T: Borrow<U>,
    U: Tr,
{
   fn foo(&self) {
       self.s.borrow().use_tr();
   }
}

struct SWithDeref<T> {
    s: T,
}

impl<T> SWithDeref<T> {
   fn new(s: T) -> Self {
        Self {
           s,
        }
   }
}

impl<T, U> SWithDeref<T>
where
    T: Deref<Target = U>,
    U: Tr,
{
   fn foo(&self) {
       self.s.use_tr();
   }
}

fn main() {
    let a = S::new(45i32);
    a.foo();
    let b = S::<_, i32>::new(std::rc::Rc::new(49i32));
    b.foo();
    //let c = SWithDeref::new(45i32); // fails
    let c = SWithDeref::new(std::borrow::Cow::Owned::<i32>(99i32));
    c.foo();
    let d = SWithDeref::new(std::rc::Rc::new(101i32));
    d.foo();
}

(Playground)

Agreed, and I did it in the above Playground.


Alternatively you could use:

-    let c = SWithDeref::new(std::borrow::Cow::Owned::<i32>(99i32));
+    let c = SWithDeref::new(Box::new(99i32));

But that's an extra heap allocation.

Or you use another more simple "deref owned" wrapper, which has no type parameter.

Yeah, I'm aware, but my (implicit) proposal would have been to only require &T: Deref. That works.

I think it only works because Rc implements Display, see modified Playground.


Also note that for<'a> &'a T: Deref is a tautology. It's always true for any type T due to this implementation (with Deref::Target = T and that doesn't allow you to do any additional thing).


Example:

fn takes_any<T>(_arg: T)
where
    for<'a> &'a T: std::ops::Deref,
{
}

fn main() {
    struct WeirdType;
    takes_any(WeirdType);
}

(Playground)

The bound for<'a> &'a T: std::ops::Deref is fulfilled for any type, including WeirdType.

Can you describe in a different way what you mean or give an example? I don't understand.

Thanks for concise explanation. If ambiguity is the problem, why don't allow this definition (T is well known, so no ambuity here) and require user to clarify exact U via turbofish syntax at the as_ref() call site in the function body?

You can do something along these lines, but the generic parameter U needs to be somewhere that can be specified via turbofish— There's no way to do that for impl block bounds. For example, you could make it a type parameter of the new function:

impl<T> S<T> {
    fn new<U: Tr>(s: T) -> Self
    where
        T: Borrow<U>,
    {
        Self { s }
    }
}

fn main() {
    let a = S::new(0i32);
    let b = S::new::<i32>(std::rc::Rc::new(0i32));
}
2 Likes

If the type U isn't part of the struct S, wouldn't then the turbofish be required whenever you want to use methods on struct S that require borrowing?

Consider:

impl<T> S<T> {
    fn foo<U: Tr>(&self)
    where
        T: Borrow<U>,
    {
        self.s.borrow().use_tr()
    }
}

fn main() {
    let a = S::new(2i32);
    let b = S::new::<i32>(std::rc::Rc::new(6i32));
    a.foo();
    b.foo::<i32>(); // removing the turbo fish here will cause a compiler error
}

(Playground)

That's why I proposed to use PhantomData in this post to store the type in S, so it doesn't need to be respecified all the time when using S.

Alternatively, Deref could be used, but this might still require type annotations and/or wrappers for the "owned" case during creation of S (see that post).

I'm honestly not sure what's best to do, but I feel like:

  • AsRef is a bad idea for unwrapping from a smart pointer.
  • Borrow or Deref could be used instead.
    • If Borrow is used, consider adding a PhantomData to the struct S.
    • If Deref is used, the "owned" case will require some sort of wrapper ("dummy pointer").
2 Likes

Here a comparison of the different approaches, including a demonstration on how the disambiguities are resolved by turbofish in each case:

use std::borrow::{Borrow, Cow};
use std::marker::PhantomData;
use std::ops::Deref;

trait Tr {
    fn use_tr(&self);
}

impl Tr for Vec<i32> {
    fn use_tr(&self) {
        println!("I'm a Vec.");
    }
}

impl Tr for [i32] {
    fn use_tr(&self) {
        println!("I'm a slice.");
    }
}

struct WithPhantom<O, T: ?Sized> {
    inner: O,
    phantom: PhantomData<fn(&O) -> &T>,
}

fn with_borrow<O, T>(owned: &O)
where
    O: Borrow<T>,
    T: Tr + ?Sized,
{
    owned.borrow().use_tr();
}

fn with_phantom<O, T>(s: &WithPhantom<O, T>)
where
    O: Borrow<T>,
    T: Tr + ?Sized,
{
    s.inner.borrow().use_tr();
}

fn with_deref<P, T>(pointer: &P)
where
    P: Deref<Target = T>,
    T: Tr + ?Sized,
{
    pointer.deref().use_tr();
}

fn main() {
    let a = vec![1, 2, 3];
    let b = vec![1, 2, 3];
    with_borrow::<_, Vec<_>>(&a);
    with_borrow::<_, [_]>(&b);
    
    let c = WithPhantom::<_, Vec<i32>> {
        inner: vec![1, 2, 3],
        phantom: PhantomData,
    };
    with_phantom(&c);
    let d = WithPhantom::<_, [i32]> {
        inner: vec![1, 2, 3],
        phantom: PhantomData,
    };
    with_phantom(&d);
    
    let e = Cow::<Vec<_>>::Owned(vec![1, 2, 3]);
    let f = Cow::<[_]>::Owned(vec![1, 2, 3]);
    with_deref(&e);
    with_deref(&f);
}

(Playground)

Output:

I'm a Vec.
I'm a slice.
I'm a Vec.
I'm a slice.
I'm a Vec.
I'm a slice.


Note that the with_borrow function requires a turbofish on calling, while in case of with_phantom and with_deref, the type can be deduced from the type of c, d, e, and f (stored in a PhantomData in case of c and d, and stored in Cow's type parameter in case of e and f).

Side note: These ambiguities have also been the reason why I added a type argument B to the deref_owned::Owned wrapper.

A different solution which avoids Deref, Borrow, and PhantomData entirely:

Click to see source which stores an "unwrap" function pointer .
trait Tr {
    fn use_tr(&self);
}
impl Tr for i32 {
    fn use_tr(&self) {
        println!("I'm integer {}.", self);
    }
}

struct S<T, U> {
    s: T,
    unwrap: fn(&T) -> &U,
}

impl<T, U> S<T, U> {
   fn new(s: T, unwrap: fn(&T) -> &U) -> Self {
        Self {
           s,
           unwrap,
        }
   }
}

impl<T, U> S<T, U>
where
    U: Tr,
{
   fn foo(&self) {
       (self.unwrap)(&self.s).use_tr();
   }
}

fn main() {
    let a = S::new(1i32, |x| x);
    a.foo();
    let b = S::new(std::rc::Rc::new(2i32), |x| &**x);
    b.foo();
    let c = S::new(std::sync::Arc::new(std::rc::Rc::new(2i32)), |x| &***x);
    c.foo();
}

Output:

I'm integer 1.
I'm integer 2.
I'm integer 2.

(Playground)


Using a helper trait Is, you could also provide convenience constructors new_direct and new_indirect:

Click to see source.
trait Is<T> {
    fn id_by_ref(&self) -> &T;
}
impl<T> Is<T> for T {
    fn id_by_ref(&self) -> &T {
        self
    }
}

impl<T, U> S<T, U> {
   fn new(s: T, unwrap: fn(&T) -> &U) -> Self {
        Self {
           s,
           unwrap,
        }
   }
   fn new_direct(s: T) -> Self
   where
       T: Is<U>,
   {
        Self {
           s,
           unwrap: Is::id_by_ref,
        }
   }
   fn new_indirect(s: T) -> Self
   where
       T: std::ops::Deref<Target = U>,
   {
        Self {
           s,
           unwrap: std::ops::Deref::deref,
        }
   }
}

fn main() {
    let a = S::new_direct(1i32);
    a.foo();
    let b = S::new_indirect(std::rc::Rc::new(2i32));
    b.foo();
}

(Playground)


I just noticed that adding an unwrap function (instead of PhantomData) to the struct introduces overhead, as it will store the function pointer in real memory (i.e. it's not zero cost).

Conclusion:

I think semantically, using Deref is the right choice, because that's what you intend to do: You store some sort of pointer which can be unwrapped ("dereferenced") to a type which implements a trait (Tr). Using the Deref::deref method exactly provides that sort of "unwrapping", and it also stores the target type Deref::Target for you (so you don't need turbofishes all the time later). The only problem with Deref is that you always need to store a pointer-like type, i.e. if you don't have an Rc but want to store the value directly, you need the dummy wrapper.


A small exercise:

"Click to see an example of how to avoid the unwrap function pointer using a zero-sized type, plus some additional remarks.

In theory (and practice), it's possible to avoid storing the function pointer by using something like a ConstFn trait. So instead of:

One could write:

trait ConstFnByRef<Arg: ?Sized> {
    type Retval: ?Sized;
    const FN: fn(&Arg) -> &Self::Retval;
}

struct S<T, U> {
    s: T,
    _unwrap: U,
}

And then use a zero-sized type instead of a function pointer:

 fn main() {
-    let a = S::new(1i32, |x| x);
+    struct UnwrapNoop;
+    impl<T> ConstFnByRef<T> for UnwrapNoop {
+        type Retval = T;
+        const FN: fn(&T) -> &T = |x| x;
+    }
+    let a = S::new(1i32, UnwrapNoop);
     a.foo();

(Playground)

But this gets very very verbose and likely isn't practical at all.

The method std::ops::Deref::deref already provides the same functionality as the const FN above, with the difference that it maps a &Self to a &Self::Target instead of mapping a &T to a &Self::Retval.

This means that using Deref is similar to the approach with the function pointer, except that a wrapper is needed in case where there less than or more than one level of smart pointers involved.


Getting back to the OP, I would propose something like this:

struct S<P> {
    p: P
}

struct Dummy<T>(pub T);

impl<T> Deref for Dummy<T> {
    type Target = T;
    fn deref(&self) -> &T {
        &self.0
    }
}

impl<T> S<Dummy<T>> {
    fn new(t: T) -> Self {
        Self {
            p: Dummy(t),
        }
     }
}

impl<P> S<P> {
    fn with_pointer(p: P) -> Self {
        Self { p }
    }
    fn foo(&self)
    where
        P: Deref,
        <P as Deref>::Target: Tr,
    {
        self.p.deref().use_tr()
    }
}

It seems to be easy to use:

fn main() {
    let a = S::new(10i32);
    let b = S::with_pointer(std::rc::Rc::new(20i32));
    a.foo();
    b.foo();
}

(Playground)

Output:

Called method `use_tr` on integer 10.
Called method `use_tr` on integer 20.

But I'm very curious what other people think about this problem, and whether there are some downsides. (I have been struggling a lot with Deref myself.)


I just noticed Deref coercion also allows writing the foo method more concise:

 impl<P> S<P> {
     fn with_pointer(p: P) -> Self {
         Self { p }
     }
     fn foo(&self)
     where
         P: Deref,
         <P as Deref>::Target: Tr,
     {
-        self.p.deref().use_tr()
+        self.p.use_tr()
     }
 }

(Playground)

That's another pro for Deref.

1 Like

@jbe , my mind is blown by amount of effort and depth of your response. I do not yet fully understand it, but I'll try to implement your suggestions and see which one fits better real code I had original problem in.

After a lot of trial and error I decided not to do it and just have T: Tr trait bound. It is caller responsibility to provide types implementing trait, Rc or not.

In practice, because Tr is a foreign trait caller will either need to upstream implementation for Rc<T> or implement Tr and Deref (for convenience) for a newtype Wrapper<T>(T).

I'd like to thank again for everyone who contributed ideas. I am sure many people will find it useful and educational.

This topic was automatically closed 90 days after the last reply. We invite you to open a new topic if you have further questions or comments.