Practice: Steps happening in deref coercion

I've learnt part of the topic of dereferencing. As a self-made exercise I tried to follow the steps of dereferencing going from &Box<i32> (one or nested) to &i32.

Here I simply expose my reasoning to see whether there are important corrections to be made.

The snippet is:

fn main() {
    let bbox:Box<i32> = Box::new(5);
    fn a (b:&i32){
        println!("{}",b);
    }
    a(&bbox); // we expect that &Box can coerce to &i32
}

Having read Deref coercions in the book and the reference the process is basically applying .deref() until the obtained type fits the explicit type.

It seems the procedure –for this random case I used as an exercise– would be:

  1. Start with &bbox=&Box<i32> which is an &T.
    • Any &-type must implement Deref, so we can do things like *&T.
    • &T will do the standard method search and find that &Box is accepted by deref(&self) so it stops there (otherwise it would try with &&Box, &mut &Box, Box,...)
    • It applies the first &bbox.deref() to get &T which should be something like &** if I understand correctly. (first dereferences &self and second one gets to the data T.)
  2. Had we nested boxes to the i32 this would be (&**(&**)) repeatedly.
  3. We now have&T=&i32.
  4. That could still keep de-referencing, but since it matches the type annotation / signature, it stops.

Method search is used only when you write code containing the method call operator, .some_method_name(). It is not used in any other part of the language, including deref coercion. Your description should be solely in terms of (doing something equivalent to) inserting &** — the fact that one of those *s is implemented via Deref::deref() is irrelevant.

But isn't it used to find the actual .deref() method? That's what I meant.

No, it is not. None of the language syntax that invokes library functions uses method lookup; each one invokes a specific function of a specific trait.

If x: T, T implements Deref, and T is not a built-in pointer type, then *x is equivalent to *<T as core::ops::Deref>::deref(&x) (except that the core name is built into the compiler and not looked up in the current scope). There is no method lookup, only a call to a method of a specifically chosen trait implementation.

2 Likes

This is from std:

#[stable(feature = "rust1", since = "1.0.0")]
#[rustc_const_unstable(feature = "const_deref", issue = "88955")]
impl<T: ?Sized> const Deref for &T {
    type Target = T;

    #[rustc_diagnostic_item = "noop_method_deref"]
    fn deref(&self) -> &T {
        *self
    }
}

We know Box<i32>: Deref<Target = i32>.

So, &Box<i32>: Deref<Targte=i32>. That is to say, it's allowed to get &i32 from Box<i32>.


And said by the doc: Values of type &T are coerced to values of type &U.

So, &Box<i32> is coerced to &i32 when passed to a function where T is Box<i32> and U is i32.

I see, well I will need some time to adjust to it. Not sure I understand right now.

When I last dove into how to think about these things, I concluded that it made the most sense to make the following distinction:

  • auto-deref goes from T to *T
  • deref-coercion goes from &[mut] T to &[mut] *T
    • Note how we have to start with and end up with a reference
    • You can think of the coercion happening beneath the outer reference

With that distinction in place, we can note that:

  • field access and method resolution use auto-deref[1]
  • passing arguments[2] uses deref-coercion

This topic is about deref-coercion; nothing else I'll mention uses auto-deref.[3]


Let's apply deref-coercion to:

// I've added more layers
fn main() {
    let bbox: Arc<Box<i32>> = Arc::new(Box::new(5));
    fn a(b: &i32) {
        println!("{}",b);
    }
    a(&&&bbox);
//    ||||______Arc<Box<i32>>
//    |||______&Arc<Box<i32>>
//    ||______&&Arc<Box<i32>>
//    |______&&&Arc<Box<i32>>
}

The "search" for deref coercion is simple, because there is only one potential target type at each step, so it's just a linear search.

Want: & i32
Have: & &&Arc<Box<i32>>
        |__ This is the candidate for deref coercion, under the outer `&`

Candidate: &&Arc<Box<i32>>
- It's not i32.  Does it implement Deref? (it has at most one implementation)
- Yes -- and Target = &Arc<Box<i32>>

Candidate: &Arc<Box<i32>>
- Not i32, Deref? Yes, Target = Arc<Box<i32>>

Candidate: Arc<Box<i32>>
- Not i32, Deref? Yes, Target = Box<i32>

Candidate: Box<i32>
- Not i32, Deref? Yes, Target = i32

Candidate: i32
- Hey we found it

So the example ends up being transformed to something like

    a(&****&&bbox);
//    ||||||||______Arc<Box<i32>>
//    |||||||______&Arc<Box<i32>>
//    ||||||______&&Arc<Box<i32>>
//    |||||________&Arc<Box<i32>> (1st *)
//    ||||__________Arc<Box<i32>> (2nd *)
//    |||_______________Box<i32>  (3rd *)
//    ||____________________i32   (4th *)
//    |____________________&i32
We could further desugar things...

As @kpreid noted, there's no method lookup. Derefs on & and Box are builtin operations. The deref operation on the Arc calls its trait method.

    let _ = *bbox; // A deref operation on Arc notionally desugars to...
    let _ = * <Arc<Box<i32>> as Deref>::deref(&bbox);
    //      | |                               |__passes the Arc by reference
    //      | |__&Box<i32>, the return type of Arc::deref
    //      |_____Box<i32>, via (built-in!) deref of the reference

    // Back to the function call
    a(&* * <Arc<_> as Deref>::deref(&  **&&bbox  ));
//       |                          |  |___the Arc<Box<i32>>
//       |__________________________|__the parts added by deref on Arc

But I think approximately no one thinks in terms of the desugaring once they've gotten the hang of trait based derefs. Derefs are pretty much always a way to go "deeper" into some pointer or wrapper type.

I.e. if I even notice a deref-coercion, I think of it as a(&****&&bbox), not the more expanded form just above.

The main potential complication to the linear search is &mut versus &. But the search for a target type is still linear, because <T as DerefMut> is required to have the same Target as <T as Deref>. So this is effectively some extra check that it's legal to create a mutable place at each inserted deref.[4][5]


  1. and in the case of method resolution, other things like auto-ref ↩︎

  2. and satisfying let annotations, etc ↩︎

  3. or other parts of method resolution like auto-ref ↩︎

  4. be it created by built-in deref, or by calling DerefMut::deref_mut, which naturally must also be implemented ↩︎

  5. example ↩︎

So, after rereading your reply, I think I understand.

the core of the idea seems that .deref used is always that one directly for the type of x that you called T. No method search in other derived types. So I rather imagine necessary * added to initial type to match the type annotation, and under the hood normally calls the deref directly in the type.

However, some minor questions arise:

  1. Isn't Deref under std::ops::Deref rather than core::ops::Deref ?
  2. When I call my_type.deref() directly then it does use method search, or is it only for non-std traits?
  3. What if it is of &T i.e built in pointer type?

I would phrase that as the deref or Deref::deref used, as . to me implies the . "operator" used for method resolution (or field access). But I think you understand. Rust doesn't have type_of, but conceptually when * is not a built-in operation:

//               *x
//   ___________/  \_________________
// /                                 \
    * <type_of(x) as Deref>::deref(&x)
  1. It's in core, std just re-exports it. If you're looking here, click Source and note how you're looking at core. Or just replace std with core in the URL and you can see it exists.

  2. my_type.deref() uses method search, which you could confirm a number of ways, like making an inherent method called deref.

  3. And you call ref_t.deref() where ref_t: &T? Method search again.

It's rare to call .deref() via method call in practice. You'd have to import the Deref trait.

1 Like

But, what I'm still confused with is, how are these two scenarios so different:

Case 1 Assign an &Box to function parameter of type &i32

Case 2 Implementing Deref::deref in my own type and call mytype.deref()

(I read the auto deref v deref coercion comparison. But I expand below.)

Because, in the Case 1 it's as I was told with the qualified path, yet in the second here it is not.

So is there a rule somehow that they use the qualified path with as Deref for type coercions at coercions sites? (Including let assignments etc.) That is, instead of the method call which would trigger the method search which is somehow not used under the hood for type coercions (FQPath is used)

They’re the same in that "a deref() function was called”. They are different in many pieces of how you got there.

There is no actual qualified path involved. The Deref trait is specially known to the compiler,[1] and the compiler directly generates a call to the trait function without going through any concrete syntax to do so. The rule is “the compiler uses the Deref trait’s deref() function”. There are no smaller pieces to it.


  1. In the implementation this is known as a “lang item”, and denoted in the standard library source code with the attribute #[lang = "deref"]. ↩︎

Method resolution is only used when there's a .method() -- a call to a method after a .. Operators, deref-coercions, and candidate list building don't use them. I don't think anything[1] desugars to a method call.

Operators, including the deref operator, are either built in or depend on a specific trait that the compiler is aware of. By specific trait, I don't mean "go look up the name" or "translate source code to a qualified path". I mean that the trait is marked as a lang item,[2] so it's definitely using the correct trait.

So the * deref operator is always using the lang item Deref specifically (when not a built-in operation).

The scenarios are pretty different, which is why I suggest drawing a distinction between deref-coercion and auto-deref. That being said, both rely on using the specific, lang item Deref.

  • For deref-coercion, the search I described is performed via the lang item Deref

  • For method resolution, the initial set of candidates is built via the lang item Deref

    // Example: rta: &Box<[i32; 10]> and there's an rta.method() to resolve
    
    // "Initial candidates"
    &Box<[i32; 10]>    // type_of(rta)
     Box<[i32; 10]>    // type_of(*rta) -- <type_of(rta) as Deref>::Target
         [i32; 10]     // type_of(**rta)
         [i32]         // array unsizing (a special case)
    
    // Then auto-ref is inserted to complete the candidate list
    &Box<[i32; 10]>, &&Box<[i32; 10]>, &mut &Box<[i32; 10]>,
     Box<[i32; 10]>, & Box<[i32; 10]>, &mut  Box<[i32; 10]>,
         [i32; 10],  &     [i32; 10],  &mut      [i32; 10],
         [i32],      &     [i32],      &mut      [i32],
    

    Then the search for method starts, which depends on the receiver and which traits have been imported and so on. And if you had rta.deref(), the search for deref could potentially find something that wasn't the lang item Deref trait. But building the candidate list doesn't go looking things up based on what's in scope, etc. It just keeps applying lang item[3] Deref, type-wise, until it no longer can.


  1. besides macros ↩︎

  2. see also ↩︎

  3. or built-in ↩︎

Your answers are so challenging to read for me, but I appreciate how complete and well put they are; thanks.

1 Like

Is this more accurate?

  1. deref coercion, uses * to get to the target type.
    • Under the hood the deref operator * is Deref::deref(&self)
    • This trait is specifically labelled.
    • There is no candidate list as there is in .method calls.
    • The rule "the compiler uses the Deref trait's deref() function" applies.

In some cases like from &mut T to &T it must re-borrow again.

  1. autoderef, happens for method calls and field access expressions.
    • In the rare case of explicitly and manually calling.deref() as in
    impl Deref for MyType { fn deref(&self){/*...*/}}
    // somewhere
    my_type_instance.deref();
    
    • Standard method search happens, as shown by @quinedot. Unsure I get what the last lines in the reply mean.
    • When &my_type_instance is assigned to a parameter of type &inner_type then the local .deref() is used, but does not go through method search.

I'm unsure sp about the last bullet above.

Oh, something like &mut to & will use a different implementation of Deref. For that one specifically, the definition seems to be here.

This looks pretty accurate to me.

I tried to think of a way to present things more simply without losing too much detail (or saying something that isn't true). For deref coercion, what I came up with was to not think in terms of operators or desugaring directly, but in terms of types: give a variable t: T, either *t has a type or t cannot be dereferenced.[1]

Then for deref coercion we have a target type, and and expression with a type that we can attempt to coerce so that the types line up:

Target Type Expression Type Types we attempt Notes
&U &T &T, &*T, &**T, &***T... I'm abusing *T etc to mean "the type you get by dereferencing T"
&U &mut T &T, &*T, &**T, &***T...
&mut U &mut T &mut T, &mut *T, &mut **T, &mut ***T... Needs a follow-up check
&mut U &T (Not allowed)

We can see if the attempts on the right are compatible with the target type by only looking at the type of each attempt, and set aside exactly how it ends up translating to code. If we find a match, we'll just end up knowing that we need to insert so many dereferences.

(The follow-up check for a &mut _ target is: every derefence must correspond to a DerefMut implementor, not just a Deref implementor.)

And then if one wants, one could say "then you notionally end up with &**t", and talk about how that notionally desugars even further into built-in operations and function calls... but that could be a distraction, and I think your typical programmer just takes it for granted that the compiler will make the dereferences happen in a sensible manner.

Deref-coercion may be applied in this case. There's no method call so the whole method call resolution thing does not apply.

In a generic context, yes. In a non-generic context, &mut T to &T is another type of coercion, and if dereferencing or reborrowing is involved, it's a built-in operation.


  1. This does depend on the compiler knowing all of the lang item Deref implementations... and it is does. ↩︎